Evaluation of Health Canada's Transfer Payment to the Rick Hansen Foundation 2007-2008 to 2012-2013

Prepared by
Evaluation Directorate
Health Canada and the Public Health Agency of Canada

March 2014

(PDF Version - 262 K)

Table of Contents

List of Tables

Executive Summary

This evaluation covered Health Canada's transfer payment provided to the Rick Hansen Foundation (the Foundation/the recipient) for the period from 2007-2008 to 2012-2013. Health Canada no longer provides funding to the Foundation. This role has now been assumed by Western Economic Diversification Canada. The evaluation was undertaken in fulfillment of the requirements of the Financial Administration Act and the Treasury Board of Canada's Policy on Evaluation (2009).

Evaluation Purpose and Scope

The purpose of the evaluation was to assess the relevance and performance of Health Canada's transfer payment (grant) provided to the Foundation. In assessing the relevance and performance of the grant, the evaluation considered the five core evaluation issues, as identified by the Treasury Board's Policy on Evaluation.

The recipient conducted their own independent evaluation to fulfil the requirements of the Funding Agreement. This evaluation was conducted to establish the departmental accountability for the management of the transfer payment program and fulfil the policy requirements noted above. However, the recipient's independent evaluation report was used as a line of evidence in this evaluation, thereby reducing duplication of effort.

In view of the 2013 conclusion of Health Canada's transfer payment and independent reviews of performance available from the recipient, the evaluation focused on lessons learned and scaled down its methodological approach to a review of documents. In particular, this evaluation used evidence provided in the final independent evaluation (2012) of the Health Canada-funded activities. The independent evaluation used a range of methods similar to those used for federal evaluations, such as surveys, case studies and interviews.

Transfer Payment Program Description

Health Canada provided a total of $30 million to the Foundation in a conditional grant. The money was used by the recipient to carry out the areas of activities identified in the Funding Agreement. The funded activities involved the implementation of a spinal cord injury data system across Canada, support for spinal cord injury research, and promotion of best practices in spinal cord injury care, with the ultimate objective of providing healthcare and quality of life improvements for people with spinal cord injuries.

Conclusions - Relevance

There is a persistent upward trend of spinal cord injuries in Canada. Each year, there are over a thousand new cases of traumatic spinal cord injury, with currently approximately 86,000 people living with traumatic and non-traumatic spinal cord injuries in Canada. Traumatic injuries are the results of accidents, such as falls or car accidents, whereas non-traumatic injuries are the results of illnessesFootnote 1. Spinal cord injuries entail enormous human, social and economic burdens. For individuals with traumatic spinal cord injuries, there are staggering costs (of care, necessary support and lost opportunities) over their lifetime (see Section 4.1). This indicates that there is a continued need to support initiatives to improve the healthcare and quality of life for people with spinal cord injuries.

Health Canada's transfer payment program was aligned with the health and safety priorities of the Government of Canada and Health Canada. Throughout the funding period, the Government continued to extend its support for the health and safety of Canadians as seen in the Speeches from the Throne. The funding program was also aligned with the strategic priorities of Health Canada for improving the health of Canadians and working with partners to strengthen the efficiency and effectiveness of the publicly-funded healthcare system.

The funding program was aligned with the federal roles and responsibilities to promote the health of all Canadians, as well as the federal interest to leverage federal investment by fostering partnerships with other sectors of the economy.

Conclusions - Performance

Achievement of Expected Outcomes (Effectiveness)

The recipient implemented a spinal cord injury registry data system (hereafterreferred to asthe Spinal Core Injury (SCI) Registryor the Registry) in 31 facilities across Canada, over the past nine years. The presence of the Registry across Canada demonstrates the achievements of the recipient in engaging and gaining support from key players in the spinal cord injury care community. Furthermore, there was a shared recognition in the community of the recipient's efforts for facilitating collaborative works to improve the knowledge and care of spinal cord injury.

As to the effectiveness of the recipient's initiatives for increasing spinal cord injury knowledge and promoting best practices in spinal cord injury care, there were mixed views and observations. In parts of the community, particularly among Registry site representatives, there was more awareness and use of best practices promoted by the recipient, participation in training activities to implement new knowledge, and optimistic expectations on future impacts of the Registry and related research and care activities. In other parts of the community, there was less awareness and adoption of best practices and new knowledge, less participation in training exercises, along with mixed views on the benefits of the recipient's initiatives. The recipient made a significant contribution to the development of the national standards for SCI care and services with Accreditation Canada. In other areas of public policy, the recipient's influence was seen to be less visible.

Demonstration of Economy and Efficiency

There were indications that the federal investments on the recipient's initiatives were leveraged to some extent. For the most part, however, the recipient relied on funding support from governments. Over the grant period, funding from the federal and provincial/territorial governments accounted for about 70% of the recipient's revenues.

Health Canada identified provisions and tools intended to ensure the sound management of the transfer payments. Throughout the transfer payment period, however, Health Canada had challenges monitoring the funded activities, due in part to the differences in reporting structures of both parties. In the final year of the transfer payment period, Health Canada and the recipient found mutually agreeable solutions that met the reporting requirements. However, the process of managing these challenges took a considerable amount of time and effort by both parties

Lessons Learned

Given that Health Canada is no longer in a funding relationship with the recipient, this evaluation does not provide recommendations for ongoing management of the fund. Instead, the evaluation provides lessons learned.

While the recipient made progress toward the achievement of the expected outcomes, closer alignment between the funding objectives and the available timeframe may have facilitated a greater likelihood that the expected outcomes would have been achieved. Specifically, the achievement of the expected outcomes within the planned timeframe as stated in the Funding Agreement was unrealistic, as it would have entailed extensive changes across the SCI care system which takes time. In addition, the timeline (by 2015) prescribed for the achievement of the outcomes was outside the transfer payment period. Health Canada experienced challenges monitoring the funded activities in part due to the differences in the recipient's reporting systems and the departmental reporting requirements. The process of managing these challenges took a considerable amount of time and effort by both parties. With these observations, the evaluation identified the following lessons learned:

  1. Expected outputs and immediate outcomes or when possible, longer-term outcomes (eg when is continued funding that had already been in place for prior funding period or when realistic to expect to see them) should be achievable and observable within the timeframe of funding provided; and
  2. The department should focus on proactive program management by identifying potential challenges and mitigation strategies early, including those associated with the recipient's reporting structures and the departmental requirements.

1.0 Evaluation Purpose

The purpose of the evaluation was to assess the relevance and performance of the transfer payment program which provided $30 million to the Rick Hansen Foundation in support of its initiatives for spinal cord injury (SCI) research and treatment for the period of 2007-2008 to 2012-2013. The evaluation was conducted to fulfil the Financial Administration Act and the Treasury Board of Canada's Policy on Evaluation (2009) to conduct a departmental evaluation of all ongoing grant and contribution programs every five years. The recipient conducted their own independent evaluation to fulfil the requirements of the Funding Agreement. This evaluation was conducted to establish the departmental accountability for the management of the transfer payment program and fulfil the policy requirements noted above. However, the recipient's independent evaluation report was used as a line of evidence in this evaluation, thereby reducing duplication of effort.

2.0 Transfer Payment Program Description

2.1 Program Context

In 2007, following the 20th anniversary of Rick Hansen's Man in Motion World Tour, the Prime Minister announced funding support to be provided for the Foundation in its search for a cure for spinal cord injuries. Health Canada provided a grant of $30 million to the Foundation for a period of five years, from 2007-2008 to 2011-2012. The funding was to support the Foundation's initiatives, known as the Spinal Cord Injury Translational Research Network. The initiatives of the SCI Translational Research Network aimed to facilitate collaboration among scientific and clinical researchers and support translational research.

The Spinal Cord Injury Translational Research Network was established in 2003 with federal investment under Western Economic Diversification Canada's initiatives. Further investment was initiated with Health Canada's transfer payment program, which was implemented for five years (2007-2008 to 2011-12). The transfer payment program was then extended for one year, with no additional funding, to allow the recipient to spend the surplus funds and fulfil the delivery of funded activities. On March 31, 2013, Health Canada's transfer payment program was concluded. In 2013, the recipient received funding from Western Economic Diversification Canada for another five years ($35 million for a period of five years beginning in 2013-2014).

2.2 Program Profile

The Health Canada transfer payment program involved the transfer of a conditional grant of $30 million to the Foundation and the administration and monitoring of the Funding Agreement according to the Treasury Board's policy requirements for transfer payments. The entire $30 million was transferred to the Foundation upon the signing of the Funding Agreement in 2007. In accordance with the terms and conditions established in the Funding Agreement, the recipient oversaw the selection and implementation of funded projects. The funded activities of the recipient were governed by a Board of Directors.

The money was used by the recipient to carry out the areas of activities identified in the Funding Agreement. The funded activities evolved over time, with the organizational changes of the recipient. In the first three years (from March 2007 to December 2009), funding was directed toward five areas of activities of the Spinal Cord Injury Translational Research Network which focused on: (a) activities to reduce the incidence of permanent paralysis; (b) quality of life improvements; (c) knowledge transfer; (d) the development of a national knowledge and technology network, including the expansion of the SCI Registry; and (e) infrastructure for innovation and management. In the following years (from December 2009 to March 2013), funding was directed toward activities reorganized under the Spinal Cord Injury Solutions Network.

In 2009, the SCI Solutions Network changed its name to the Rick Hansen Institute (the Institute), which was incorporated as an independent entity. The Funding Agreement was subsequently amended to reflect these changes(see Note). The amended program objectives include:

  1. a significant reduction in the incidence and severity of permanent paralysis resulting from Spinal Cord Injury (SCI) by 2015;
  2. a significant increase in restoration of physical function following SCI by 2015;
  3. a significant reduction in the incidence and severity of secondary complications associated with SCI by 2015;
  4. a significant increase in level of satisfaction with quality of life and community participation among people with SCI by 2015;
  5. ensuring that customized responses to priority unmet needs are available to 100% of individuals with SCI throughout their journey to full participation by 2015; and
  6. establishing a world class Canadian SCI registry and data management platform by 2012.

Note: For this evaluation, the funded activities carried out by the Institute were not separately identified from those carried out by the Foundation. All funded activities were accounted for as those of the Foundation, as the Funding Agreement was with the Foundation, and the distinction between the two had little substantive bearing for the evaluation. In the subsequent sections, the recipient refers to both the Foundation and the Institute unless distinguished otherwise.

2.3 Program Logic Model and Narrative

Long-term expected outcomes

The long-term outcomes focused on health and quality of life improvements for people living with SCI disabilities.

Activities and outputs

The recipient carried out its work in five areas. Key activities and outputs are described as follows:

SCI Registry: This area of activity involved developing a Canada-wide data system (the SCI Registry) which compiled SCI cases from all sectors of SCI treatment and services, including acute care hospitals, rehabilitation and community-integration service organizations. The data captured a range of SCI care activities and outcomes, such as incidence, severity, interventions/ treatments and their successes. In order for the Registry to be comprehensive, the Foundation had to forge partnerships with healthcare organizations across the country and engage key players in the SCI care systems, including those in acute care hospitals and rehabilitation centres, researchers, service providers, as well as people with SCI. Major activities associated with the Registry included: providing member organizations with data management tools, training and consultations; facilitating the use of data; and providing support for research and multi-centre trials.

Translational research: This area of activity involved the translation of basic-science discoveries or clinical trials to every day care-practice ("the lab-to-bedside" translation). The recipient's translational research involved: identifying and translating SCI research discoveries into practical applications; developing best-practice guidelines for care areas; and supporting clinical trials. The activities related to translational research involved the engagement of researchers and the facilitation of SCI knowledge translation, in the form of therapies, protocols, practices, policies and other measures.

Best practices: This area involved promoting the adoption of best practices identified through the recipient's Translational Research Program, among care and service providers as well as people with SCI. The recipient promoted best practices through a variety of activities such as education, information, advocacy, social marketing, and development of communities of practice.

Community partnerships: This area involved forging alliances with key players in the SCI community, such as hospitals, community and educational institutions, government policy makers, and service providers. The recipient promoted a specific approach (known as the Solutions Model) to support people with SCI, from injury to community re-integration.

Network leadership and coordination: This activity area involved providing leadership and coordination for improving SCI resources (e.g., funding and supporting) and networking. The specific activities included securing funding and support from partners and identifying networking opportunities with key stakeholders in the SCI community.

Immediate outcomes

The immediate outcomes of the funded initiatives focused on: building engagement and support of stakeholders in the implementation of the national data system (the SCI Registry); knowledge creation; coordination and collaboration across the innovation spectrum; and building network capacity for knowledge translation.

Intermediate outcomes

The intermediate outcomes focused on: increased access to SCI research and practices, and improved collection, management and use of evidence by all key system actors; adoption of technology infrastructures; adoption of validated practices; supportive public policy for the adoption of best practices; enhanced customized response to needs of people with SCI; and increased community participation of people with SCI and reduction in community and system burden.

Reach of the program

The reach of the funded initiatives covered the entire SCI community across Canada, including: acute care hospitals, rehabilitation centres, researchers, clinicians, universities, hospitals, local solution team members, policy makers, healthcare practitioners, service providers, SCI organizations, funding agencies, private companies, donors, as well as people with SCI and their families.

The connection between these activity areas and the expected outcomes is shown in the Logic Model (see Appendix 2). The evaluation assessed the extent to which the expected outcomes were achieved over the evaluation timeframe.

2.4 Program Alignment and Resources

The transfer payment program was part of Health Canada's Program Alignment Architecture (PAA) Strategic Outcome 1: A health system responsive to the needs of Canadians, and sub-program PAA 1.1.1: Canadian health system priorities. The sub activity provides leadership for health system priorities that benefit Canadians through improved access to quality healthcare services. A total of $30 million was provided up-front in a conditional grant to cover activities from 2007-2008 to 2012-2013. Conditional grants are funding provided for a specific purpose that may not be used for any other purposes.

3.0 Evaluation Description

3.1 Evaluation Scope, Approach and Design

The scope of this evaluation covered the period from April 1, 2007 to March 31, 2013, and included all of the funded activities described in Section 2.3.

The issues that were examined in this evaluation were aligned with the Treasury Board's Policy on Evaluation and considered the five core issues under the two themes of relevance and performance, as shown in Appendix 3. Corresponding to each of the core issues, specific questions were developed based on program considerations and these guided the evaluation.

A non-experimental, outcome-based evaluation approach was used for the evaluation to assess the progress made towards the achievement of the expected outcomes and the identification of lessons learned. A key source for the evaluation was the recipient's independent evaluation, which covered the period from 2007-2008 to 2011-2012 with a focus on the period after the mid-point review from 2010 onwardsFootnote 2. The evaluation does not cover the last year (2012-2013) of the transfer payment period. The independent evaluation used a range of methods similar to those used for federal evaluations, such as surveys, case studies and interviews. It thereby provided a window of observations, for this evaluation, on multiple lines of evidence.

Given that Health Canada is no longer in a funding relationship with the recipient, and independent reviews of the recipient were recently conducted, this evaluation focused on lessons learned and scaled down its methodological approach to a review of documents.

3.2 Limitations and Mitigation Strategies

The table below outlines the major limitations of this approach along with the key mitigation strategies.

Table 1: Limitations and Mitigation Strategies
Limitation Impact Mitigation Strategy
Limited primary data collection. No direct input from ultimate recipients on program impacts. Reviewed a broad range of documents in efforts to triangulate data sources to increase the validity of the evaluation findings.
Reliance on progress reviews and evaluation documents provided by third parties. The third-party documents may not have the necessary data or may have methodological limitations that would allow a full assessment of the proposed evaluation questions. All documents provided a discussion of limitations. These limitations, as well as missing data, were considered when the analysis was performed.
Data in the evaluation documents by third parties were limited to the first five years (2007-2008 to 2011-2012) of the funding period. The outcomes of activities concluded in the final year of the funding period are not assessed in this evaluation. The absence of data for the final year was recognized. This recognition was considered when the analysis was performed.

4.0 Findings

4.1 Relevance: Issue #1 - Continued Need for the Program

There is a continued need to address issues concerning spinal cord injury in view of its increasing prevalence and economic burden.

A recent study of Canadian cases of spinal cord injury estimated that each year there were about 1,400 new cases of traumatic spinal cord injuries.Footnote 3 This data has been used as a reference for the annual incidences of traumatic spinal cord injury in Canada.Footnote 4 Another study of SCI in Manitoba identified a rising trend of SCI. The study indicated that the number of (traumatic and non-traumatic) cases of spinal cord injury doubled over the past two decades (1981-1985 to 2003-2007).Footnote 5 The document review confirms that there appears to be a persistent upward trend of spinal cord injuries in Canada and, currently, there are about 86,000 people living with spinal cord injuries in Canada.

Spinal cord injuries entail significant human, social and economic burden. Spinal cord injuries require substantial medical care involving hospitalization, rehabilitation, long-term care and often involve secondary complications. In addition to health care costs, there are considerable economic and social costs as a significant number of people with SCI are unemployed or underemployed and many are in need of income support as well as social and personal support.

A recent economic study Footnote 6 provided estimates for the lifetime costs of living with traumatic spinal cord injury. The study factored in a wide range of direct and indirect costs, such as acute care, rehabilitation, attendant care, prescription drugs, adaptive equipment, and home modification. Based on the estimates of this study, a case of traumatic injury at the age of 35 years would incur $1.5 million to $3 million in the lifetime costs, depending on the severity of injury. With the annual average of 1,400 new cases of traumatic injury, the total economic burden in Canada is estimated at $3 billion annually. From the document review, the annual direct costs of traumatic spinal cord injury patients are significantly higher than the annual direct costs of managing stroke patients with paralysis on a per patient basis.

The document review indicated that the number of people with spinal cord injuries will likely reach 121,000 by 2030. Spinal cord injuries are also expected to generate a substantial economic burden on the healthcare system in the future.

4.2 Relevance: Issue #2 - Alignment with Government Priorities

The transfer payment program was aligned with the Government of Canada's ongoing commitment to protecting the health and safety of Canadians, as well as the priorities of Health Canada. The transfer payment program was aligned with the Government's economic priorities for supporting leading-edge research.

The protection of the health and safety of Canadians is an ongoing commitment of the Government of Canada. This commitment was seen throughout the Speeches from the Throne during the period of the transfer payment program. In 2006, in the Speech from the Throne, the Government confirmed its commitment to protecting the health of Canadians by extending its support for innovative healthcare delivery and improving the Canada's healthcare system. In the Speeches from the Throne in 2008 to 2010, the Government communicated its broader objective for the well-being of Canadians; and in the Speeches from the Throne in 2011 to 2013, the Government articulated its specific objective to protect the health and safety of Canadians and their families. The transfer payment program, with its objectives to improve the quality of life for people living with spinal cord injuries, was aligned with the ongoing commitments of the Government.

Likewise, the transfer payment program was aligned with the priorities of Health Canada. In 2007 when the transfer payment program was introduced, Health Canada identified four strategic priorities, two of which were concerned with contributing to improvement of the health of Canadians; and working with others to strengthen the efficiency and effectiveness of the publicly-funded healthcare system. With the recipient's objectives to expand the knowledge base for SCI care by working with partners from across the SCI community and thereby improve patient outcomes, the transfer payment program was aligned with the priorities of Health Canada.

In addition to Health Canada's priorities, the transfer payment program was intended to support the economic objectives of the Government as identified in Advantage Canada: Building a Strong Economy for Canadians (2006). Advantage Canada is a long-term, national economic plan which commits the Government to: work with partners to enhance the excellence of Canadian research; help people enter the labour market and reach their full potential; and create the social and economic conditions under which all Canadians can improve their quality of life.

As indicated earlier, it is estimated that about 86,000 Canadians are living with SCIFootnote 7. A significant number of them are unemployed or underemployed, and many live with economic hardships. With the recipient's long-term outcomes for reducing the human, social and economic costs of SCI, the transfer payment program was aligned with the economic objectives of the Government.

4.3 Relevance: Issue #3 - Alignment with Federal Roles and Responsibilities

The transfer payment program was aligned with the legislated mandate of Health Canada to promote the health of Canadians. The transfer payment program was consistent with the federal leadership role for fostering collaboration with stakeholders for health promotion.

The transfer payment program was aligned with the legislated mandate of Health Canada to promote and preserve the health of Canadians.The objectives of the transfer payment program were to reduce disabilities from spinal cord injuries and provide health and quality of life improvements for people with spinal cord injuries. These objectives were aligned with the overarching objectives of the Health Canada Act which states that "the primary objective of Canadian healthcare policy is to protect, promote and restore the physical and mental well-being of residents of Canada and to facilitate reasonable access to health services without financial or other barriers." Similar healthcare objectives are embedded in the Department of Health Act which sets powers, duties and functions of the Minister of Health.

The federal government has a role to support health promotion and health research initiatives that encompass all provinces and territories. The transfer payment program supported the initiatives that were national in scope, over which the federal government had both authority and influence. The funded initiatives aimed to develop a data system that would operate across Canada and a national network that drew key players in the SCI community, including researchers, clinicians and SCI patients. The initiatives also aimed to facilitate the translation of research discoveries to practical applications and promote best practices across the SCI care continuum and the SCI community across Canada.

4.4 Performance: Issue #4 - Achievement of Expected Outcomes (Effectiveness)

4.4.1 To what extent have the immediate outcomes been achieved?

The funded initiatives worked toward several immediate outcomes, as shown in the Logic Model (Appendix 2). Findings on the achievement of these outcomes are summarized with the following highlights.

The recipient has successfully engaged key players in the SCI community and gained their support for the implementation of the SCI Registry.

The recipient sponsored the implementation of the SCI Registry in Vancouver in 2003. Since then, the recipient has expanded the Registry across Canada. In 2013, the Registry was operational in 31 facilities across 15 cities. This was considered as a major achievement of the recipient, as it would enable other expected outcomes to materialize.

The Registry was expected to serve as the single integrated database for all SCI cases in Canada. The Registry collected SCI incidence data through the acute, rehabilitation and post-injury phases of patient care. The data from the Registry was used for SCI research and treatment. The recipient engaged key players (such as care practitioners, patients, and researchers) in the SCI care system in the implementation of the Registry. The set up and ongoing expansion and operations of the Registry entails a lot of activities on the part of the recipient, such as soliciting participation of health organizations in the Registry, negotiations with participating organizations, and increasing awareness of the Registry among patients.

After nine years of implementation, the Registry covered about 6% of the SCI cases in Canada at the time of the final independent evaluation. Based on the document review, this level of coverage was considered comparable to the coverage achieved by a similar data system in the United States (National Spinal Cord Injury Database) where it took 20 years to reach a 13% coverage. Based on the information provided by the recipient, the Registry is intended to cover new SCI incidences at participating sites; and the Registry coverage reached approximately 50% of new SCI incidences in Canada over nine years. The patient coverage of the Registry is expected to increase as the data system matures. The presence of the Registry across Canada demonstrates the recipient's achievements in engaging key stakeholders and garnering their support. The table below (Table 2) shows the results of a stakeholder survey in which the level of support for the Registry among practitioners and the Registry sites was strong.

Table 2: Support for the SCI Registry
Question Response Total surveyed
With respect to the idea of collecting information on SCI in a registry, for the purposes of research and support to local program administration and practices:      
To what extent do you agree? Somewhat or totally 96% 110-221 Practitioners
To what extent do key clinicians in your centre agree? Somewhat or totally 96% 24-27 Centres
To what extent do decision-makers in your centre agree? Somewhat or totally 100% 24-27 Centres
Source: Survey of practitioners and representatives of Registry centres. Compiled from Exhibit C6.2 and C6.3 in Appendix C of the Summative Evaluation of Rick Hansen Institute Funded by Health Canada (hereafter, the Independent Evaluation Report) (2012).

Note: The source report indicated the total respondents in a range as they varied from question to question.

The recipient has achieved a measure of success in engaging SCI researchers and gaining their support for the recipient's research initiative.

The recipient engaged SCI researchers through the implementation of the SCI Registry as well as through research grant projects. Between 2007 and 2012, the recipient funded approximately 62 research projects which engaged approximately 450 researchers in a variety of activities, such as clinical trials, development and implementation of best practices, translational research, and networks of clinicians and researchers for SCI knowledge development. Some examples of the projects that engaged researchers were:

The recipient also engaged other stakeholders in the SCI community, and ensured a balanced representation of diverse and relevant interests in its governance structures, as this was an important stepping stone for the recipient's initiatives. Based on the interviews of key informants in the independent evaluation, there was perceptible recognition among stakeholders of the work done by the recipient in this area. About 52% of informants provided a positive response on the recipient's engagement of: health administrators, practitioners and service providers in treatment and rehabilitation centres; policy makers in government and healthcare organizations; and people living with SCI. To a lesser extent, key informants (42%) also had a positive view on the engagement of people working with SCI patients in local communitiesFootnote 8. Likewise, there were representations with diverse and relevant interests and perspectives on SCI in the recipient's governing bodies including senior management, Board of Directors, and the Translational Research Advisory Committee. Most respondents confirmed that there was a balance of interests and perspectives in the recipient's networks of SCI researchers and practitioners.

There was a widely shared view among stakeholders that coordination and collaboration across the SCI community improved over the last five years.

Surveys in the independent evaluation demonstrated that the views of key players in the SCI community were generally positive on the progress made in coordination and collaboration across the SCI community. According to practitionersFootnote 9, there was more coordination of patient care across the continuum of care, particularly from acute care to rehabilitation. Practitioners noted improved coordination from rehabilitation to community integration, and more collaboration between health professionals. Likewise, the representatives of the Registry sites indicated that there was more coordination from acute care to rehabilitation and from rehabilitation to community integration, as well as more collaboration with other professionals.

As to collaboration between researchers and practitioners, the overall perception was generally positive with some differing viewsFootnote 10. Most researchers (92%) indicated that there had been more collaboration compared to five years ago, whereas 70% of practitioners echoed this view. Experts positively identified the improvement made in cross-sector coordination and collaboration over the past five years. It is however difficult to determine, with certainty, the changes that are directly attributable to the recipient's initiatives, as SCI care processes are complex and involve diverse actors carrying out inter-related activities in a dynamic environment.

The funded projects covered areas of SCI research. SCI knowledge building was expected to continue after 2013 as research projects would come to fruition.

The projects funded by the recipient through Health Canada's transfer payment were expected to produce new knowledge in the form of validated science, approaches, therapies, protocols, policies, practices, and measures. The recipient's data showed that the projects have generated a diverse range of knowledge products. They included academic articles, standards, guidelines and conference presentations, as shown in Table 3.

Table 3: Knowledge products generated by the funded projects
Type of knowledge product No. of outputs No. of projects involved in generating this output
Peer-reviewed articles published 37 16
Standards and guidelines produced 3 2
Conference presentations 143 34
Abstracts 20 8
Participants involved 3,420 30
Source: Exhibit C6.10 in Appendix C of the Independent Evaluation Report

The contributions made by some of the funded projects were highlighted in the document review, along with recognition from SCI experts. Below are some of their contributions.

The implementation of the SCI Registry stands as the best concrete achievement for knowledge development and increasing the capacity of the knowledge base for SCI.

The recipient worked toward building the 'capacity' (networks and related resources) to support translational research and knowledge development related to the whole continuum of SCI care. This involved the implementation of the Registry and building networks of key players in the SCI community to facilitate SCI knowledge translation and development. The Registry data management system was used to support multi-centre trials and research. Based on the document review, the implementation of the Registry was regarded among stakeholders as the recipient's significant achievement for building the capacity for knowledge translation. In other areas, the initiatives were not consistently effective. In a broad measureFootnote 11, they indicated that: the initiatives were effective in building the network capacity for knowledge translation in rehabilitation and care for secondary complications; but the initiatives were less effective in building the capacity for knowledge translation in acute care, community integration, and restoring physical function. There were mixed views on and limited recognition of the achievement of the initiatives for building capacity for meeting the priority needs of SCI patients with enhanced responses.

4.4.2 To what extent have the intermediate outcomes been achieved?

The successful implementation of the funded activities is expected to induce changes to the SCI community and behaviours of those involved in the systems. The Logic Model (Appendix 2) identifies several areas of changes which are expected in an intermediate timeframe. The progress made is summarized with the following highlights.

The use of best-practices and best-practice products varied considerably across the care continuum and from practice to practice and from product to product. There was some indication that SCI healthcare practitioners were adopting new assessment tools or therapies in their practices.

The recipient facilitated the development and promotion of best-practice knowledge and knowledge products, such as Spinal Cord Injury Rehabilitation Evidence, Physical Activity Guidelines for Adults with Spinal Cord Injury, Pressure Ulcer Risk Assessment, and Acute Spinal Cord Injury: State-of-the-Art Reviews. The recipient also promoted the use of the Registry data. Target users of the Registry data and best-practice knowledge and products were practitioners in acute care, rehabilitation and community integration. Surveys of practitioners found that about 75% of 215 practitioners surveyed had awareness of SCI best-practice documents or information. However, there was a significant difference between the care sectors. For example, the level of awareness in rehabilitation and community integration (88% were aware) was considerably higher than the level of awareness in acute care (62% were aware).

The document review also noted the under-utilization of best practices. The level of use of best-practice products by practitioners was found to be considerably lower than the level of awareness. For example, among rehabilitation practitioners, 55% used best practice documents in rehabilitation; and among acute care practitioners, 48% used best practices documents in their care area. In community integration, the use of best practice documents was lower at 14%.Footnote 12

The level of benefits seen by practitioners was found to be less than the level of use or awareness. In rehabilitation where most of the best-practice products were targeted, they were seen beneficial to only 41% of practitioners. In other sectors, the level of perceived benefits was even lower. In acute care, 32% of practitioners indicated that the best practices were beneficial while only 12% of practitioners in community integration were able to see the benefits of adopting best practices.

The surveys of practitioners (see Table 4) showed training activities that had taken place to implement new SCI knowledge. It indicated that about two in three practitioners received training on new assessment tools or new treatment techniques. Most, however, did not identify the recipient as the main source of their training. For new treatment techniques, only 18% identified the recipient as the main source of their training; for new assessment tools, about one third received training offered by the recipient. The surveys also showed a moderate level of adopting new knowledge among practitioners. For example, more than half of practitioners indicated that they adopted new assessment tools or treatment techniques. Most, however, did not identify the recipient as the main source for the new knowledge they adopted. For new assessment tools, 30% identified the recipient as their main source of information whereas, for new treatment techniques, only 21% stated that they acquired half or more of their information from the recipient.

Table 4: Adoption of new SCI knowledge
Adoption of new SCI knowledge Response %
Yes
# of Respondents
Assessment tools or measures
In the last five years, I have received training on assessment tools or measures for patient with SCI Somewhat
or totally
63% 225-241
What proportion of this training was offered by the RHI? Half or more 35% 87-128
In the last five years, I have implemented new assessment tools or measures in my practice with patients with SCI. Somewhat
or totally
50% 225-241
What proportion of these new approaches was acquired using information disseminated through RHI? Half or more 30% 87-128
Therapies, practices or techniques
In the last five years, I have received training on therapies, practices or techniques for patients with SCI. Somewhat
or totally
68% 225-241
What proportion of this training was offered by the RHI? Half or more 18% 87-128
In the last five years, I have implemented new therapies, practices or techniques in my practice with patients with SCI. Somewhat
or totally
72% 225-241
What proportion of these new approaches was acquired using information disseminated through RHI? Half or more 21% 87-128
Source: Compiled from Exhibit C 7.10 and C7.11 in Appendix C of the Independent Evaluation Report.

Among the Registry sites and rehabilitation sector, there were consistent signs of progress. There was more active knowledge training and implementation, and wider recognition of the recipient as the major conduit.

In the Registry sites, there were more consistent signs of progress and optimistic expectations on future impacts. All representatives of the Registry sites active in rehabilitation and community integration indicated their sites had the awareness of best practice documents (100%). Among the Registry sites active in acute care, 73% of site representatives indicated that there was awareness of best practice documents in their sites (see Table 5). The Registry data was used actively among half (55%) of the Registry sites, and most sites (96%) expected that their use of the Registry data would reach a significant level within five years, as shown in the data below. In most sites, however, the use of the Registry data has remained limited as data collection has not fully matured.

Table 5: Use of the Registry data and awareness of best practice documents in the Registry centres
Awareness of best practice documents
and use of the Registry data
Response % (Yes) Number of centres
Awareness of best practice documents
Centres active in rehab. and community integration   100% 13
Acute care centres   73% 11
All centres   88% 24
Use of the Registry data
To what extent has your centre used data from the Registry so far? Moderately or substantially or
Totally
55% 22
How likely do you think it is that your centre will use data from the Registry within five years? Moderately or substantially or
Totally
96% 22
Source: Compiled from Exhibit C 7.5 and C7.9 in Appendix C of the Independent Evaluation Report based on surveys of the Registry site representatives.

Training on new assessment and treatment among those involved in the Registry sites was more active. In more than 80% of the Registry sites, there was a recognizable level of training activities and the level of implementation activities was high. In addition, the recipient was identified as the main source of their new knowledge. Specifically, 75% of the Registry sites implemented new assessment tools (somewhat or totally) and the recipient was the main source for the new tools. Likewise, 69% of the sites implemented new treatment techniques and in most cases, the recipient's information was the main source for new techniquesFootnote 13.

Overall, there was indication that training on and adoption of SCI best practices was taking place across the SCI community, with the recipient's greatest influence among SCI Registry sites.

The recipient successfully partnered with Accreditation Canada to develop national standards for SCI care and services. In other areas of public policy, the recipient's influence was seen as less visible.

There was a widely shared view that the recipient made a contribution to embedding evidence-based best practices for SCI care and services in the standards of excellence, by working with Accreditation Canada. Accreditation Canada has more than 1,200 member organizations across Canada. They include hospitals, walk-in clinics, labs, emergency medical services, long-term care homes, home care services, community health programs, among others. The development of SCI care standards with Accreditation Canada is regarded as the significant influence that the recipient has made besides the development of the Registry. In other areas of public policy, the recipient's influence was seen as less visible. From the document review, about half (48%) of the key informants interviewed did not think that the recipient made an influential contribution to supportive public policy. This view was also observed among the experts interviewed.

Among key players, the funded initiatives were seen as responsive to the needs of people with spinal cord injuries. However, there are differing views in the SCI community on improvements made in responding to the needs of people with SCI more generally.

Surveys of practitioners and the Registry site representatives showed that many key players felt that the funded projects were responsive to the needs of SCI patients (70% to 96%). No direct input was available from people with spinal cord injuries on the funded initiatives.

In the independent evaluation report, 40% of key informants had the view that the recipient's initiatives enhanced overall responsiveness to the needs of those with SCI, while roughly another 40% did not. With respect to meeting treatment needs, the majority of practitioners were positive about improvements made over the past five years. For example, 81% of practitioners provided a positive response about improvements made in patient outcomes of acute treatment.

A survey of the SCI community, sponsored by the recipient in conjunction with other organizations, identified six major needs of people with spinal cord injuries: equipment, medical supply, communication devices and technical aids; general healthcare; accessible housing; short distance transportation; SCI-specialized healthcare; and long-distance transportation. For the most part, these needs were outside the scope of the funded initiatives. As the survey suggests, in addition to treatment needs, people with SCI have unique economic and social needs. The findings of the SCI community survey indicated that many unmet needs were related to income support, leisure and employment participation. These needs were not only outside the scope of SCI healthcare but also outside the focus of the funded initiatives.

4.4.3 To what extent have the long term outcomes been achieved?

The successful achievement of the immediate and intermediate outcomes of the funded initiatives is expected to lead to the achievement of the ultimate outcomes. The Funding Agreement established six ultimate outcomes expected of the funded initiatives. They were: reduction in permanent paralysis; restoration of physical function; reduction in secondary complications; improvement in quality of life; enhanced responses to unmet needs; and the implementation of the Canadian SCI registry data system (the SCI Registry).

The introduction of the SCI Registry data system across Canada was seen successful. The other five outcomes were expected to be achieved by 2015 under the Funding Agreement. These outcomes have not fully materialized within the timeframe of this evaluation. These outcomes focus on the improvement of SCI patient outcomes, and involve system-wide changes and more complex challenges, all of which would take a considerable amount of time to achieve. While it would be unrealistic to expect the achievement of the outcomes by 2015, the funded activities were expected to make progress toward the stated objectives. Based on the independent evaluation report, the evaluation explored the views of stakeholders on the progress made toward some of the defined outcomes.

There was a strong recognition of the recipient's contribution to the reduction in secondary complications and increased restoration of physical function following SCI. On the whole, the recipient's initiatives were well received by providers of SCI care and services while the impacts as experienced by people with SCI remain unknown.

Stakeholders provided positive views on the contribution of the recipient's initiatives. In particular, there was a strong recognition of the recipient's contribution to the reduction in secondary complications. About 76% of practitioners acknowledged the recipient's contribution to this area, and a majority of researchers (88%) and Registry site representatives (71%) echoed this view. Likewise, there was significant acknowledgement, across all three groups of stakeholders, of the recipient's contribution to increased restoration of physical function following SCI. As to the reduction of permanent paralysis, about 50% of stakeholders positively acknowledged the recipient's contribution.

Among those who were working with the SCI care system (i.e., practitioners, researchers and the Registry site representatives), there was strong recognition of the recipient's contribution toward improving the quality of life for people with SCIFootnote 14. In the community outside the SCI care system, on the other hand, there was limited recognition of the recipient's contribution to improving responsiveness to unmet needs of people with SCI and increasing their community participation.

4.5 Performance: Issue #5 - Demonstration of Economy and Efficiency

The Treasury Board of Canada's Policy on Evaluation (2009) and guidance document, Assessing Program Resource Utilization When Evaluating Federal Programs (2013), defines the demonstration of economy and efficiency as an assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes. This assessment is based on the assumption that departments have standardized performance measurement systems and that financial systems link information about program costs to specific inputs, activities, outputs and expected results.

In general, the data structure of the financial information provided for program evaluation does not facilitate the assessment of whether program outputs are produced efficiently, or whether expected outcomes are produced economically. Specifically, the lack of output/outcome-specific costing data limits the ability to use cost-comparative approaches. Considering these issues, this evaluation carried out a limited scope of a study, which included the leveraging of federal funds by the recipient and efficiency of program management.

The recipient positively leveraged the value of federal contributions, including Health Canada's transfer payment, to support its activities. The recipient, however, relied significantly on government funding to carry out its activities.

As a charity organization registered with the Canada Revenue Agency, the Rick Hansen Foundation files the Registered Charity Information Return (T3010) annually in which charities report on their activities, sources of revenue and expenditures. The following was observed in a review of the Foundation's revenue information filed with the Canada Revenue Agency (see Table 6)Footnote 15.

Between 2007-2008 and 2012-2013, the Foundation reported $124 million in total revenues. Health Canada's transfer payment of $30 million represented 24% of the total revenues. In addition to this fund, the Foundation received federal contributions totalling approximately $20 million, mostly from Western Economic Diversification Canada. In total, the federal government provided approximately $50 million, which accounted for 40% of the Foundation's total revenues over this period. The ratio of non-federal funds ($74 million) over federal funds was 1.48. Therefore, for each federal dollar, an additional $1.48 was drawn from non-federal sources. Of the non-federal sources, provincial/territorial governments were the largest funding contributors. Their contributions totalled $38 million, which accounted for about 30% of the Foundation's total revenues.

Table 6: Health Canada's transfer payment and other sources of revenue
Sources of revenue Revenues ($000) % of total revenues Note
Health Canada transfer payment $30,000 24%  
Other federal funds (Western Economic Diversification and others) 20,009 16%  
Total federal funds 50,009 40%  
Provincial/territorial governments 38,125 31% (1)
Total government funds 88,134 71%  
Gifts and fundraising 23,829 19% (2)
Other sources (e.g., investment incomes) 12,061 10%  
Total revenues $124,024 100% (3)
Source: Registered Charity Information Return (T3010) data published by the Canada Revenue Agency; and Public Accounts of Canada (Section 6: Transfer Payments)

Note:  Figures were rounded

  • Revenues combined from entries in Canada Revenue Agency (T3010) codes 4550
  • Revenues combined from entries in Canada Revenue Agency (T3010) codes 4500, 4530 and 4630
  • Revenues combined from Canada Revenue Agency (T3010) code 4700

Over the funding period, the funding contributions from the federal/provincial/territorial governments totalled approximately $88 million, which accounted for 71% of the Foundation's funding sources (see Table 6). In addition to government funds, the Foundation raised $24 million through gifts and fundraising campaigns. The Foundation raised $0.27 from non-government sources for each dollar of government funding.

The provisions ensuring the sound stewardship of Health Canada's transfer payment to the recipient were in place. The Funding Agreement had provisions consistent with the statutory and policy requirements for transfer payment. Some of the key provisions include: annual work plans including budgets, objectives and expected outcomes; annual reports with audited financial statements; a third-party progress report within two years of the funding; a third-party evaluation within five years; compliance and performance audits; and investment provisions ensuring the prudent management of an upfront multi-year fund.

Likewise, program management tools were in place at Health Canada to manage the transfer payment. The Health Canada transfer payment program developed a monitoring framework that identified the recipient's reporting requirements and processes. For the most part, monitoring focussed on reviewing annual reports against annual work plans, and comparing those to the overall objectives and activities outlined in the Funding Agreement. Health Canada however had challenges monitoring the funded activities due in part to the differences in reporting structures of both parties.

Particularly, during the early years of the transfer payment period, it was difficult for Health Canada to identify the operational and financial activities specific to the Health Canada funding in part due to the differences in the recipient's financial and project management systems and the Health Canada's reporting structures, and also in part due to lack of clear distinctions across different funding sources in the recipient's narrative reporting. Health Canada found that the recipient's budgeted activities in the annual work plans at times did not clearly align with activities described in the annual reports. As a result, the transfer payment program had challenges identifying activities and expenditures specific to the Health Canada funding, and assessing the achievement of the recipient's activities. The recipient, on the other hand, had its own challenge in meeting the reporting requirements. The recipient found that the transfer payment program reporting requirements did not align well with its reporting structures. In the last year of the funding period, Health Canada and the recipient found mutually agreeable solutions that appropriately met the reporting requirements.

The independent performance audit, provided by the recipient to fulfil a requirement of the Funding Agreement, identified that the recipient had appropriate systems and practices in place to support compliance with the terms of the Agreement, and to ensure efficiency and economy in the implementation of funded activities. The audit covered the period from January 1, 2010 to September 30, 2011. This period was a transition period in which the Rick Hansen Institute became operational with its revised activities for the funded program. The audit rendered, in general, a positive assessment on the funded program, such as effective organizational oversight, transparent approval and monitoring of projects and grants, performance measurement and reporting, and financial monitoring.

Over the funding program period, both parties invested a considerable amount of time, resources and effort to meet the reporting requirements of the Funding Agreement. At the outset of the Funding Agreement, it might have been beneficial for both parties to review reporting/monitoring requirements to identify potential challenges and mitigation strategies in relation to the recipient's organizational and reporting structure.

5.0 Conclusions

5.1 Relevance

Ultimate beneficiaries of this transfer payment program are people with spinal cord injuries in Canada. Currently nearly 86,000 people are living with spinal cord injuries, and each year more than a thousand people experience spinal cord injuries. Spinal cord injuries have serious consequences on individuals; and care and support entail enormous social and economic burdens. No national-scale program exists that is dedicated to improving the healthcare and quality of life for people with spinal cord injuries, other than the initiatives of the Rick Hansen Foundation/Rick Hansen Institute. The transfer payment program supported the national initiatives intended to provide health and quality of life improvements for Canadians with spinal cord injuries, which was aligned with the role of the Government of Canada as well as the strategic objectives of Health Canada to protect the health and safety of Canadians.

5.2 Performance

5.2.1 Achievement of Expected Outcomes (Effectiveness)

On the whole, the recipient's initiatives funded by Health Canada were well received by the SCI community. The recipient achieved a measure of success in introducing a national data system for spinal cord injury (the SCI Registry) across Canada. The implementation of the data system continues to this date; and there is a shared expectation in the SCI community that the registry data system upon its full implementation will help enhance SCI research and treatment. The recipient also carried out a wide range of activities promoting collaboration across the SCI care continuum and evidence-based best practices in the care and community integration of people with spinal cord injuries. There was shared recognition in the SCI community of the recipient's efforts for building collaboration among practitioners and researchers alike, and improving SCI knowledge and care.

The recipient's initiatives were envisioned to induce significant changes in the SCI care systems in Canada, which would take a considerable length of time. While there is shared recognition in the community of the potential benefits of the funded initiatives, the expected outcomes have not fully materialized within the timeframe of the evaluation. It would be unrealistic to expect the achievement of these outcomes within the planned timeframe as these outcomes would entail extensive changes across the SCI care system, which would take a considerable length of time.

5.2.2 Demonstration of Economy and Efficiency

Health Canada's grant was provided with an expectation that the recipient would leverage the grant to raise additional funds. The federal investments on the recipient's initiatives were leveraged to some extent. For the most part, the Foundation relied significantly on government funding. Over the grant period, federal/provincial/territorial governments together financed about 70% of the Foundation's revenues.

Throughout the funding period, Health Canada had challenges monitoring the funded activities, as the recipient's reporting at times was not aligned with the stated requirements. The challenges arose in part due to the structural differences in the financial reporting systems of both parties. In the end, Health Canada and the recipient worked together and found solutions that met the reporting requirements.  The process of managing the challenges has taken a considerable amount of  time and effort of both parties. The challenges experienced by Health Canada underscore a need to enhance program management strategies so as to allow the program to determine progress and ensure accountability.

6.0 Lessons Learned

Given that Health Canada is no longer responsible for the transfer payment program for the recipient, this evaluation does not provide recommendations. Instead, the evaluation provides lessons learned to inform similar federal programming

The achievement of the expected outcomes within the planned timeframe as stated in the Funding Agreement was unrealistic, as it would entail extensive changes across the SCI care system which takes time. In addition, the timeline (by 2015) prescribed for the achievement of the outcomes was outside the transfer payment period.

As mentioned above, the process of managing the reporting challenges took a considerable amount of time and effort by both parties.

With these observations, the evaluation identified the following lessons learned:

  1. Expected outputs and immediate outcomes or when possible, longer-term outcomes (eg when is continued funding that had already been in place for prior funding period or when realistic to expect to see them) should be achievable and observable within the timeframe of funding provided; and
  2. The department should focus on proactive program management by identifying potential challenges and mitigation strategies early, including those associated with the recipient's reporting structures and the departmental requirements.

Appendix 1 - References

  1. Canada Revenue Agency. Registered Charity Information Return (T3010): Rick Hansen Foundation. 2007-2008 through to 2012-2013. Retrieved from Charities and Giving, Canada Revenue Agency: http://www.cra-arc.gc.ca
  2. Finance Canada (2006). Advantage Canada: Building a Strong Economy for Canadians. Retrieved from: http://www.fin.gc.ca/ec2006/pdf/plane.pdf
  3. Government of Canada (2013). News Releases (June 05, 2013): Harper Government Advanced Knowledge and Treatment of spinal Cord Injury. Retrieved from: http://www.wd.gc.ca
  4. Government of Canada (1985). Canada Health Act(R.S.C., 1985, c. C-6). Retrieved from: http://laws-lois.justice.gc.ca/eng/acts/c-6/
  5. Government of Canada (1996). Department of Health Act (S.C. 1996, c. 8). Retrieved from: http://laws-lois.justice.gc.ca/eng/acts/H-3.2
  6. Government of Canada. Speech from the Throne. From the first session of the 39th Parliament through to the second session of the 41st parliament. Retrieved from: http://www.pco-bcp.gc.ca/index.asp?lang=eng&page=wp-pa&doc=sft-ddt_e.htm
  7. Health Canada (2007). Funding Agreement: Rick Hansen Foundation Spinal Cord Injury Translational Research Network.
  8. Health Canada (2009). Amended Funding Agreement (2007).
  9. Health Canada. The grant program management files, including Monitoring Framework.
  10. Health Canada (2007). Report on Plans and Priorities 200702008. Retrieved from: https://www.tbs-sct.gc.ca/rpp/2007-2008/hlth-sant/hlth-santtb-eng.asp
  11. Krueger. H et al. (2013). "The economic burden of traumatic spinal cord injury in Canada," Chronic Diseases and Injuries in Canada, Vol.33, No.3, June 2013. Retrieved from: http://www.phac-aspc.gc.ca/publicat/cdic-mcbc/33-3/ar-01-eng.php
  12. Rick Hansen Institute (2012). Summative Evaluation of the Rick Hansen Institute Initiatives Funded by Health Canada, prepared by Circum Network Inc.
  13. Rick Hansen Institute (2010). Report of the Independent Midpoint Review of Health Canada Funding to the Rick Hansen Institute, prepared by Performance Management Network Inc.
  14. Rick Hansen Institute. Annual Report Fiscal Year 2013.
  15. Rick Hansen Institute. Health Canada Appendices to the RHI Annual Report 2011-2012 and 2012-2013.
  16. Rick Hansen Institute. Financial Statements 2011, 2012 and 201.
  17. Rick Hansen Institute (2012). Rick Hansen Institute Performance Audit, prepared by Price Waterhouse Cooper (PWC).
  18. Rick Hansen Foundation. Annual Report 2010, 2011, 2012, and 2013. Retrieved from http://www.rickhansen.com/language/en-CA/Who-We-Are/About-the-Foundation/Annual-Report-Financial-Information.aspx
  19. Rick Hansen Foundation. Audited Financial Statements 2010, 2011, 2012, and 2013. Retrieved from http://www.rickhansen.com/language/en-CA/Who-We-Are/About-the-Foundation/Annual-Report-Financial-Information.aspx
  20. Public Works and Government Services Canada. Public Accounts of Canada. Volume III. Section 6: Transfer Payments. 2007-2008 through to 2012-2013. Retrieved from Library and Archives Canada: http://epe.lac-bac.gc.ca/100/201/301/public_accounts_can/pdf/index.html

Appendix 2 - Logic Model

Table 7: Health Canada's Transfer Payment to the Rick Hansen Foundation Logic Model
Logic Model
Components
Stream 1 Stream 2 Stream 3 Stream 4 Stream 5
Activities SCI Registry/Information Platform

Strategy 4: Collecting and analyzing data on SCI in Canada, including acute, rehabilitation and community components, and supporting research and multi-centre trials, utilizing a web-based national technology platform and collaborating globally where appropriate.

Primarily attributed to Health Canada Funding
Translational Research

Strategy 1: Development and validation of best practice guidelines for emergency response, treatment and access to primary health care.

Strategy 2: Supporting multi-centre clinical trials in acute care, rehabilitation and community and fostering global collaboration where possible; and supporting promising pre-clinical research imminently ready for translation.

Primarily attributed to HC Funding
Best Practices

Strategy 5: Facilitating the adoption and implementation of validated best practices as identified by the Translational Research Program and through properly evaluated public policy and community based programs aimed at improving treatment, care and support and communicating these best practices directly to Canadians with SCI.

Primarily attributed to Health Canada Funding
Community Partnerships

Strategy 6: Working with RHI partners to enhance service delivery through full implementation of the Solutions Model (solutions tream, navigators).

Primarily attributed to Health Canada Funding
Network Leadership and Coordination

Strategy 8: Providing excellence in leadership coordination and program support at the national and international level.

Primarily attributed to Health Canada Funding
  Strategy 3: Supporting the development of a "Best and Brightest" program to nurture new SCI researchers.

Primarily attributable to Western Economic Diversification funding

  Strategy 7: Working with RHI partners to enhance capacity and develop initiatives that respond to the priority needs of people with SCI, utilizing collaborative funding mechanisms such as Wheels in Motion (WIN), provincial/territorial 20th Anniversary investments, etc.

Primarily attributable to Western Economic Diversification funding

 
Outputs SCI Registry: Registry data reports and business cases.

Primarily attributed to Health Canada Funding

Translational Research: Translational research projects.

Primarily attributed to Health Canada Funding

Best Practices: Completed projects and early outputs of ongoing projects (previously 14).

Primarily attributed to Health Canada Funding

Community Partnerships: Grants.

Attributable to a combination of federal and other funding

Network Leadership and Coordination: Network Management Strategies Partnership development.

Primarily attributed to Health Canada Funding

Reach Acute care, hospitals, rehab centres, researchers, clinicians, people with SCI.

Primarily attributed to Health Canada Funding

Researchers, clinicians, universities, hospitals, rehab centres, local solution team members, and people with SCI.

Primarily attributed to Health Canada Funding

Researchers, clinicians, policy makers, practitioners, service providers, universities, hospitals, rehab centres, people with SCI.

Primarily attributed to Health Canada Funding

Researchers, clinicians, policy makers, practitioners, service providers, universities, hospitals, rehab centres, people with SCI.

Attributable to a combination of federal and other funding

SCI organizations, government, foundations and granting agencies, corporate and private donors, employees.

Primarily attributed to Health Canada Funding

Immediate Outcomes Engagement and support of Deputy Minister and Registry by key system actors such as acute care hospitals, rehab centres, researchers, clinicians, and people with SCI.

Primarily attributed to Health Canada Funding

Engagement and support of key researchers in Working Groups,

E-scans, white papers, reviews (literature, Delphi, systematic) and evaluation Request for Applications, Request for Proposals, clinical trials, universities, hospitals, rehab centres, local solutions teams, people with SCI.

Primarily attributed to Health Canada Funding
Engagement of Canadians with SCI, key institutions (hospitals, rehab centres, universities), policy makers, health administrators, practitioners and service providers; and those engaged with Working Groups,

E-scans, white papers, reviews (literature, Delphi, systematic), Request for Applications, Request for Proposals, clinical trials, and networking functions, etc.

Attributable to a combination of federal and other funding
Engagement and support of key stakeholders in networking functions, including governance

Primarily attributed to Health Canada Funding

Engagement and support of potential funding partners such as government, foundations and granting agencies, corporate and private donors.

Attributable to a combination of federal and other funding

Validated science, approaches, therapies, protocols, policies, practices and measures (knowledger creation).

Primarily attributed to Health Canada Funding

Cooperation, coordination and collaboration across the innovation spectrum.

Network capacity built for knowledge translation.

Primarily attributed to Health Canada Funding
Intermediate Outcomes Health Canada#6 - Increased access to SCI research and practices, and improved collection, management and use of evidence by all key system actors

Primarily attributed to Health Canada Funding

Adoption of technology infrastructure/standards/measures (e.g. Spinal Cord Injury Rehabilitation Evidence) enabling improved innovation system performance

Primarily attributed to Health Canada Funding

Supportive public policy for the adoption of good (best) practices.

Primarily attributed to Health Canada Funding

    Adoption and adaptation of validated approaches, therapies, protocols, policies, practices, measures, and the necessary supports and services by care and service providers and use of these by consumers.

Primarily attributed to Health Canada Funding

    Enhanced customized response to priority unmet needs available to Canadians with SCI.

Increased community participation among Canadians with SCI (active living education, employment).

Primarily attributed to Health Canada Funding
Optimized personal, community and system burden.

Primarily attributed to Health Canada Funding

Ultimate Outcomes Disability minimized for people with SCI:

Health Canada #1 - Reduced incidence and severity of permanent paralysis.

Health Canada #2 - Increased recovery of function following injury

Health Canada #3 - Reduced incidence and severity of secondary complication associated with SCI.

Primarily attributed to Health Canada Funding
  Immproved (Mazimized) quality of life for those with SCI:

Health Canada #4 - Increased level of satisfaction with quality of life and community participation among people with SCI.

Health Canada #5 - Increased level of customized responses to priority unmet needs available to 100% of individuals with SCI (throughout their journey to full participation).

Attributable to a combination of federal and other funding

Appendix 3 - Evaluation Description

Evaluation Scope

The scope of the evaluation covered the period from April 1, 2007 to March 31, 2013, and included review of the achievement of the immediate, intermediate and ultimate outcomes, through the implementation of activities, as outlined in the Logic Model (Appendix 2).

Evaluation Issues

The specific evaluation questions used in this evaluation were based on the five core evaluation issues prescribed in the Treasury Board of Canada's Policy on Evaluation (2009). These are noted in the table below. Corresponding to each of the core issues, evaluation questions were tailored to the program and guided the evaluation process.

Table 1: Core Evaluation Issues and Questions
Core Issues Evaluation Questions
Relevance
Issue #1: Continued Need for Program Assessment of the extent to which the program continues to address a demonstrable need and is responsive to the needs of Canadians
  • Was the program responsive to the needs of people with SCI?
  • Is there a continued need for the program?
Issue #2: Alignment with Government Priorities Assessment of the linkages between program objectives and (i) federal government priorities and (ii) departmental strategic outcomes
  • Was the program aligned with government priorities and departmental strategic outcomes?
Issue #3: Alignment with Federal Roles and Responsibilities Assessment of the role and responsibilities for the federal government in delivering the program
  • Was the program aligned with federal roles and responsibilities?
Performance (effectiveness, economy and efficiency)
Issue #4: Achievement of Expected Outcomes (Effectiveness) Assessment of progress toward expected outcomes (incl. immediate, intermediate and ultimate outcomes) with reference to performance targets and program reach, program design, including the linkage and contribution of outputs to outcomes
  • To what extent has the program been effective in achieving the expected outcomes?
Issue #5: Demonstration of Economy and Efficiency Assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes
  • To what extent has the program been efficient in drawing external investments to support the funded activities?
  • To what extent has the program been efficiently managed to help the funded activities achieve the expected results?

List of Acronyms

PAA
Program Alignment Architecture
RHI
Rick Hansen Institute
SCI
Spinal Cord Injury

Page details

Date modified: