Evaluation of the Contribution to Brain Canada Foundation’s Canada Brain Research Fund 2011-12 to 2015-16
January 2017
Table of Contents
- Executive Summary
- Management Response
- 1.0 Evaluation Purpose
- 2.0 Program Description
- 3.0 Evaluation Description
- 4.0 Findings
- 4.1 Relevance: Issue #1 – Continued Need for the Program
- 4.2 Relevance: Issue #2 – Alignment with Government Priorities
- 4.3 Relevance: Issue #3 – Alignment with Federal Roles and Responsibilities
- 4.4 Performance: Issue #4 – Achievement of Expected Outcomes (Effectiveness)
- 4.5 Performance: Issue #5 – Demonstration of Economy and Efficiency
- 5.0 Conclusions
- 6.0 Recommendation
- Appendix 1 – Logic Model
- Appendix 2 – Organizations and Research Background by Project
- Appendix 3 – Summary of Findings
List of Tables and Figures
- Figure 1: CBRF funding commitments according to funding mechanism category
- Table 1: Program Financial Data
- Table 2: Core Evaluation Issues and Questions
- Table 3: Limitations and Mitigation Strategies
- Table 4: Cost estimates of the five diagnostic categories with the highest total costs, Canada, 2008
- Table 5: Comparison between CIHR and CBRF selected funding mechanisms
- Table 6: Thematic breakdown of CBRF projects
- Figure 2: Breakdown of non-federal funding
- Table 7: Amounts received by funder category and size of donations 2011-2015
- Table 8: MIRI projects expected to conclude by year and program
- Table 9: Funding Model Environment Scan
- Table 10: Number of LOIs received, full applications and funded projects by research program.
List of Acronyms
- ABI
- Acquired brain injury
- ALS
- Amyotrophic lateral sclerosis
- BCF
- Brain Canada Foundation
- CBRAIN
- Canadian Brain Imaging Research Platform
- CBRF
- Canada Brain Research Fund
- CIHI
- Canadian Institute for Health Information
- CIHR
- Canadian Institutes of Health Research
- CQDM
- Consortium Québécois sur le développement des médicaments
- EIC
- Experimental Imaging Centre
- IDRC
- International Development Research Centre
- LOI
- Letter of Intent
- MIRI
- Multi-Investigator Research Initiative
- MRgFUS
- Magnetic resonance guided focused ultrasound surgery
- PHAC
- Public Health Agency of Canada
- PSG
- Platform Support Grants
- RPC
- Research Policy Committee
- SAC
- Science Advisory Council
- SPB
- Strategic Policy Branch
- WHO
- World Health Organization
Executive Summary
The evaluation of the Contribution to Brain Canada Foundation’s Canada Brain Research Fund (CBRF) was undertaken in fulfillment of the requirements of the Financial Administration Act.
Evaluation Purpose and Scope
The purpose of the evaluation was to assess the relevance and performance of the contribution to the Brain Canada Foundation’s CBRF for the period ranging from March 2012 to March 2016. In addition, the evaluation reviewed the funding model currently in place.
Program Description
The 2011 federal budget included an allocation of $100 million to support the establishment of Brain Canada Foundation’s CBRF. This federal commitment was supplemented by an additional $20 million which was announced in Budget 2016. To access the full $120 million, Brain Canada Foundation (BCF) must raise the equivalent amount through non-federal donation sources in order to trigger a 1:1 match by Health Canada.
The objective of CBRF is to accelerate discoveries by supporting research that advances knowledge of the brain and promises development in the diagnosis of, and therapies associated with brain conditions. Funding supports the full spectrum of brain and mental health research, with a focus on higher risk projects through innovative collaborations that are conducted with the hopes of producing higher rewards. Research funding is provided through three mechanisms: the Multi-Investigator Research Initiatives (MIRIs); Platform Support Grants (PSGs); and training awards.
The MIRIs support multidisciplinary teams and aim to accelerate novel and transformative research that will change the understanding of nervous system function and dysfunction.
The PSGs are intended to facilitate and accelerate research by funding the maintenance and operation of major existing research platforms, providing national or regional technical capability to multiple neuroscience investigators.
To promote the next generation of researchers, the training awards are granted to outstanding doctoral students and postdoctoral fellows undertaking original research related to the origins and consequences of brain diseases and conditions.
Conclusions - Relevance
Neurological conditions have been estimated to affect approximately 3.6 million Canadians and are the most costly of health conditions when accounting for combined direct and indirect costs. The number of individuals living with neurological conditions and the cost of associated care are expected to rise as the Canadian population ages.
The brain has been cited as the last frontier of health research. Key informants and document review sources confirm that there is a continued need for research to better understand how the brain functions. Not-for-profit organizations specific to health research involving the brain have been created in disease-specific silos (e.g., the Alzheimer’s Society, the MS Society), and as a result there have been limitations in the linking of findings across diseases impacting the brain. A whole-of-brain approach to research in this field is expected to help identify similar degenerative pathways linked to a variety of neurological diseases.
A focus on neuroscience research is consistent with departmental priorities and has been outlined as a federal priority throughout the period captured in this evaluation. The CBRF’s innovative approach in working towards generating discoveries contributes to current federal science priorities. Although some overlap exists, for the most part, key informants agreed that the work of CIHR and CBRF are more complementary than duplicative.
Conclusions - Performance
While it is still too early to assess the long-term outcomes resulting from the CBRF, it is evident that MIRI grants and PSGs facilitate collaboration within the brain community and across disciplines. This collaborative and multidisciplinary approach is seen as promising in researching common mechanisms presenting across neurological and psychiatric diseases. Currently, the portfolio of projects funded appear to be well balanced in terms of translational, pre-clinical and clinical research.
An assessment of intermediate outcome impacts would be more appropriate once the intended outcome timeframe of 7-15 years has passed. At this point in time, anecdotal evidence suggests that some projects demonstrate potential for downstream impacts, including new discoveries and the use of new therapies, interventions, diagnostics, tools and products stemming from CBRF research discoveries.
To date, CBRF’s private-public funding partnership model has facilitated the incorporation of new sources of funding within the research field that are typically not engaged by federally funded research organizations. However, the operationalization of the funding model could be improved by moving away from requiring donations be transferred to the CBRF and verified through audited financial statements in order to receive the matched federal funding. A shift to a less onerous approach has the potential to better support the objective of the CBRF and build on BCF’s capacity to maintain relationships with partners interested in increasing overall investments in brain research. Mechanisms are in place to continually assess projects’ progress and their use of funds. In terms of administrative efficiencies, the letter of intent (LOI) peer review process for CBRF has served to streamline the application and project selection processes.
Recommendation
Health Canada should consider adjusting the CBRF Funding Model.
While the CBRF funding model has clearly resulted in increased funds being dedicated to brain research in Canada, it still faces challenges including:
- Capturing administrative costs associated with the management of all funding and its impact on funding available for matching.
- Requirement of audited financial statements prior to initiating federal funding match which results in delays to spending matched funding.
- Operationalization of the flow-through funding model that includes requirements to physically transfer funds to BCF’s account in order to trigger a federal funding match has impacted the ability of not-for-profits and provinces to donate money. Not-for-profits do not typically transfer funds to intermediaries due to lack of control of disbursement of donated funds, and provincial funders require funding to stay within the province.
To address these challenges Health Canada should continue its examination of the funding model and make necessary adjustments that ensure it meets the needs of both BCF and Health Canada.
Management Response and Action Plan
Recommendations | Response | Action Plan | Deliverables | Expected Completion Date | Accountability | Resources |
---|---|---|---|---|---|---|
Recommendation as stated in the evaluation report | Identify whether program management agrees, agrees with conditions, or disagrees with the recommendation, and why | Identify what action(s) program management will take to address the recommendation | Identify key deliverables |
Identify timeline for implementation of each deliverable | Identify Senior Management and Executive (DG and ADM level) accountable for the implementation of each deliverable | Describe the human and/or financial resources required to complete recommendation, including the source of resources (additional vs. existing budget) |
Health Canada should consider adjusting the Canada Brain Research Fund Program’s funding model. | Health Canada program management agrees with the recommendation | Summer 2016 - Health Canada consulted with other government departments with a matched funding model. Program officials also consulted extensively with the Recipient and internally with Legal Services, CFOB and other functional areas. Fall 2016 - A Treasury Board submission seeking approval for modifications to the funding model was approved. Feb/Mar 2017 –A new funding agreement will be put in place which will modify the funding model. |
New Funding Agreement | March 31, 2017 | Executive Director Health Programs and Strategic Initiatives (HPSI) |
This recommendation will be completed using existing SPB human and financial resources. |
1.0 Evaluation Purpose
The purpose of the evaluation was to assess the relevance, achievement of outcomes, as well as the economy and efficiency of the CBRF, which was partially established through a federal contribution. The scope of the evaluation includes CBRF activities during the period of 2011-2012 to 2015-2016. This evaluation also examined the funding model developed for the Fund.
This was a scheduled evaluation as per the Public Health Agency of Canada/Health Canada Five-Year Evaluation Plan 2015-2016 to 2019-2020, in accordance with the requirements outlined in the Financial Administration Act.
2.0 Program Description
2.1 Program Context
In Budget 2011, the Government of Canada announced the establishment of the CBRF, which was formalized in 2012 when Health Canada entered into a contribution agreement with the BCF. Through this Agreement, currently managed by Health Canada’s Strategic Policy Branch (SPB), the federal government initially committed up to $100 million to fund the BCF’s CBRF. It was later supplemented with an additional $20 million through the 2016 federal budget. Access to the full amount of federal funding is determined by BCF’s ability to raise equivalent funds from non-federal governmental sources, based on a 1:1 matching model.
The initial contribution agreement was amended in July 2015, updating the contribution agreement’s end date to March 31, 2020. This extension was sought to allow for the matching of private funds raised towards the end of the original cycle.
BCF, a registered charity first established as NeuroScience Canada in 1998, manages the CBRF. Between 2003 and 2011, the organization operated without funding from the Government of Canada, relying mostly on philanthropy. NeuroScience Canada/ BCF recognized an underfunding of neuroresearch. BCF advocated for focusing research activities on better understanding the functioning of the brain as a whole, believing it would likely lead to a greater understanding of common mechanisms across various conditions and diseases affecting the brain.
2.2 Program Profile
Acting as a focal point for non-federal investments in brain research, the objective of CBRF activities is to accelerate discoveries by supporting research that advances knowledge of the brain and promises development in the diagnosis of, and therapies aimed at, improving the health and quality of life for Canadians who suffer from brain disorders. CBRF funding supports the full spectrum of brain and mental health research. Funding is primarily intended to support higher risk projects conducted through innovative collaborations between relevant disciplines including physical science, applied and social sciences1. Research funding is provided through three distinct mechanisms: Multi-Investigator Research Initiatives (MIRI), PSG and training awards. During the years covered by the evaluation, the BCF launched 24 calls for proposals within these funding mechanisms. Figure 1 outlines the percentage of funds committed according to the CBRF funding mechanism category.
Figure 1: CBRF funding commitments according to funding mechanism category
-
Figure 1 - Long Description
This figure depicts a pie graph outlining the percentage of funds committed according to the CBRF funding mechanism category. The graph shows that as of December 2015, 74% of CBRF funding committed was for Multi-Investigator Research Initiatives (MIRI) projects, while 24% was for Platform Support Grants (PSG) with the remaining 2% for training awards.
1. Multi-Investigator Research Initiatives
MIRI supports multidisciplinary teams and aims to accelerate novel and transformative research that will fundamentally change the understanding of nervous system function and dysfunction and its impact on health.2
Seventeen calls for proposals were associated with the MIRI mechanism, dating back to 2012. Proposals can address any aspect of nervous system function or dysfunction that has relevance to neurological and/or psychiatric diseases and conditions, including those of the special sensesFootnote i and the peripheral nervous system. The focus of proposals must be on investigating commonalities among multiple or related neurological and/or psychiatric conditions. Applicants’ proposals are assessed by an international review committee made-up of scientists, clinicians, and researchers from various fields. They assess the strength of proposals by benchmarking them against global standards of excellence and innovation, while considering the proposed project’s likelihood of producing novel and transformative nervous system function and dysfunction research findings. The MIRI calls for proposals yielded 82 projects,Footnote ii,3 of which five concluded in 2015. The remaining project completion dates fall outside of the evaluation period: 61 research projects will conclude between 2016-18, and the remaining 16 will finish in 2019-20. Total grants awarded within this category varied from $100,000 to $9,996,0874 per project, with most exceeding $1 million. To date, $115,403,973 has been committed to MIRI projects, accounting for 74% of CBRF funding committedFootnote iii (as of December 2015). During the 2011-15 period, $35,335,3805 was disbursed to fund MIRI projects.
2. Platform Support Grants
PSGs are intended to facilitate and accelerate research by filling a funding gap for operations and maintenance of major research platforms providing national or regional technical capability to multiple neuroscience investigators. PSGs are not intended to replace current sources of funding. Examples of PSGs could include brain banks, imaging facilities, data repositories and data-sharing systems.
Three calls for proposals took place under the PSG stream, with the first call launched in 2014. All three calls yielded a total of 22 platforms. These projects, which are all still in progress, accounted for 24% ($37,353,711) of funding committed up until December 2015. The PSG award values varied from $138,750 to $6,105,000. One project received their funding in 2014, 19 in 2015, and an additional two in 2016. A total of $5,063,8406 was disbursed for PSGs during the 2011-2015 period.
3. Training Awards
Training Awards enable the next generation of young researchers to receive high-quality training under the direction of leading Canadian researchers. The awards are granted to outstanding doctoral students and postdoctoral fellows undertaking a program of research training that incorporates original research exploring the origins and consequences of neurological and/or psychiatric diseases and conditions.
Four calls for proposals took place under the Training Award mechanism yielding 34 awardsFootnote iv, of which 10 concluded in 2015, 22 will conclude in 2016/2017, and an additional two in 2019/2020. Less than three percent ($3,947,500) of the funding committed up until December 2015 was allocated to 34 training awards. The award value for students and fellows varied from $5,000 to $195,000. During the 2011-15 period, a total of $1,584,058 was disbursed for Training Awards.
Types of Funding Competitions
CBRF offers two different types of funding competitions: sponsored and partnered. Sponsored competitions are developed by the Foundation, whereas Partnered Initiatives are developed in partnership with donors.
All competitions begin as ‘open calls’, meaning scientists in any field of study can submit an application if their research matches the eligibility criteria particular to that competition.7 Applications are assessed primarily on the basis of their scientific merit and innovation potential, although other criteria such as feasibility can be used depending on the program. Applications are also screened for potential short and long-term impacts, meaning research that can change fundamental understandings of the brain and nervous system.
Governance
BCF, which is responsible for the administration of the CBRF, is governed by a board of elected directors responsible for supervising the management of the activities and affairs of the Foundation. The Board of Directors is currently comprised of 16 members with nine honourary members8. While the mandate of the Board is broad, members’ key responsibilities are to: advance the interests of stakeholders and the overall organization; review recommendations of the Research Policy Committee (RPC) with respect to BCF’s areas of science focus and priority; as well as assist with fundraising and maintaining relationships with key program and funding partners, including government and voluntary health organizations.9
The RPC10 is a committee of the BCF board which can include members from the Canadian philanthropist community, corporate executives and scientists. They offer advice and perspectives to the Board of Directors at large on the research funded through the CBRF. More specifically, the RPC advises the Board on matters such as the goals of the research funding programs and progress towards the achievement of those goals; the competition and review processes; post-award reporting; and approval of ongoing funding.
The RPC is also responsible for making recommendations to the Board to seek input from the Science Advisory Council (SAC) on emerging scientific issues and opportunities. This Council, which is comprised of 23 members originating primarily from Canada and the United States, provides the RPC with intelligence about developments in the field, within Canada and internationally, and provides a broad assessment of the CBRF’s research programs. Activities of the SAC enable the RPC to assess whether changes to the program should be made in order to ensure that it is appropriately flexible and responsive to the science. They also monitor the achievement of the intended outcomes and impacts11.
Other internal committees reporting to the Board and guiding BCF operations include:
- The Audit, Finance and Investment Committee responsible for reviewing financial statements and recommending approvals to the Board;
- The Governance, Nominating and Ethics Committee provides oversight in matters related to governance, nomination of people to serve as directors and committee members, as well as ensures the highest ethical standards for all staff; and
- The Public Policy and Communications Committee provides advice/reports on public policy and communications including positioning, branding and government relations.
The BCF governance structure was awarded a Conference Board National Award in Governance in 2006.12 Beyond the creation of the SAC in 2013, the current BCF governance structure remains largely unchanged. This award celebrates “innovations and best practices in governance in the private, public and not-for-profit sectors”.13
2.3 Program Narrative
This is the first evaluation to examine activities of the CBRF. In consultation with the SPB and BCF, the existing and detailed logic model developed by the BCF was simplified to guide the evaluation (Appendix 1).
Activity areas and outputs are related to funding and overseeing MIRIs, PSGs and training awards. The immediate outcomes expected to materialize from these activities are: increased research collaboration in identified priority areas; increased number of highly qualified peopleFootnote v in brain disease research; and increased discoveries related to knowledge of the brain, new therapies, interventions, diagnostics and products.
It is expected that positive results for these immediate outcomes will contribute to a strengthened brain disease research community; and increased use of new therapies, interventions, diagnostics and products; to ultimately improve the health and quality of life for Canadians suffering from and affected by neurological and psychiatric diseases.
The connection between these activity areas and the expected outcomes is depicted in the logic model. The evaluation explored the degree to which the defined outcomes have been achieved over the specified timeframe of the evaluation, while acknowledging the limitations of performing an assessment at such an early stage in the Fund’s operation.
2.4 Program Alignment and Resources
Through the CBRF, Health Canada aims to accelerate discoveries focused on improving the health and quality of life for Canadians living with brain disorders. This is aligned with Health Canada’s work on Health System Priorities (subprogram 1.1.1) which refers to “working closely with provincial and territorial governments, domestic and international organizations, health care providers, and other stakeholders to develop and implement innovative approaches, improve accountability, and responses to meet the health priorities and health services needs of Canadians”. This subprogram is one of two that supports Canadian Health Systems Policy (Program 1.1), which in turn addresses Strategic Outcome 1: A health system responsive to the needs of Canadians.
The design of CBRF’s funding model is a 1:1 match, where the federal government matches all funds raised through non-federal sources on a dollar-for-dollar basis. This model will be discussed in greater detail later in the report. Table 1 presents CBRF’s revenues for the years 2011-12 to 2015-16. The time periods compared are not a direct match since CBRF (non-federal funding column) operates on the basis of a calendar year, whereas Health Canada structures their financial data by fiscal year.
Year | CBRF Sources of Funding ($) | ||
---|---|---|---|
Non-Federal Fundinga | Fiscal Year | Health Canada Contributionb | |
2011 | - | 2011-12 | 10,000,000Footnote vi |
2012 | 6,747,567 | 2012-13 | 10,000,000Footnote vi |
2013 | 5,404,909 | 2013-14 | 6,747,567Footnote vii |
2014 | 6,180,793 | 2014-15 | 5,404,909 |
2015 | 17,337,305 | 2015-16 | 6,180,793 |
2016Footnote viii | 5,783,478 | ||
Total | 41,454,052 | Total | 38,333,269 |
- a Source: Brain Canada Foundation, Audited Financial Statements 2011 to 2015 and updates
- b Source: Health Canada financial information
3.0 Evaluation Description
3.1 Evaluation Scope, Approach and Design
The scope of the evaluation covered the period ranging from March 2012 to March 2016, and included an examination of the Health Canada contribution to establish the CBRF. Given that fundraising activities started in 2012, the scope of the evaluation was kept narrow, focusing on CBRF activities related to research funding and the funding model itself. In addition, program data at this point in time is output focused as it is too early for many of the outcomes to have materialized.
The evaluation issues were aligned with the Treasury Board of Canada’s Policy on Evaluation (2009) and considered the five core issues under the two themes of relevance and performance, as shown in Table 2 below. Corresponding to each of the core issues, specific questions were developed based on program considerations and therefore guided the evaluation process.
Core Issues | Evaluation Questions |
---|---|
Relevance | |
Issue #1: Continued Need for the CBRF | What is the current need for research on brain diseases, disorders and injuries? |
Issue #2: Alignment with Government Priorities | To what extent do the activities and outcomes of CBRF align with the priorities of the federal government and departmental strategic outcomes? |
Issue #3: Alignment with Federal Roles and Responsibilities | To what extent are BCF mandate and activities aligned with federal roles and responsibilities? |
Performance (Effectiveness, economy and efficiency) | |
Issue#4: Achievement of expected outcomes | To what extent did the CBRF progress towards producing the intended outputs and achieving its intended outcomes? |
Issue #5: Demonstration of Efficiency and Economy | To what extent does the current delivery model allow for an efficient and economical use of federal government resources? |
The Policy on Evaluation (2009) guided the identification of the evaluation design and data collection methods. A non-experimental design was used based on the evaluation matrix, which outlined the evaluation strategy.
Data collection activities were primarily carried out between March and June 2016.Footnote ix Information for the evaluation was collected using various methods, including: document/literature review, administrative and financial data review, and interviews with internal and external key informants. A total of 17 interviews were conducted due to the small scope of the evaluation and its timing of taking place within the early stages of the Fund’s existence. Key informants were drawn from the following groups: Health Canada (n=3); BCF (n=1); donors and former or active board members (n=5), lead investigators (n=2); peer reviewers (n=2); one academic not currently receiving funding from the CBRF; one representative from the Canadian Institutes of Health Research (CIHR); one representative from the Ontario Brain Institute (OBI); and one from the Canadian Institute for Advanced Research. Data was analyzed by triangulating information gathered from different sources to increase the reliability and credibility of the evaluation findings and conclusions.
3.2 Limitations and Mitigation Strategies
Most evaluations face constraints that may have implications for the validity and reliability of evaluation findings and conclusions. Table 3 outlines the limitations encountered during the implementation of the selected methods for this evaluation. Also noted are the mitigation strategies put in place to ensure that the evaluation findings can be used with confidence to guide program planning and decision-making.
Limitation | Impact | Mitigation Strategy |
---|---|---|
Too early for a thorough assessment of the program’s achievement of outcomes | Progress towards long-term outcomes has not been explored and findings for earlier outcomes are limited. | The report refers to anecdotal evidence as early indications of progress, and where possible provides support from literature and document review sources. Limitations have been discussed in each section to provide context for potential progress. |
Key informant interviews are retrospective in nature | Interviews provide perspective on past events which can impact validity of views reported and related to the activities implemented or the results achieved. | Triangulation of other lines of evidence to substantiate or provide further information on data received in interviews. The literature / document review provide background knowledge, helping contextualize and interpret interview findings. |
Complex financial model | Financial data must be reported in a manner that is not typical when compared to most federal grant and contribution programs. | Information provided was validated with BCF audited financial statements and updates from BCF staff, as well as the contribution agreement with the department. Detailed explanations are also provided in the report, all of which have been validated by BCF. |
Limited ability to assess efficiency and economy | Reduced ability to quantify resource utilization. | Used other lines of evidence, including key informant interviews and administrative data review to qualitatively assess efficiency and economy. |
4.0 Findings
This section provides a summary of the evaluation findings organized under two broad headings:
- Relevance: continued need for the program; the program’s alignment with government priorities; and whether a federal role in brain research exists.
- Performance: achievement of intended outcomes (effectiveness)Footnote x; efficiency and economy of CBRF activities.
4.1 Relevance: Issue #1 – Continued Need for the Program
The significant and continued rise of prevalence rates of neurological diseases, the associated costs of care as well as the need to better understand the brain as a whole to enable cross-cutting discoveries, supports an ongoing need for dedicated brain research.
Conservative estimates indicate that approximately 3.6 million people in Canada are living with some form of neurological diseaseFootnote xi (excluding mental illnesses).14 These findings were highlighted through the report Mapping Connections – An Understanding of Neurological Conditions in Canada (2014), produced in partnership by the Neurological Health Charities Canada, the Public Health Agency of Canada (PHAC), Health Canada, and CIHR which examined 17 of approximately 600 neurological diseases.15 The World Health Organization (WHO) has stated, “diseases of the brain are predicted to surpass cancer as the second leading cause of death in Canada by 2040”.16
Not only are the number of people impacted by diseases of the brain expected to increase, but more specifically, costs associated with neuropsychiatric conditionsFootnote xii are the highest in terms of the economic burden of illness and injury in Canada, in comparison to other conditions. Between 2005-08, the direct and indirect costs of neuropsychiatric conditions were the highest of all health conditions with an impact of approximately $12.5 billion worth of direct and indirect costs. This represents 6.6% of the total economic burden of illness and injury in Canada.17,Footnote xiii Since the prevalence and incidence rates of some of the most common neurological conditions tend to increase with age, both the number of individuals facing these challenges and the cost of associated care are expected to rise as the Canadian population ages.13
As illustrated in Table 4, neuropsychiatric conditions rank second overall in terms of total direct costs in Canada between 2005-08Footnote xiv, consuming 6.6% of all system resources spent in this category for the period, almost as high as the amount of resources used to support cardiovascular diseases. As it pertains to indirect costs, neuropsychiatric conditions consumed 6.2% of all system resources spent in this category, ranking fourth after injuries, respiratory infections and musculoskeletal diseases. Indirect costs refer to mortality and morbidity related costs. Morbidity costs are based on lost productivity costs associated with an individual’s own illness and injury. Lost productivity due to informal caregiving for the sick and injured was not included.
Type of Cost | Total Cost of illnesses | Neuropsychiatric Diseases | Cardiovascular Diseases | Digestive Diseases | Injuries | Musculoskeletal Diseases |
---|---|---|---|---|---|---|
Drug | 27,922,400,000 | 3,551,265,100 | 4,272,675,500 | 1,434,019,700 | 259,677,900 | 1,982,500,200 |
Hospital | 49,122,500,000 | 5,520,276,800 | 5,068,039,500 | 2,839,407,600 | 3,395,837,400 | 1,795,850,600 |
Physician | 23,780,300,000 | 2,347,011,500 | 2,352,012,100 | 1,232,587,100 | 1,435,008,700 | 2,002,466,300 |
Other | 71,179,600,000 | - | - | - | - | - |
Total Direct | 172,004,800,000 | 11,418,553,400 | 11,692,727,100 | 5,506,014,400 | 5,090,524,000 | 5,780,817,100 |
Morbidity | 16,396,400,000 | 1,024,400,000 | 269,600,000 | 151,100,000 | 2,909,000,000 | 1,395,500,000 |
Mortality | 454,000,000 | 19,153,800 | 92,660,600 | 24,560,700 | 84,600,000 | 2,493,900 |
Total Indirect | 16,850,400,000 | 1,043,400,000 | 362,000,000 | 175,600,000 | 2,993,500,000 | 1,398,000,000 |
All Costs | 188,855,200,000 | 12,462,000,000 | 12,054,700,000 | 5,681,600,000 | 8,084,100,000 | 7,178,800,000 |
Source: Economic Burden of Illness in Canada, 2005-2008 (p. 10-11)
Several key informants indicated that discoveries in neurological research are behind in comparison to discoveries in other fields. As noted by WHO, “despite the huge burden they cause, neurological conditions are largely absent from the international health agenda. Moreover, country health plans frequently do not cover neurological disorders at the same level as other illnesses.”18 In addition, pharmaceutical companies have had little success in central nervous system drug development or in uncovering any leads related to neuropharmaceutical research. Consequently, their funding within this area of research has significantly decreased19, in spite of an aging population “which promises a dramatic surge in brain diseases”.20 This is evidenced in changes dating back to 2011 which saw GSK, AstraZeneca and Novartis announcing closures of neuroscience divisions globally and in the case of Pfizer, Sanofi, Janssen and Merck, significant downsizing of their central nervous system research operations.21 As indicated by a few key informants, the challenge with research in this field is that there is a lack of good targets to guide research due to the limited breakthroughs in neuroscience to date, reflecting the vast gaps in knowledge that still exist within this area of research and the need to share knowledge and findings through collaborative research and shared data platforms.
Although organizations for other organ-specific areas of research have existed for quite some time (e.g., the Heart and Stroke Foundation, the Kidney Foundation, etc.), foundations focused on health research involving the brain have been created in disease-specific silos (e.g., the MS Society, the Alzheimer’s Society, etc.). This has created limitations in the linking of findings across diseases impacting the brain. As highlighted by witnesses to the Subcommittee on Neurological Diseases of the Standing Committee on Health (2012), there is a need to support research that approaches brain diseases collectively to understand the similar degenerative pathways involved in different neurological diseases, which would be best addressed by the conduct of multidisciplinary research.14
4.2 Relevance: Issue #2 – Alignment with Government Priorities
The CBRF is aligned with departmental priorities and is contributing to federal science priorities linked to brain health.
Through funding to the CBRF, Health Canada aimed to accelerate discoveries that improve the health and quality of life for Canadians who suffer from brain disorders. This is aligned with Health Canada’s work on Health System Priorities (subprogram 1.1.1), described in annual Departmental Performance Reports as aimed at developing and implementing innovative approaches and responses to meet the health priorities and health service needs of Canadians.22
According to Mobilizing Science and Technology to Canada’s Advantage (2007)23, scientific discoveries and new technologies can contribute to improving overall health while providing solutions to important social challenges and improving Canada’s economic competitiveness. Health and related life sciences and technologies were outlined as a priority, emphasizing a new approach that encourages partnerships with the private sector, non-profit and academia to increase the application and commercialization of research. In this context, the creation of the CBRF aligns with the Government of Canada Science and Technology Strategy’s commitment to “strengthen public-private research and commercialization partnerships”.24 This engagement in a public-private partnership model has allowed greater access to funds supporting brain research by tapping into private donor sources that are not typically available to a federal research agency (e.g., CIHR).
Neuroscience research in general has been on the federal agenda since 2011, as demonstrated through a variety of federal budget commitments, including:
- Budget 2011 allocated up to $100 million to BCF to help establish CBRF.
- Budget 2014 provided $15 million per year to CIHR for a number of priorities including the creation of the Canadian Consortium on Neurodegeneration in Aging.
- Budget 2015 announced $42 million worth of funding over five years, starting in 2015–16, to help establish the Canadian Centre for Aging and Brain Health Innovation administered by Baycrest Health Sciences. The mission of the Centre is to accelerate the development, commercialization and adoption of innovative products, services and best practices to support brain health with a specific focus on aging.25 The organization also receives funding from the Government of Ontario, the private sector and donors of the Baycrest Foundation.
Furthermore, in Budget 2016, BCF’s CBRF obtained an additional $20 million as one of the investments made to “attract and retain world-class researchers and developing promising discoveries into applications that create value for Canadians”.26 This last budget announced that the Minister of Science would undertake a comprehensive review of all elements of federal support for fundamental science, including granting councils. The report, planned to be available by the end of 2016, intends to assess the Government of Canada’s flexibility to respond to emerging research opportunities; opportunities to increase the impact of federal support in research; the support for promising researchers; and to bring greater coherence to the federal research and development priorities and funding instruments.
4.3 Relevance: Issue #3 – Alignment with Federal Roles and Responsibilities
The role of the CBRF aligns with the federal government roles in research and science. Although there may be some overlap between CBRF and CIHR research funding offerings, the two organizations are more complementary than duplicative.
The Department of Health Act establishes Health Canada’s role in the promotion and preservation of the physical, mental and social wellbeing of the people of Canada (section 4(2)(a.1)). Health research provides a greater understanding of effective prevention and treatment approaches to support the desired health outcomes listed above. As stated in the Standing Committee on Health report (2012), the federal government has a research and surveillance role related to neurological diseases; this role is carried out through the work of CIHR, the Canadian Institute for Health Information (CIHI), and PHAC.
CBRF was positioned at its outset as a contribution to the Federal Science and Technology Strategy (2007) managed by the Department of Innovation, Science and Economic DevelopmentFootnote xv, the federal lead for science matters.
The discussion of duplication and complementarity in the area of health research can seem clear at first glance. There are multiple research agencies and, within the federally funded research landscape, CIHR traditionally awards federal health research funds and collaborates with research institutes, foundations and charities that are diverse in terms of geography, focus and capacity. In comparing CIHR and BCF, several key informants were in agreement that differences in governance structures and types of projects funded by the two research bodies lent more so to complementarity than duplication (Table 5). During the period under evaluation, CIHR research grants did not promote interdisciplinary research or multi-investigator teams, which are the foundational criteria for CBRF grants. CBRF requires a cross-cutting collaborative approach in order to promote research that understands the brain as a whole and consequently can advance understandings of the organ in its overall function thereby benefiting various areas of research, versus a more disease-specific approach. Furthermore, CIHR pursues research objectives through their 13 different siloed institutesFootnote xvi covering the many facets of health, ranging from policy research, identity factors linked to health (e.g., gender, aging, Aboriginal peoples), and disease-specific areas of research (e.g., cancer). Not only is health funding distributed across the full spectrum of health research categories, within the institute that relates to neuroscience, they must manage their research funds across the competing categories of mental health and addictions as well. Brain Canada, on the other hand, dedicates all of their research funding to neuroscience.
Federally funded research is not typically funded through the securing of major gift donations from private donors. The donation source has implications for the way in which projects are developed and the nature of the research conducted. CIHR must fund research that aligns with the institutes’ long-term strategies, and operates within the context of an organization comprised of 13 institutes covering all aspects of the health portfolio, rather than a focused specialization. BCF has a brain-specific research mandate and has the flexibility to develop research priorities in consultation with donors and scientists without requiring certain amounts of funding to be channeled within particular research themes.
BCF could be characterized as taking on projects with a greater degree of risk, in comparison to a government agency, while also taking steps to implement risk mitigation mechanisms (e.g., ongoing evaluations). A few donor and peer review key informants confirmed BCF’s selection of more innovative projects. The most commonly cited difference between CIHR and CBRF mentioned by key informants was Brain Canada’s funding of research teams versus individual investigators. Of note, since the launch of CBRF, CIHR implemented the new Open Suite of Programs funding schemesFootnote xvii,27 in 2016 for the health portfolio at large, which are “designed to meet the needs of a broader disciplinary mix of researchers within CIHR’s mandate”.28 The degree of experience the two organizations have in funding research varies greatly. BCF has funded 138 projects in the first few years of its existence, while CIHR has funded 1,765 grantsFootnote xviii between 2011/12-2015/16 (of which 37 were team grants).29
CIHR Project Scheme | CIHR Foundation Scheme | CBRF MIRI Programs | |
---|---|---|---|
Objective | Designed to capture ideas with the greatest potential for important advances in the field of health by supporting projects with a specific purpose and defined endpoint. | Designed to contribute to a sustainable foundation of health research leaders by providing long-term support for pursuit of innovative, high-impact programs of research. | Designed to accelerate novel and transformative research through multidisciplinary collaboration that will fundamentally change the understanding of brain function and dysfunction, especially those with high potential for impact. |
Grant Value (per year) |
Approx. $25,000 to $750,000 | Approx. $50,000 to $1.5 million | Approx. $50,000 to $2,000,000 |
Grant Duration | 1 to 5 years | 5 to 7 years | 1 to 5 years |
Eligibility | One or more independent researcher(s) and/or knowledge users from any health field and at any career stage. | Teams of two or more investigators in any scientific discipline who are eligible to apply for research grants from the Canadian federal granting agencies. | |
Assessment Criteria |
|
|
|
In the case of the PSGs, several key informants and an environmental scan demonstrated that there is little overlap within the field for this type of work despite a crucial need for further maintenance and capacity building to enable the use of platforms to link research findings across the brain community.Footnote xix,30 As one key informant noted, in the past there have been many opportunities to fund these mechanisms but little engagement to do so. Typically, the few organizations involved in funding PSGs target the development of new platforms, whereas CBRF is concerned with the maintenance of existing platforms and capacity building activities to ensure researchers are maximizing platforms already in place.
4.4 Performance: Issue #4 – Achievement of Expected Outcomes (Effectiveness)
In this section, we outline the extent to which key program outcomes have been achieved. Given the recent launch of the Fund, the evaluation has focused on the immediate and intermediate outcomes that in time will lead to the ultimate outcome featured in the CBRF’s logic model. Therefore, we have examined CBRF performance in the areas of: increasing research collaboration and the number of discoveries related to knowledge of the brain, therapies and interventions; strengthening the brain research community; and increasing the use of the new therapies, interventions, diagnostic tools and products.
Projects that have received funding through CBRF are still in progress. At the time of the evaluation, only five of the 82 MIRI projects had concluded (see Table 8). As a result, this limits the extent to which outcomes can be assessed. Therefore with such a small sample, outcome-related findings may change over time as projects continue to progress through their funding periods (which will come to an end over the next few years, some as late as 2020).
4.4.1 To what extent have the outcomes been achieved?
Outcome #1: Increased research collaboration in identified priority areas.
Collaboration and multidisciplinarity are mandatory requirements to access CBRF funds, which, by design, translate into atypical disciplines taking part in research projects in the area of brain sciences.
Brain Canada takes a whole-of-brain approach designed to broaden discoveries that could further impact understandings of the brain as well as related pathologies across brain diseases. Consequently, specific priority research areas were not clearly defined by Brain Canada beyond the general criteria that relates to “any area of neural function and dysfunction, including special senses, neurological and mental health, and may involve biomedical, clinical, health services or population health approaches.”31,32 Commonly held research challenges across the neuro research community have identified brain research as the next frontier with much to learn, presenting challenges in identifying specific targets for the field at large. Although this creates challenges in assessing the extent to which the Fund has increased research in ‘identified priority areas’, the approach does speak to the context within which the brain research community operates. Since 2013, BCF has conducted consultations in regards to underfunded areas of research, and determined more work is needed in relation to early diagnosis and prevention, stem cells, neuroplasticity and epigenetics. According to Brain Canada, funding initiatives are kept broad to respect the importance of work at all stages of the brain research continuum with the understanding that projects are approved based on their likelihood of producing beneficial understandings which contribute to the field. This approach is reflected through the thematic breakdown of CBRF projects (including MIRIs, PSGs, and awards) awarded thus far:
- Neurodegenerative (40/138 projects representing 27.3% of funding committed) focused on Alzheimer’s and Parkinson’s diseases, amyotrophic lateral sclerosis, mild cognitive impairment, Lhermitt-Duclos disease and front temporal lobar degeneration and dementias.
- Multiple diseases (34/138 or 19.2% of funds committed) refer to projects which span several categories and/or cannot be classified into one specific category. For instance, a project may pertain to an underlying mechanism of disease or injury that is applicable to several neurological disorders or to a platform for drug development, imaging, data-sharing etc. that is applicable to several neurological disorders.
- Neurodevelopmental (20/138 projects or 16.3% of funds committed) focused on autism spectrum disorder, fragile X syndrome, developmental dyscaculia, self-regulation and fetal alcohol spectrum disorder.
- Neuropsychiatric and mental health (12/138 projects or 13.4% of funds committed) focused on topics such as addictions, schizophrenia, mood disorders, stress, anxiety and depression.
- Injuries (16/138 or 11.0% of funds committed) focused on stroke, paediatric brain injury, spinal cord and traumatic brain injuries.
- Brain cancer (12/138 or 7.6% of funds committed) focused on medullablastoma, brain tumor and metastases.
- Other (4/138 or 5.2% of funds committed) include projects on epilepsy, migraines and the visual system.
Areas of Research | Percentage of Funds Committed |
---|---|
Neurodegenerative | 27.3% |
Multiple diseases | 19.2% |
Neurodevelopmental | 16.3% |
Neuropsychiatric and mental health | 13.4% |
Injuries | 11% |
Brain cancer | 7.6% |
Other | 5.2% |
Considering the crossroads of the context of neuro research and assessing the program’s progress towards achieving its intended outcome of collaboration in identified priority areas, it is important to explore the degree of influence donors have on research priorities. Interviews with CBRF donors demonstrated varying perceptions of their degree of influence in defining research priorities. This ranged from some who felt the process was collaborative (e.g., they expressed their specific area of interest and Brain Canada suggested broadening the approach thereby increasing the likelihood of uncovering new insights and findings that could impact the field as a whole). Another organization engaged in CBRF research projects based on alignment with their current programs. The remaining interviewee explained donors can determine where funding goes. This environment is consistent with funder engagement exemplified in the Canadian Council on Social Development’s study of 100 non-profit and voluntary sector organizations across the country. Their research found that “funders are adopting an increasingly targeted approach to funding”.33 Ultimately, CBRF documentation and some key informant interviews expressed that typically there is alignment among donor and BCF research areas of interest, which is particularly important for charitable foundations that need to ensure the funds they donate are used for research that matches the expectations of their donors. Furthermore, BCF has an arm’s length council, SAC, in place to ensure that research funded continues to push the neuroscience field forward. This council is composed of prominent international researchers representing the spectrum of brain disorders. They provide foresight on the research opportunities, trends and challenges in the field to situate Canada globally in the latest developments, and so that CBRF funded initiatives are at the forefront of neuro research.
Collaboration and multidisciplinarity are fundamental elements of the CBRF delivery model. Many of the key informants found the collaborative approach of CBRF to be a positive approach and one of the Fund’s distinguishing factors. As expressed by the WHO, “partnerships are advantageous in enhancing the effectiveness of interventions, increasing the resources available through joint actions and avoiding a duplication of efforts”.34 Not only were key informants pleased with the type of research CBRF was able to facilitate, several key informants made note of the impact such resources have on attracting and retaining well respected neuroscientists in Canada. Some key informants highlighted that team grants have brought new people into the community to work on brain sciences, allowing for new perspectives to be incorporated thereby overcoming some of the siloed approaches experienced in the research field.
By the end of 2015, more than 700 researchers based out of 70 institutions had been engaged in at least one of Brain Canada’s 138 projects.35 Types of partners involved included, but were not limited to, provincial organizations, research institutes, and the pharmaceutical industry. The nature of the MIRI grants ultimately provides the funding mechanism to foster open collaboration among the research community, which according to Brain Canada, had been missing in the past. A few key informants also highlighted that the funding mechanism through CBRF helped alleviate some of the challenges of pursuing funding for multidisciplinary research.
Moreover, the uniqueness of the types of multidisciplinary research efforts were of note. The involvement of experts from varying disciplines provides the opportunity for a more fulsome understanding and additional learning opportunities “between individuals, between disciplines, and between types of knowledge”.36 Examples of brain research projects that demonstrated a mix of disciplines and interests not traditionally observed in the field of brain research, include projects on:Footnote xx
- Neurodegeneration, which involved investigators in epigenetics and biophotonics;
- Carrier-mediated delivery into the brain relied on investigators from the fields of biochemistry, mental health / molecular and cellular cognition, molecular genetics, clinical immunology and developmental and stem cell biology;
- The use of endogenous neural stem cells to promote brain cell repair involves investigators in cell and molecular developmental neurobiology, developmental psychology, cognitive neurobiology, hematology / neuro-oncology;
- Non-invasive treatment of paediatric neurological disorders using MR-guided focused ultrasound relies on researchers in biomedical engineering and neurosurgical imaging; and
- Focus on Brain project, a collaborative effort involving Consortium Québécois sur le développement des médicaments (CQDM) and the OBI involves research focused on the blood-brain barrier. This partnership with the pharmaceutical industry has the ability to facilitate a greater understanding of pharma priorities, potential targets, and the challenges associated with accelerating drug development for neuro-related treatments.
As expressed by one of Brain Canada’s international review members, “what has impressed me the most is the synergistic interaction of the various research groups that would likely otherwise have not collaborated without the financial support of Brain Canada. In my view, this has moved the science significantly forward.”Footnote xxi
Outcome #2: Increased discoveries related to knowledge of the brain, new therapies, interventions, diagnostics and products.
The majority of the projects have only recently been initiated, therefore it is too early to determine the breadth of discoveries relating to the brain, therapies, interventions, diagnostic tools and products that would be stemming from CBRF funded research activities. There are early indications of findings in specific cases, that have the potential to meaningfully contribute to the brain science field. At this point in time, it appears the portfolio of projects that have been funded is well balanced in terms of translational, pre-clinical and clinical research.
The original contribution agreement between Health Canada and BCF came into effect March 2012. Only one PSG project started receiving funding in 2014 and the 21 remaining projects received funding in 2015. Furthermore, training awards account for a relatively small portion of CBRF funding. As a result, this section focuses primarily on MIRI projects. BCF’s 2014 annual report stated 77 projects had been confirmed under the following research categories: 25 basic research, 19 translational research, 15 pre-clinical, 14 projects at the clinical stage, two public health, and two projects with combined research categories.
Considering the time needed to raise funds and to conduct the peer review process for selecting research projects, as well as carrying out the project itself, only five projects had concluded at the time of the evaluation. This limits the feasibility of assessing program outcomes at this point in time. As a result, this section will focus on relevant outputs that serve to lay a foundation for future outcomes to materialize.
The five projects that concluded in 2015 stemmed from the 2014 Amyotrophic lateral sclerosis (ALS) Hudson and Discovery grants, a partnered program with ALS Societies of Canada. According to ALS Canada, the Discovery Grant program provided $100,000 for funding highly novel ALS research which would otherwise have had difficulty in obtaining funding from traditional sources. ALS discovery grants were aimed at helping researchers build a foundation of results to boost their future applications to multi-year opportunities.,37 Research results from the five completed projects had not been published at the time of this evaluation.
Other projects have resulted in publications contributing to the greater brain science body of knowledge. BCF, based on reporting received from 43 of 63 projects, identifies a total of 169 publications with Brain Canada acknowledgement, out of which 83 stem from MIRI projects, 16 from PSGs and 70 from training awards. Articles are published in peer reviewed journals and other academic publications such as the American Journal of Psychiatry, Biological Psychiatry and Genome Research. At the time of this report, 20 projects had yet to reach their first submission deadline, therefore the amount of publications cited above is most likely a modest representation.
Based on key informant interviews, below are some examples of discoveries resulting from CBRF projects:
- Azrieli Neurodevelopmental Research Program Phase 1- Structural and functional networks in Autism Spectrum Disorder and Fragile X syndromeMIRI: The team was able to build off their previous work which identified early biomarkers for Autism Spectrum Disorder (ASD) beginning at two years of age, earlier than the field’s typical timing of at least three years old. The work completed through the MIRI grant replicated that research process in order to uncover ASD biomarkers at the 12 and 6 month age ranges. At the time of this evaluation, these findings were scheduled to be published within the weeks to follow. The importance of this research is that it contributes to facilitating ASD clinical diagnoses at earlier ages, before most of the damage has already been done. This team also produced a data review by using a platform based out of 13 research centres located in North America and Europe. As a result of this project, data across studies was cleaned to identify findings that had not previously been uncovered through this public database. Prior to their research, the science community had found essentially no difference between normally developing children and those with autism. The research through the Acquired brain injury (ABI) dataset, identified a pattern of changes in the cortical thickness throughout different parts of the brain from ages 6 to 30 years old.
- Antipsychotic treatment in a genetic subtype of schizophrenia: Novel insights from neuroimaging and pharmacogenetics (Training Award): the award recipient was able to complete a systemic evaluation of antipsychotic response with patients carrying the 22q11.2DS marker, a high risk marker for Parkinson’s disease and a molecular subtype for schizophrenia. She has authored 11 scientific manuscripts, including four as first author, on the expression of schizophrenia and Parkinson’s disease in adults presenting with the important genetic syndrome. Her independent work as well as research she has contributed to has proposed, “practical strategies for the recognition, evaluation, surveillance, and management of the associated morbidities”.Footnote xxii,38
It is still premature to assess the breadth of discoveries emerging out of CBRF funded work that has just recently begun, but anecdotal evidence gathered suggests progress is expected in the coming years, with some of the work having been identified as potentially contributing to advances in the field of brain sciences (e.g., epigenetics, motor neurons).
Outcome #3: Strengthened Canadian brain research community
While it is still early in the research process for projects funded by CBRF to fully achieve this outcome, indications from interviews and processes like establishing and maintaining partnerships, and the selection of broad-based research projects, have started to contribute to a strengthened brain research community.
The CBRF approach to strengthening the brain research community39 first rests on the need to consider the brain as one system, recognizing that common mechanisms are involved among a range of neurological diseases and disorders, mental illnesses, addictions, brain and spinal cord injuries. Secondly, many people from diverse, and potentially non-traditional backgrounds and expertise from equally diverse organizations can work together to accelerate progress.
It is only recently that the whole-brain approach and cross-disease commonalities advocated by BCF have garnered more interest among researchers, and support by other funding organizations and governments, as indicated in the report released by the Subcommittee on Neurological Diseases of the Standing Committee on Health (2012). This approach is considered to be an innovative and beneficial approach to neurological research, as evidenced by the document review and many of the key informant interviews.
Although sources confirmed the benefits of a multidisciplinary approach to neuro research (driven by a whole-of-brain lens), this particular intermediate outcome is anticipated to materialize within 7-15 years of the initial launch of the program. As such, it is too early to assess the extent to which CBRF has strengthened the Canadian brain research community. Therefore, the following section is not to serve as a determination of the extent to which this outcome has been achieved; rather, it serves to highlight early indications of positive contributions in this area based on annual reports and key informant interviews.
The general structure of MIRI grants are designed with the intent of strengthening the brain research community. As indicated on the Brain Canada MIRI proposal webpage, “the focus of proposals must be on investigating commonalities among multiple or related neurological and/or psychiatric conditions. The proposal must show how the research will generate findings applicable to multiple nervous system diseases or conditions.”Footnote xxiii As a result, the CBRF encourages proposals coming from investigators working in diverse and sometimes atypical disciplines for the field of research being funded, as a means to incorporate broadened perspectives.
As mentioned earlier, it is still too early to make an assessment across projects as to the impact from these collaborative research opportunities. However, anecdotal evidence suggests that such grants have facilitated cross-disciplinary teams, often with innovative approaches. Ultimately, this exposure to and coming together of researchers who often work in siloed but complementary fields leads to a more all-encompassing approach to brain science research. For example, in the case of the 2014 grant awarded to a lead researcher based out of the Centre for Addiction and Mental Health, the open call linked to that grant was seeking a team to develop effective strategies for the prevention of Alzheimer’s disease and related disorders with the additional criteria of originality, feasibility and multidisciplinarity. Up until that point, the lead researcher had dedicated his career to conducting clinical trials for older patients with mood disorders.40 Through his MIRI grant research, his team was able to establish a link between the mechanisms involved in depression and those occurring in Alzheimer’s disease. A key informant noted that this group of researchers, because of their untraditional areas of expertise, would most likely not have applied nor been considered for other funders’ open call competitions (e.g., Canadian Consortium on Neurodegeneration in Aging). Leaders in the field of Alzheimer prevention were a part of the selection committee and recognized that this project was the most promising alternative currently available to prevent the development and progression of Alzheimer’s disease and related disorders.41 The experts estimate that it may realistically lead to a medical intervention accessible to Canadians within a five year timeframe.
Not only are CBRF’s MIRI projects supporting the coming together of researchers from various fields, there are indications of relationship development across complementary organizations. Specifically, BCF is liaising with the Mental Health Commission of Canada in order to stay abreast of the findings of their research, providing BCF the opportunity to pursue lines of study that have been determined as requiring further investigation. The eventual goal is that BCF will be able to incorporate some of the learnings from the Mental Health Commission of Canada’s research into the mental health research programs BCF might eventually develop.
The relationships and teams developed through CBRF have expanded beyond Canadian borders. There has been international interest in learning more about Brain Canada’s public/private partnership. As a result, Brain Canada’s President/CEO has presented to audiences in Holland, Israel, the USA, and the UK, based on their respective parties’ interest in creating models that would stimulate more donations to brain research.
A few key informants indicated that the funding CBRF injects into brain research has helped to attract key international scientists to the Canadian research community. Furthermore, the availability of funding, according to a few key informants, has also served to provide opportunities that appeal to local researchers, facilitating their continued work in Canada.
The PSGs for major research platforms are designed to enhance technical and research capacity to support the sharing of research evidence among multiple investigators working in the brain sciences field, irrespective of location. CBRF does not fund the development of the infrastructure itself, it is complementary to the Canada Foundation for Innovation’s grants for research platform infrastructure. BCF had found that there were platforms in existence which would provide greater access to research findings for other investigators across the country, but found a limited number of people were accessing them due to a lack of technical expertise and limited funds for upkeep and expansion. These PSGs allow the recruitment and training of specialized personnel needed to operate the platforms and allow for those tools and techniques to be more widely accessible and used. The applicability of these platforms is wide reaching and can provide a base of findings for researchers working in a variety of disease specific areas by exploring trends in relationships between different factors (variables), further supporting a whole-of-brain research approach.
There were specific examples where the PSGs had enabled greater access to research findings across the brain science field:
- Canadian Brain Imaging Research Platform (CBRAIN): The platform connects over 300 researchers in brain imaging centres located within Canada and internationally. It provides access to large data sets and combined capacity in analyzing findings across multi-centre research projects. The platform also “provides a framework that can be configured to accept and analyze data from any discipline” which going forward allows it to be used in a variety of different brain research settings at a national level.25 As of October 2013, CBRAIN was serving over 200 users, based out of 52 cities across 17 countries42, demonstrating an increase in users from August 2010 (40 members) and December 2011 (116 members).43
- The Experimental Imaging Centre- A Local Brain Canada Platform for Preclinical MR Neuroimaging: Following the closure of the National Research Council Institute of Biodiagnostics in 2012, the technical support provider for the Experimental Imaging Centre (EIC) at the Cumming School of Medicine (University of Calgary), the continued existence of the program came into question. The program credits CBRF’s platform grant for the financial bridging which made it possible to continue their MIRI-based research, as well as expand their neuroscience research capacity.44
As indicated earlier, the evaluation acknowledges early indications of a strengthening of the brain research community, but a further assessment of impacts should take place once the intermediate outcome timeframe (7-15 years) has come to fruition.
Outcome #4: Increased use of new therapies, interventions, diagnostics tools and products
The CBRF’s projected intermediate outcome timeline of requiring up to 15 years for increased use of research findings to materialize, is consistent with research implementation studies. At this stage, it is too early to assess the extent to which the outcome has been achieved. Anecdotal evidence suggests that some of the discoveries resulting from CBRF research have the potential for downstream impacts in this area.
As indicated by a few key informants and in some health research impact evaluation literature, there is often a delay between the generation of research discoveries and the dissemination and/or application of that research in the community.45 The downstream impacts of health research are rarely demonstrable in the short-term. Furthermore, only 14% of clinical research actually progresses to full implementation. In those select cases where original (i.e., discovery) clinical research becomes integrated into physical practice, the average amount of time to progress to full implementation was 17 years.31 This is consistent with the 7-15 year timeframe assigned to this CBRF outcome, and provides insight into expectations surrounding the extent to which this outcome is likely to be achieved. A recent impact assessment of health intervention research funded by Australia’s National Health and Medical Research Council recommends that assessments of the impact of health research should allow sufficient time between impact data collection and completion of the original research.46 The study was based on research that found it can take up to 17 years for evidence to be translated into practice.47,48
Given that CBRF research projects started receiving funding in 2013, it is too early to assess many of the resulting impacts, including the use of new therapies, interventions, diagnostics tools or products. Literature review sources suggest that anticipated milestones that could demonstrate progress towards use-related outcomes resulting from research discoveries include: the initiation of clinical trials (including investment, partnerships and ethics approval), regulatory approval, and business plans or pharmaco-economic evaluations for marketization of products, where applicable.Footnote xxiv
At this point in time, some key informants have indicated that, in select cases, early evaluations and clinical trials for therapeutic approaches that build on CBRF-funded discoveries are in place. The research described below is anecdotal evidence gathered through key informant interviews, demonstrating progress toward pre-evaluative or clinical trial milestones:
- Mobilizing stem cells in the brain to treat brain injury in children (2013 W. Garfield Weston Foundation – Brain Canada Foundation Multi-Investigator Research Initiative): Through her prior research, the lead researcher made a discovery on the reprogramming of skin cells into neural cells. With CBRF funding, this investigator is now able to pursue further research to determine if treatment with metaformin and/or physical exercise could stimulate recruitment of endogenous healthy brain stem cells to the site of injury in children and teenagers. A key informant indicated that the findings from this project, which is expected to wrap up in 2016, have the potential to lead to new treatments for brain repair and would allow for rapid uptake.
- ALS-Canada – Brain Canada J. Hudson Translational Team Grant (2015): The team received funding to advance their preliminary discovery of an active ingredient found in Withaferin somnifera (a plant) which acts as an inhibitor of an inflammatory response involved in ALS. It is difficult to source and extract this active ingredient in large volumes, presenting challenges in proceeding to clinical trials. The CBRF grant is supplying funding for the pre-clinical (laboratory) studies of a drug the lead researcher identified in collaboration with ImStar Therapeutics Inc, that may share the characteristics of the original active ingredient. If successful, the research will continue through to clinical trials at the Montreal Neurological Institute.49 According to one key informant, the best-case scenario in which pre-clinical and clinical trials would proceed successfully without significant barriers, would result in clinical trials beginning within four years and the compound would be approved for market eight years from now.
4.5 Performance: Issue #5 – Demonstration of Economy and Efficiency
The Treasury Board of Canada’s Policy on Evaluation (2009) and guidance document, Assessing Program Resource Utilization When Evaluating Federal Programs (2013), defines the demonstration of economy and efficiency as an assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes. This assessment is based on the assumption that departments have standardized performance measurement systems and that financial systems link information about program costs to specific inputs, activities, outputs and expected results.
In the case of CBRF, the implementation of the fund is assumed by a third party (BCF), which maintains a data structure aligned with the requirement associated with the production of financial statements rather than activity-based costing. Therefore, the data structure necessary for assessing whether program outputs were produced efficiently, or whether expected outcomes were produced economically was not available. Considering these issues, the evaluation provided observations on economy and efficiency based on findings from some key informant interviews as well as relevant administrative and financial data available.
Observations on Economy
Operating expenses that were associated with the administration of CBRF are below the allowed levels. Mechanisms are in place and used to assess projects’ progress and use of funds to ensure CBRF funds are disbursed diligently.
Under the terms of the funding agreement, a maximum of up to 10% of federal funding can be used for operating expenses related to the management, administration and overhead specific to the CBRF or for the design of the research programs, call for proposals, peer review, monitoring and evaluation of research awards.50 Administration fees are calculated on a case-by-case basis for the non-federal funds and can range up to 10% as well. CBRF 2011 to 2015 operating expensesFootnote xxv were $7,085,615 which accounts for 8.8% of overall Health Canada funding ($38,333,269) and CBRF spending ($42,336,719 ).
The 2007 Evaluation of Foundations, conducted by the Government of Canada, found that the majority of foundations with yearly disbursements under $40 million, dedicated between 29% and 51% to operating expenses, which suggests that CBRF’s operational expenditures are more lean than the norm when benchmarked against similar organizations. Larger foundations disbursing between $40 million and $80 million annually operate on 7% to 11% of total expenses while granting councils typically operate on 5% to 6% of total expenses. In 2015, CIHR operating expenses amounted to 6.05% of their total expenses51 which is comparable to CBRF within the same time period.
A key informant reported that the detailed annual progress reports requested from investigators are used to validate progress towards achieving project objectives and serve to monitor expenditures made during a given year. Variances in project expenditures that exceed 20% must be explained and a justification for modifications to project budgets is required. Since BCF tends to pursue ideas that are slightly riskier than federal granting agencies, they in turn make efforts to mitigate that risk through a comprehensive project evaluation process. Monitoring ensures that CBRF funds are used towards the completion of research project objectives and prevents a continued transfer of funds to projects that have yet to use the full amount of funding provided. There are rarely incidents of underspending; in the select cases where it has occurred, projects have provided justification that are beyond the control of the project lead, such as contracting issues.
Description of the funding model
A key feature of the CBRF, as detailed in their contribution agreement, is related to acquiring private investments to facilitate brain-related research, which ultimately provides the base for a matching federal government contribution. This engagement of private donors diversifies the donor base and is seen to help strengthen the case for brain research as a philanthropic cause and provides impetus for sustained private funding, beyond the period in which a federal government investment is in place.
Eligible contributions may come from: provincially-funded organizations with independent governance, such as universities, hospitals and research institutes; provincial funding agencies; charitable and philanthropic associations and foundations; individual donors; and private-sector corporations. Governmental and non-governmental organizations that receive more than 51% of their funding from the federal government are not eligible.
While the focus of the contribution agreement is on philanthropy, the BCF noted that its fundraising activities were also a way to attract provincial investments in brain research. Challenges in acquiring provincial donations will be discussed later in the section related to the structure of the funding model.
CBRF operates under a deferral accounting method for contributions, where donations received are allocated to covering expenses of future periods and are earmarked for specific programs. In the interim, Health Canada funds are only permitted to be invested in low risk vehicles (e.g., interest bearing certificates of deposit or Treasury Board bills issued by the government) while the peer review and selection process take place.
In 2011/12- 2012/13, Health Canada provided BCF with a combined advance of $20 million (out of the original $100 million allocated) in order to help launch their fundraising and research programs. The contribution agreement requires that the advance be matched before the end of the funding agreement. BCF intends to match the advanced amount from the first two years starting in 2016-17 through to 2018-19.
Amounts of non-federal funding raised
The amount received by the Foundation from non-government sources (Figure 2) increased from $6,747,567 in 2012 up to $17,337,305 in 2015, for an overall total of $35,670,574. Within the first three months of 2016, an additional $5,783,478 had been raised. In addition, $44,795,948 in donations, to be received across 2016/17-2019/20, have already been committed through contractual agreements with non-governmental donors. Therefore, to date $86,250,000 in non-federal funding has been secured. In order to match the initial $100 million commitment and the additional $20 million influx, BCF has an outstanding $33,750,000 to raise from non-federal sources within the timeframe of 2016-17 to 2019-20.
Figure 2: Breakdown of non-federal funding
-
Figure 2 - Long Description
This figure depicts a pie graph showing the amount received by the Foundation from non-government sources and the amount that still needs to be raised to meet program commitments. The graph shows that from 2012 to 2015 a total of $35,670,574 in donations had been received with an additional $5,783,478 having been raised in the first three months of 2016. In addition, it shows that $44,795,948 in donations, to be received across 2016/17-2019/20, have already been committed through contractual agreements with non-governmental donors. It further shows that in order to match the initial $100 million commitment and the additional $20 million influx, BCF has an outstanding $33,750,000 to raise from non-federal sources within the timeframe of 2016-17 to 2019-20.
Sources of non-federal funding
To achieve its objective of securing the original $100 million in federal funding through a 1:1 matching structure, the BCF’s plan was to target gifts over $100,000 from private donors (non-governmental sources). Ultimately, during the period of 2011-2015 (Table 7), 33 (59%) of the 56 donations qualified as major gifts, of which approximately 49% of those donations exceeded $500,000. Although funding from foundations (35.6%) and academic institutions (18.2%) were the highest value donor groups from non-governmental sources, only 40% (4 of 10) of foundations’ and approximately 50% (11 of 21) of academic institutions’ donations qualified as major gifts. Whereas, mid-range and smaller donor categories comprised of voluntary health organizations (4.2% of CBRF funds raised), private donors (14.5%), and ‘other’ donors (16.6%) were fully made-up of major gift donations.
The remaining donations to CBRF that did not qualify as a major gift had a value ranging from $10,000 to $99,000, with the exception of one which was less than $10,000. Although the original goal of a solely major gifts approach to raising funds accounted for a little less than 2/3 of overall donations, the remaining amounts secured through individual donor sources were still of a significant dollar value.
Foundations | Academic Institutions | Other | Private Donors | Provincial Agencies | Voluntary Health Organizations | Corporations and Financial Institutions | Total Donors |
% | |
---|---|---|---|---|---|---|---|---|---|
Total Donations (%) |
12,705,200 (35.6%) |
6,477,867 (18.2%) |
5,906,400 (16.6%) |
5,169,193 (14.5%) |
2,525,348 (7.1%) |
1,509,901 (4.2%) |
1,376,667 (3.9%) |
- | 100 |
1,000,000 or more | 3 | 3 | 2 | 1 | 0 | 0 | 0 | 9 | 16 |
500,000 to 999,999 | 0 | 2 | 0 | 0 | 3 | 1 | 1 | 7 | 13 |
100,000 to 499,999 | 1 | 6 | 1 | 1 | 1 | 3 | 4 | 17 | 30 |
10,000 to 99,999 | 5 | 10 | 0 | 0 | 3 | 0 | 4 | 22 | 39 |
less than 10,000 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 2 |
Total | 10 | 21 | 3 | 2 | 7 | 4 | 9 | 56 | 100 |
Source: BCF, Amounts received by funding sources by year
Research Activity and Funds Disbursed
According to BCF data, the annual amounts disbursed since 2011 through grants and awards ranged from $78,513 in 2011 to $30,874,988 in 2015. These amounts increased quite drastically from one year to the next due to the start of new projects which overlapped with those already in their second or third year.
The shaded zones in Table 8 on the following page highlight the duration of MIRI research projects and the number of projects expected to conclude by year and research program. There are lags between when projects are selected and the start date of the actual research projects. For example, the funds associated with grants awarded under the Sponsored MIRI 2012 started being disbursed in 2014 to allow for the completion of the peer review process. The milestones are similar for the British Columbia Alzheimer’s Research Award Program for which the request for applications was launched in December 2013, although funding was not received until 2015.
As CBRF was getting established, there were delays in securing donations and consequently, because of the structure of the matched funding model, Health Canada had to reprofile funding for the program. As well, the focus on major gift donations ($100K or more) has resulted in a high proportion of donations being committed in instalment payments over several years. Health Canada funding was reprofiled in November 2014 and August 2015 to match the amount of non-federal donations that BCF received, which was below the departmental reference level. In 2015, the contribution agreement between Health Canada and BCF was extended to March 31, 2020 to allow for the disbursement of the funds raised and matched towards the end of the original cycle. In 2016, funding for CBRF will be reprofiled again.
Research Programs | 2013b | 2014 | 2015 | 2016 | 2017 | 2018 | 2019 | 2020 |
---|---|---|---|---|---|---|---|---|
Sponsored MIRI 2012 | - | X | X | X (5) | - | - | - | - |
Sponsored MIRI 2014 | - | - | X | X | X (4) | - | - | - |
- | - | X | X | X | X (2) | - | - | |
- | - | - | X | X | X (4) | - | - | |
Sponsored MIRI 2015 | - | - | - | X | X | X (4) | - | - |
ALS-Hudson and Discovery Grants | - | - | X (5) | - | - | - | - | - |
- | - | X | X | X (1) | - | - | - | |
- | - | - | X | X (8) | - | - | - | |
- | - | - | X | X | X | X | X (4) | |
- | - | - | X | X | X (1) | - | - | |
Alzheimer Society of Canada | - | - | X | X | X | X (3) | - | - |
Alzheimer’s Association (US) | - | - | - | X | X | X (3) | - | - |
Azrieli – Phase 1 | - | X | X | X | X | X | X (4) | - |
Azrieli – Phase 2 | - | - | - | X | X | X (1) | - | - |
British Columbia Alzheimer’s Research Award Program | - | - | X | X | X (3) | - | - | - |
- | - | X | X | X | X (2) | - | - | |
Canadian Cancer Society Impact Grants | - | - | X | X | X | X | X (4) | - |
Chagnon Family ADRD MIRI | - | X | X | X | X | X | X (1) | - |
Focus on Brain | - | - | X | X | X | X (6) | - | - |
Heart & Stroke Foundation | - | - | X | X | X (7) | - | - | - |
W. Garfield Weston Foundation MIRI | - | X | X | X | X (1) | - | - | - |
X | X | X | X (4) | - | - | - | - | |
The Jewish General Hospital Foundation | - | - | X | X | X (1) | - | - | - |
Royal Bank of Canada Foundation | - | - | - | X | X | X (1) | - | - |
Canadian Institute for Advanced Research | - | - | X | X | X | X | X | X (3) |
- a One project, which is not included in this table, was started as part of the Brain Repair Program that preceded the CBRF.
- b Years 2011 and 2012 when no projects were active. However, this period of approximately 18 months reflects the lags associated with the process to select and start research projects.
- X: Project in progress
- (#): Number of projects completed during the highlighted (X) period
Matching
The strength of this funding arrangement is its foundational structure of a private-public partnership. The matching of private donations by the federal government was seen by key informants as a powerful incentive that appeals to non-federal donors interested in investing in brain research, and taps into new sources of funding for this particular field. There were a few instances where key informants criticized a matching model as shying away from a predetermined government commitment to fund an area of research deemed a priority. These key informants highlighted that the matching model puts the onus on the organization to raise funds in order to get the full federal amount of funding pledged for a research area which the government itself has prioritized. Despite these concerns, overall, key informants indicated that this model has resulted in an increase of funds available for brain research. Furthermore, an environmental scan of funding models for other research organizations indicates that matching models are quite common, although the matching ratios vary (Table 9). CBRF is the only organization reviewed that had a 1:1 matching formula.
Some key informants called into question research-related expenses eligible for matching. Unlike the other research organizations explored through the environmental scan (e.g., OBI, Genome Canada, Grand Challenges (Table 9)), CBRF does not capture in-kind contributions in the total value eligible for matching. As expressed by one MIRI team lead, the exclusion of the in-kind value associated with their staff time, administration costs, and equipment was cause for concern since their institution already has an indirect cost ratio of roughly $0.52 on the dollar.
Administration Costs
Capturing administration costs within the matched funding model was regularly cited as a challenge among external and BCF key informants. According to the contribution agreement, up to 10% of federal funds received can be applied towards administration costs. The environmental scan found that CBRF’s administration cost ratio is in line with select organizations featured in the environmental scan (e.g., Grand Challenges Canada, Genome Canada). A few key informants challenged the idea of the CBRF funding model where operational expenses for the organization are not distinctly funded (outside of the funds received through the matching structure). Although the contribution agreement’s stipulations concerning the percentage of funding allocated to administrative costs is only specific to federal funds received, without distinct funds for these costs, BCF applies up to a 10% administration fee on some non-federal funds received in order to cover the balance of operational expenses. BCF and donor key informants confirmed that some funders are reluctant to have their money cover administrative costs and at times alternate arrangements are pursued in order to respect the expectations of donors. Examples include having designated administrative costs funnelled to funding interns working at Brain Canada. Also, since the contribution agreement requires all federal funds to be matched, including administration fees (up to 10% of federal funding), this decreases the advertised 1:1 matching ratio to an approximately 90% match of funding from the federal government for research conducted through CBRF. As discussed earlier, CBRF administration costs account for 8.8% of all funding received, which is below the maximum threshold of 10%. As to be expected, the administrative costs (as a ratio) were higher during the initial start-up years of the Fund but have consistently decreased over the period under evaluation, ranging from approximately 22% in 2013 to 4.4% in 2015.
Challenges in the operationalization of the flow-through method
Based on an environmental scan, and according to key informants from BCF and OBI, the model of requiring donations to go through CBRF’s account, then detailed in their audited financial statements in order to trigger a federal transfer of funds to facilitate the match, which in turn is disbursed to the researcher, is an atypical format. The environmental scan of select Canadian-based research organizations (Genome Canada, OBI, Grand Challenges Canada, Brain Canada) produced for this evaluation demonstrated that all of the organizations included in the review engage in some form of a matching model. Most of these organizations release funding to researchers based on a binding attestation that the funds have been secured, rather than the requirement faced by CBRF to have funds physically transferred to their account in order to secure the federal funding match (Table 9). According to key informants, traditionally the matching or blending of research funds tends to take place at the research institution (e.g., university) and is formalized through a multipartite legal agreement which stipulates the proportion of funds provided by each partner. At the time of the evaluation, BCF and Health Canada had confirmed that consultations/discussions related to revising the funding model are currently underway. As indicated in internal Health Canada documents, spending of matched funding on CBRF projects was delayed due to the process of requiring audited financial statements prior to initiating the federal funding match.
Despite creating time consuming administrative practices, a challenge identified by BCF, other key informants acknowledged limitations in their ability to donate funds to the organization based on the way the flow-through model has been operationalized. This was particularly relevant in the case of not-for profit organizations using funds from their respective donors, and agencies that receive provincial funding.
According to the Canada Revenue Agency, a registered charity must devote its resources to the charitable activities that directly further the charitable purposes for which the charity was registered.52 While charitable activities can be carried out through intermediaries, the charities that choose to do so “must be able to demonstrate that it takes an active and controlling part in a program or project that directly achieves its charitable purposes”.Footnote xxvi In the case of the BCF, the Foundation does not qualify as an intermediary carrying out charitable activities on behalf of donors since it is not directly involved in the conduct of research, but rather serves as a funding coordinator. Charities are accountable to their donors and must illustrate their full control of the disbursement of donated funds.
Charities which receive provincial funds face limitations with a flow-through model that requires a physical transfer of funds, based on stipulations that require that the funding they receive stay within the province. According to BCF, the goal of increasing the flow of provincial government investments in the field is constrained by this same intermediary structure which is often perceived as administratively burdensome and contributes to reduced transparency.
Sourcing Funds
In order for a federal funding match to take place, CBRF must receive all donations by a pre-established cut-off date. The end date of the CBRF contribution agreement was extended to 2020, allowing for extra time to disburse research funding but requiring all funds eligible for a federal match to be received by 2017. As a result, funding for current multi-year grants must be received in a lump sum upfront to trigger the match versus annual donations throughout the research process. According to Brain Canada, this poses challenges for not-for-profit organizations as they are unable to accumulate money that covers a commitment spanning three years. Not-for-profit organizations make multi-year commitments based on cash flow but they often do not have the liquidity to supply the funding any further out than on an annual basis. Other donors also take issue with lump sum donations covering the grant period since research funding is disbursed to teams based on milestones achieved and as a result the money sits in CBRF’s account until the installments are paid out over the grant period. Donors often have more detailed investment policies in place, meaning should the funds sit in their account until the point where it is needed a greater amount of interest would accumulate in the interim, as opposed to CBRF’s minimal GIC interest rates.
Organization | Type of research funded | Matching Structure | Administrative Costs | Matching Requirements | Timing for Release of Funds to Researcher |
---|---|---|---|---|---|
Brain Canada | Funds partnered brain science research initiatives | 1:1 Donor: Health Canada |
Up to 10% of federal funding can be applied to administrative costs. | Money must be in Brain Canada’s account prior to being recognized for matched federal funding. In-kind contributions do not qualify for the federal funding match. |
Upfront money on an annual basis |
Genome Canada 53,54,55 |
Genomics and genomic technologies |
1/3: 2/3 Genome Canada: Co-Funder + User (minimum 1/3 provided by the User(s)) |
Administrative costs must not exceed 5% of the project budget. | Released based on binding agreement of secured funding. If that is not in place, Genome only pays their share. Qualifying in-kind contributions can be considered as co-funding. |
One quarter in advance The remaining funding is disbursed quarterly (subject to) receipt of reporting) |
Ontario Brain Institute (OBI)56,57 | Funds Integrated Discovery Programs related to entrepreneurship, education/training, business development, and knowledge translation Does not fund research |
2/3: 1/3 Province of Ontario: User |
Ranges based on project size. | Match based on commitment (e.g., letter of attestation). In-kind contributions are eligible for matching. |
First two quarters funded upfront The remaining installments are disbursed based on a confirmation that funding has been raised |
Grand Challenges Canada58,59,59 | Health implementation innovators from low and middle income countries | No matching Federal government is the main funder through the Development Innovation Fund in Health (IDRC)61 |
Up to 12% of the grant value can cover indirect costs. For transition-to-scale grants, the indirect cost amount must also be matched and is considered part of the total project amount. |
In general, no matching component with the exception of transition-to-scale projects (1:1). Matching through in-kind contributions are possible but not preferred. |
Quarterly |
Observations on efficiency
Efficiencies resulting from BCF’s peer review process for the Fund and the annual projects’ progress review were identified. While the matched funding model has contributed to a greater influx of brain research money in Canada, the model still presents challenges for BCF and their partners.
The MIRI and PSG peer review processes are similar despite differences in criteria pertaining to: the eligibility of applicants, components of the application, and assessments.62 The BCF peer review process for the Fund includes the submission of a LOI, which provides basic information on the project, the team and the budget. The LOI process is designed to identify the most competitive projects, and in doing so, requesting full applications simply from the teams within that niche. The letters are submitted during the pre-panel stage where they are examined by a subset of the peer review committee. The LOI process has the advantage of being less time consuming for applicants and reviewers alike by reducing the number of full applications drafted and requiring review. In some cases, it also gives more time for applicants to secure commitment from their sponsors. CIHR recently incorporated an LOI stage for some of its funding opportunities although it has not been used extensively at this point.
The efficiency of the peer review process was determined by reviewing the proportion of proposals rejected/consolidated at the LOI stage. This data was only available for three MIRI competitions (Table 10). According to this limited source of information, 81% of the 2012 and 46% of 2014 sponsored MIRI applications were rejected at the LOI stage, meaning 170 applications that were not of priority interest to Brain Canada, did not require further applicant time for development or dedicated time by panel members’ for the full peer review process.
Research Programs | LOIs Received | Full Applications | Funded projects |
---|---|---|---|
MIRI 2012 | 165 | 31 | 10 |
MIRI 2014 | 52 | 28 | 15 |
Chagnon Family ADRD MIRI | 23 | 11 | 1 |
Total | 240 | 70 | 26 |
Source: Brain Canada Foundation. (2013). A smart investment in Canadian brain research – Annual Report 2013; Brain Canada Foundation. (2014). One brain. One community – Annual Report 2014.
The projects’ progress is reviewed annually to monitor their use of CBRF funds and to mitigate risks associated with the projects that were selected. Most of the time, this review is conducted internally by the program team. One of the peer reviewers who rated the original proposal is asked to conduct the annual assessment in the case where the project is not straightforward. Projects are evaluated on an ongoing basis and funding installments are only disbursed if projects are proceeding as planned and the funds received to date have been spent.
5.0 Conclusions
5.1 Relevance Conclusions
Neurological conditions have been estimated to affect approximately 3.6 million Canadians and are the most costly of health conditions when accounting for combined direct and indirect costs. The number of individuals living with neurological conditions and the cost of associated care are expected to rise as the Canadian population ages.
The brain has been cited as the last frontier of health research. Key informants and document review sources confirm that there is a continued need for research to better understand how the brain functions. A whole-of-brain approach to research in this field is expected to help identify similar degenerative pathways linked to a variety of neurological diseases.
Neuroscience research has been a federal priority throughout the period captured in this evaluation, and is aligned with departmental priorities. The CBRF’s innovative cross-disciplinary and multi-investigator approach in working towards generating discoveries contributes to current federal science priorities. Although some overlap exists, for the most part, key informants agreed that the work of CIHR and CBRF are more complementary than duplicative.
5.2 Performance Conclusions
While it is still too early to assess the long-term outcomes resulting from the CBRF, it is evident that MIRI grants and PSGs facilitate collaboration within the brain community and across disciplines. This collaborative and multidisciplinary approach is seen as promising in researching common mechanisms presenting across neurological and psychiatric diseases. Currently, the portfolio of projects funded appear to be well balanced in terms of translational, pre-clinical and clinical research.
An assessment of intermediate outcome impacts would be more appropriate once the intended outcome timeframe of 7-15 years has passed. At this point in time, anecdotal evidence suggests that some projects demonstrate potential for downstream impacts, including new discoveries and the use of new therapies, interventions, diagnostics, tools and products stemming from CBRF research discoveries.
To date, CBRF’s private-public funding partnership model has facilitated the incorporation of new sources of funding within the research field that are typically not engaged by federally funded research organizations. However, the operationalization of the funding model could be improved by moving away from requiring donations be transferred to the CBRF and verified through audited financial statements in order to receive the matched federal funding. A shift to a less onerous approach has the potential to better support the objective of the CBRF and build on BCF’s capacity to maintain relationships with partners interested in increasing overall investments in brain research. Mechanisms are in place to continually assess projects’ progress and their use of funds. In terms of administrative efficiencies, LOI peer review process for CBRF has served to streamline the application and project selection processes.
6.0 Recommendation
Health Canada should consider adjusting the CBRF Funding Model.
While the CBRF funding model has clearly resulted in increased funds being dedicated to brain research in Canada, it still faces challenges including:
- Capturing administrative costs associated with the management of all funding and its impact on funding available for matching.
- Requirement of audited financial statements prior to initiating federal funding match which results in delays to spending matched funding.
- Operationalization of the flow-through funding model that includes requirements to physically transfer funds to BCF’s account in order to trigger a federal funding match has impacted the ability of not-for-profits and provinces to donate money. Not-for-profits do not typically transfer funds to intermediaries due to lack of control of disbursement of donated funds, and provincial funders require funding to stay within the province.
To address these challenges HC should continue its examination of the funding model and make necessary adjustments that ensure it meets the needs of both BCF and Health Canada.
Activities | Fund and oversee multidisciplinary, multi-investigator neuroscience research initiatives | Fund and oversee technology Platform Support Grants | Fund and oversee Training Awards |
---|---|---|---|
Outputs | Funded MIRI Grants Knowledge Products |
Funded Platform Support Grants Knowledge Products |
Funded Training Awards Knowledge Products |
Immediate Outcomes | Increased research collaboration in identified priority areas Increased highly qualified people (HQP) in brain disease research Increased discoveries related to knowledge of the brain, new therapies, interventions, diagnostic tools and products |
||
Intermediate outcomes | Strengthened brain disease research community Increased use of new therapies, interventions, diagnostic tools and products |
||
Ultimate Outcome | Improved health and quality of life for Canadians suffering from and affected by neurological and psychiatric diseases |
Appendix 2 – Organizations and Research Background by Project
The table below presents the home organization, disciplines and/or research interests of the lead investigator and collaborators on projects stemming from one sponsored and one partnered initiative launched during the period covered by the evaluation. The purpose of the table is to illustrate the variety of research interests involved and collaborating on CBRF-funded projects.
Projects | Organizations | Examples of disciplines / research interests |
---|---|---|
Sponsored MIRI – 2012 | ||
Propagated protein misfolding of SOD1 in ALS: exemplar for neurodegeneration PI : N. Cashman |
|
|
Non-invasive treatment of paediatric neurological disorders using MR-guided focused ultrasound (MRgFUS) PI: J.M. Drake |
|
|
Dissecting acetylcholine/glutamate co-transmission in the striatum: importance of individual neurotransmitter in addiction and movement disorders PI: S.E. Mestikawy |
|
|
Carrier-Mediated Delivery of Therapeutic Proteins into the Brain PI : R.A. Melnyk |
|
|
Brain Channelopathies – Target Validation and Novel therapeutic Strategies PI: T. Snutch |
|
|
The W. Garfield Weston Foundation | ||
Validation of ocular measures as potential biomarkers for early detection of brain amyloid and Neurodegeneration PI: S. Black |
|
|
Neuronal polarity defects as an underlying cause of neurological diseases PI: M. Cayouette |
|
|
Epigenetics and Mental Health PI : M. Meaney |
|
|
Recruitment of endogenous neural stem cells to promote repair following acquired brain injury in children PI: F. Miller |
|
|
Restoration of visual function: a cellular reprogramming and bioengineering approach PI: Valerie Wallace |
|
|
Appendix 3 – Summary of Findings
Rating of Findings
Ratings have been provided to indicate the degree to which each evaluation issue and question have been addressed.
Relevance Rating Symbols and Significance:
A summary of Relevance ratings is presented in Table 1 below. A description of the Relevance Ratings Symbols and Significance can be found in the Legend.
Evaluation Issue | Indicators | Overall Rating | Summary |
---|---|---|---|
Continued need for the program | |||
What is the current need for research on brain diseases, disorders and injuries? | Evidence of:
|
High | Neurological conditions have been estimated to affect approximately 3.6 million Canadians and are the most costly of health conditions when accounting for combined direct and indirect costs. Up until recently, within the realm of brain research, health foundations have been created in disease-specific silos (e.g., the MS Society, the Alzheimer’s Society), as opposed to an overarching organ focus (e.g. the Kidney Foundation). Key informant and document review sources highlighted a need to support multidisciplinary brain research in order to understand the similar degenerative pathways involved in different neurological diseases. |
To what extent do the activities and outcomes of CBRF align with the priorities of the federal government and departmental strategic outcomes? |
|
High | Neuroscience research has been a federal priority throughout the period captured in this evaluation, and is aligned with departmental priorities. The CBRF’s innovative approach in working towards generating discoveries also contributes to current federal science priorities. |
To what extent are BCF’s mandate and activities aligned with federal roles and responsibilities? |
|
High | The role of the CBRF aligns with the federal government roles in research and science. CIHR typically awards federal health research funds. The evaluation found CBRF plays a complementary rather than duplicative role within the federally funded research environment as a result of its brain-specific and multi-disciplinary approach to innovative research through team grants. |
Performance Rating Symbols and Significance:
A summary of Performance Ratings is presented in Table 2 below. A description of the Performance Ratings Symbols and Significance can be found in the Legend.
Issues | Indicators | Overall Rating | Summary |
---|---|---|---|
Achievement of Expected Outcomes (Effectiveness) | |||
To what extent did the CBRF progress towards producing the intended outputs and achieving its intended outcomes? Outcome #1 : Increased research collaboration in identified priority areas |
|
Progress Made; Further Work Warranted |
Collaboration and multidisciplinarity are mandatory requirements to access CBRF funds, which translates into atypical disciplines taking part in research projects in the area of brain sciences. The whole-of-brain approach adopted by BCF and difficulties in identifying targets in the brain research field at large, lends to broad and evolving priority areas. |
Outcome #2: Increased discoveries related to brain, therapies, interventions, diagnostics and products |
|
Unable to Assess Early signs of progress |
While it is too early to gather concrete evidence on the discoveries related to brain that would be stemming from the funded research activities, anecdotal evidence gathered suggests significant progress is expected in the few coming years. Some of this work carries the potential for significant discoveries such as the work on the blood-brain barrier and the identification of new targets for drug development. |
Outcome #3: A strengthened Canadian brain research community, due to CBRF approach and its funding of MIRI, PSGs and awards |
|
Progress Made; Further Work Warranted |
CBRF projects are in their early years therefore the assessment of the extent to which they contributed to strengthening the brain research community is limited. There are early indications from interviews, and evidenced by processes like establishing and maintaining partnerships and the selection of broad-based research projects, that have started to contribute to a strengthened brain research community. |
Outcome #4: Increased use of therapies, interventions, diagnostics tools and products |
|
Unable to Assess | Concrete evidence could not be collected to support the use of discoveries at this point in time. Anecdotal evidence suggests that some of the research currently funded may reach the stage of use within the next decade. |
Demonstration of Economy and Efficiency | |||
To what extent does the current delivery model allow for an efficient and economical use of federal government resources? |
|
Progress Made; Further Work Warranted |
BCF’s private-public funding partnership model has facilitated the incorporation of new sources of funding within the research field that are typically not engaged by federally funded research organizations. The mechanics of the funding model could be improved to better support the objective of the organization and build on BCF’s capacity to maintain relationships with partners interested in increasing overall investments in brain research. Mechanisms are in place to continually assess projects’ progress and their use of funds. |
Endnotes
1 Brain Canada Foundation. (2013). Strategic Plan 2013 – Building Sustainable Momentum
2 Brain Canada Foundation. (2013). Annual Summary Corporate Plan
3 Brain Canada Foundation. (2016). Grants awarded per year net of admin fees.
4 Brain Canada Foundation. (2016). Funds committed per year by grant type.
5 Brain Canada Foundation. (2016). Amount of research funding disbursed by year.
6 Brain Canada Foundation. (2016). Grants awarded per year net of admin fees.
7 Brain Canada Foundation. (year unspecified) Research Program Procedure Manual
8 Retrieved from http://braincanada.ca/en/Board_of_directors
9 Brain Canada Foundation. (year unspecified). Mandate of the Board of Directors.
10 Brain Canada Foundation. (year unspecified). Mandate of the Research Policy Committee
11 Brain Canada Foundation. (year unspecified). Mandate of the Science Advisory Committee
12 Retrieved from http://www.conferenceboard.ca/events/2009awards/winners.aspx
13 Spencer Stuart. (2012). Canadian Spencer Stuart Board Index: Board Trends and Practices of Leading Canadian Companies 2012 (17th ed.). Retrieved from http://www.swc-cfc.gc.ca/initiatives/wldp/wb-ca/cssbi_eng.pdf.
14 Public Health Agency of Canada. (2014). Mapping Connections: An Understanding of Neurological Conditions in Canada. Retrieved from http://www.phac-aspc.gc.ca/publicat/cd-mc/mc-ec/index-eng.php
15 Smith, J M.P. (2012). Report of the Standing Committee on Health.
16 Robarts Research. (2016). Leading the Way in Brain Research. Retrieved from http://www.robarts.ca/brain-disease-research
17 Public Health Agency of Canada. (2013). Economic Burden of Illness in Canada, 2005-2008. Retrieved from http://www.phac-aspc.gc.ca/publicat/ebic-femc/2005-2008/index-eng.php
18 World Health Organization. (2006). Neurological Disorders: Public Health Challenges. Retrieved from http://who.int/mental_health/neurology/neurodiso/en
19 Drug Discovery World. (2013). The Great Neuro-Pipeline Drain (and why Big Pharma Hasn't Given Up on CNS Disorders. Retrieved from http://www.ddw-online.com/therapeutics/p216813-the-great-neuro-pipeline-brain-drain-(and-why-big-pharma-hasn-t-given-up-on-cns-disorders)-fall-13.html
20 Herper, M. (2015). The Coming Boom in Brain Medicines. Forbes Magazine. Retrieved from http://www.forbes.com/sites/matthewherper/2015/02/11/brain-boom-the-drug-companies-bringing-neuroscience-back-from-the-brink/#4397f8cd52a2
21 Drug Discovery World. (2013). The Great Neuro-Pipeline Drain (and why Big Pharma Hasn't Given Up on CNS Disorders. Retrieved from http://www.ddw-online.com/therapeutics/p216813-the-great-neuro-pipeline-brain-drain-(and-why-big-pharma-hasn-t-given-up-on-cns-disorders)-fall-13.html
22 Health Canada. (2013). Departmental Performance Report 2013-2014. Retrieved from http://www.hc-sc.gc.ca/ahc-asc/performance/estim-previs/dpr-rmr/2013-2014/report-rapport-eng.php
23 Minister of Justice. (1995). Department of Industry Act, clause 4 (1c). Retrieved from http://laws-lois.justice.gc.ca/eng/acts/i-9.2/FullText.html
24 Government of Canada. (2007). Mobilizing Science and Technology: to Canada's Advantage. Retrieved from http://www.ic.gc.ca/eic/site/icgc.nsf/eng/h_00856.html
25 Retrieved from http://www.ccabhi.com/about-us/
26 Government of Canada. (2016). Growing the middle class. Retrieved from http://www.budget.canada.ca/2016/docs/download-telecharger/index-en.html
27 Canadian Institutes of Health Research. (2012). Designing for the Future: The New Open Suite of Programs and Peer Review Process. Retrieved from http://www.cihr-irsc.gc.ca/e/46099.html
28 Canadian Institutes of Health Research. (2015). Health Research Roadmap II: Capturing Innovation to Produce Better Health and Health Care for Canadians – Strategic Plan 2014-15 – 2018-19. Retrieved from http://www.cihr-irsc.gc.ca/e/48964.html
29 Canadian Institutes of Health Research (n.d.). Canadian Research Information System [Database]. Accessed August 24, 2016, from http://webapps.cihrirsc.gc.ca/funding/Search?p_language=E&p_version=CIHR
30 Retrieved from http://www.genomecanada.ca/en/request-applications-genomics-technology-platforms-operations-support-and-technology-development
31 Brain Canada Foundation. (2014). Multi-Investigators Research Initiative Request for Applications.
32 Brain Canada Foundation. (2015). Multi-Investigators Research Initiative Request for Applications. Retrieved from http://www.braincanada.ca/en/MIRI
33 Scott, K. Canadian Council on Social Development. (2003). Funding Matters: The Impact of Canada's New Funding Regime on Non-profit and Voluntary Organizations. Retrieved from http://www.ccsd.ca/index.php/research/funding-matters
34 World Health Organization. (2006) Neurological Disorders: Public Health Challenges. 182. Retrieved from http://who.int/mental_health/neurology/neurodiso/en
35 Brain Canada Foundation. (2015). Big Science, Bold Science, Brain Science – 2015 Annual Report.
36 Haapasaari, O., Kulmala, S., and Kuikka, S. (2012), Growing into Interdisciplinarity: How to Converge Biology, Economics and Social Sciences in Fisheries Research, Ecology and Society. 17(1). 6. Retrieved from http://www.ecologyandsociety.org/vol17/iss1/art6/
37 Retrieved from http://als.ca/en/news/awareness/als-canada-announces-funding-2015-discovery-grants
38 Butcher. N.J. et al. (2015). Response to clozapine in a clinically identifiable subtype of schizophrenia. Br J Psychiatry. Retrieved from http://bjp.rcpsych.org/content/206/6/484.long
39 Brain Canada Foundation. (2014). One brain. One community – Annual Report 2014. Retrieved from http://www.braincanada.ca/en/Annual_reports
40 Retrieved from http://www.camh.ca/en/research/about_research_at_CAMH/scientific_staff_profile/Pages/Benoit-Mulsant.aspx
41 Brain Canada Foundation. (2013). Brain Canada Annual Report 2013: A Smart Investment in Canadian Brain Research. Retrieved from http://www.braincanada.ca/files/LR_BrainCanada_AR2013_English.p1_WEB.pdf.
42 Sherif, T. et al. (2014). CBRAIN: a web-based, distributed computing platform for collaborative neuroimaging research. Front. Neuroinform. Retrieved from http://journal.frontiersin.org/article/10.3389/fninf.2014.00054/full
43 Rousseau, M-E. (2012). An International Computing Platform for Neuroimaging. Dr. Alan C. Evans Laboratory. Retrieved from http://www.bic.mni.mcgill.ca/uploads/Seminars/2012-02-27_-_CBRAIN_-_Marc-Etienne_Rousseau.pdf
44 Brain Canada Foundation. (2015). Big Science, Bold Science, Brain Science – 2015 Annual Report. http://www.braincanada.ca/en/Annual_reports
45 Brownson, R.C. et al. (2006) Translating Scientific Discoveries Into Public Health Action: How Can Schools of Public Health Move Us Forward? Public Health Reports. 12(1). 97-103.
46 Cohen, G. et a. "Does health intervention research have real world policy and practice impacts: testing a new impact assessment tool" Health Research Policy & Systems 2015.13:3 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4292987/
47 Donovan C, Butler L, Butt AJ, Jones TH, Hanney SR: Evaluation of the impact of national breast cancer foundation-funded research. Med J Aust 2014, 200:214–218.
48 Grant J, Cottrell R, Cluzeau F, Fawcett G: Evaluating "payback" on biomedical research from papers cited in clinical guidelines: applied bibliometric study. BMJ (Clin Res Ed) 2000.
49 ALS Society of Canada and Brain Canada Foundation. (year unspecified). Arthur J. Hudson Translational Team Grants. Retrieved from https://www.als.ca/sites/default/files/files/Hudson_Grants-v04.pdf
50 Health Canada. (2011). Multi-year Contribution Agreement with the Brain Canada Foundation.
51 Canadian Institutes of Health Research. (2015). Annual Report 2014-2015.
52 Retrieved from http://www.cra-arc.gc.ca/chrts-gvng/chrts/prtng/ctvts/menu-eng.html
53 Genome Canada. (2014). Genomic Applications Partnership Program (GAPP) – Investment Strategy & Guidelines.
54 Genome Canada. (2014). Guidelines for Funding Research Projects.
55 Genome Canada. (year unspecified). Genomic Applications Partnership Program (GAPP).
56 Ontario Brain Institute. (2016). Retrieved from http://www.braininstitute.ca/about-us
57 Ontario Brain Institute. (2014). Ontario Brain Institute Phase 1 Report 2010-2013. Retrieved from http://www.braininstitute.ca/sites/default/files/obi_phase_1_report_english_2014.pdf
58 Grand Challenges Canada. (2016). Saving Brains: Unlocking the potential for development – Request for Proposals.
59 Grand Challenges Canada. (2015). Results: The First Five Years – Annual Report 2014-2015.
60 Grand Challenges Canada. (2014). Saving Brains Request For Proposal. Retrieved from http://www.grandchallenges.ca/wp-content/uploads/20140711_SB_R4_RFP_EN.pdf
61 Adams, O., et al. Oxford Policy Management. (2015). Development Innovation Fund : Health – Summative Evaluation Report. Oxford Policy Management. Retrieved from http://www.grandchallenges.ca/wp-content/uploads/DIF-H_SummativeEvaluation_web_10-20-15.pdf
62 Brain Canada Foundation. (2015). The 2015 Platform Support Grant (PSG) Sponsored Opportunity Request for Applications
Page details
- Date modified: