Evaluation of the Canada Brain Research Fund Program 2016-17 to 2020-21
Organization: Health Canada and the Public Health Agency of Canada
Published: 2022-06-03
Table of contents
- Executive summary
- 1.0 Evaluation purpose
- 2.0 Evaluation scope, approach, and design
- 3.0 Program profile
- 4.0 Findings: alignment with priorities
- 5.0 Performance: achievement of expected outcomes (leveraging resources, building partnerships, and knowledge translation)
- 6.0 Findings: Delivery model
- 7.0 Conclusion
- 8.0 Recommendations
- 9.0 Management Response and Action Plan
- Appendix 1: Comparative analysis chart
- Appendix 2: Impacts of two-year funding for contribution agreements
- Appendix 3: Evaluation methods and limitations
- Endnotes
List of acronyms
- ALS
- Amyotrophic lateral sclerosis
- BCF
- Brain Canada Foundation
- CAD
- Canadian Dollars
- CQDM
- Consortium de recherché biopharmaceutique
- CIFAR
- Canadian Institute for Advanced Research
- CIHR
- Canadian Institute of Health Research
- EDI
- Equity, Diversity, Inclusion
- GoC
- Government of Canada
- HC
- Health Canada
- INMHA
- Institute of Neuroscience, Mental health and Addiction
- IPSC
- Induced pluripotent stem cells
- ISED
- Innovation, Science, and Economic Development
- KT
- Knowledge Translation
- PIP
- Performance Information Profile
- PM
- Performance Measurement
Executive summary
Program profile
Through various budgets, the Government of Canada has committed a cumulative total of $200 million for the Canada Brain Research Fund Program within Health Canada to support Canadian neuroscience research, and to help the medical community better understand the brain and brain health. This program supports research with the greatest potential to lead to scientific discoveries that advance therapies and approaches to improve the health and the quality of life of Canadians at risk of, or affected by, brain health conditions. The Brain Canada Foundation (the BCF), the sole recipient of program funding, works in partnership with a range of donors from across the private and charitable sectors to provide competitively awarded funding for research across Canada.
The federal funding model requires the BCF to raise 1:1 matching funds from donors outside the federal government in order to support research projects. Examples of BCF's donors and partners include research institutions, health charities, and corporations.
The BCF aims to shape its research priorities by engaging with the neuroscience community and bringing research stakeholders together to discuss and advance key brain health issues and opportunities in the health sector. Its flagship programs, which include team, platform, and capacity building awards, are designed to fill gaps identified by stakeholders for increasing research capacity and strategically advancing the prevention, diagnosis, and treatment of brain health disorders.
Conclusions
There is a continued need for research in the area of brain and neurological health, given that the current financial burden of disease associated with brain and neurological health conditions and disorders reaches into the tens of billions of dollars annually, and is only increasing over time.
Through the BCF, the Program addresses research and policy priorities identified by brain research stakeholders, either as a shared priority or by engaging with research partners to conduct research in specific priorities of interest. Over the course of the period covered by the evaluation, the BCF has more than doubled the federal funding provided through donations leveraged from private and non-governmental sources, surpassing the 1:1 matching model.
The BCF is a well-established organization and its knowledge and expertise in the brain-related health sector have helped to connect researchers with funders, as well as create partnerships and collaborations among researchers with diverse scientific and operational backgrounds. These relationships continue to support "high risk, high reward" research and contribute to enhancing the brain-related research ecosystem in Canada:
- Funding partners have indicated that their collaboration with the BCF has helped them support quality research they may not have otherwise pursued, especially when they were lacking scientific or operational expertise in the sector. In addition, the BCF has met or exceeded most of its performance targets related to capacity building, demonstrating that the Program is contributing to the development of researchers within the ecosystem.
- Some research funded through BCF partnerships is starting to be translated into intellectual property, demonstrating that intermediate outcomes are being achieved.
As noted in key informant interviews, Health Canada's Canada Brain Research Fund Program has developed a positive and constructive relationship with the BCF, resulting in a well-managed contribution agreement. Given the maturity of the BCF, a review of the terms and conditions of the contribution agreement between the BCF and Health Canada could reduce process irritants, which will further support success, as well as measure and communicate impact more effectively in support of decision making.
Expanding the BCF's partnership reach internationally could result in more opportunities to fund research that increases capacity for brain research in Canada, promotes access for Canadian researchers to research platforms in other countries, and shares knowledge generated by Canadian-produced research more broadly. As such, supporting international collaborations with BCF-funded researchers could both increase the expertise available to Canadian brain research efforts, and broaden the dissemination and use of knowledge generated by BCF funding partnerships at the international level.
Recommendations
The Canada Brain Research Fund Program's 1:1 matching funding model has added value to the brain research ecosystem, and has provided support for addressing the financial burden of brain and neurological diseases in Canada. The following recommendations focus on areas that could support continuous improvement in this area:
Recommendation 1: Where possible, consider any modifications to program parameters or their interpretation in order to enable greater opportunities for collaboration and potential international research partnerships.
The current terms and conditions of the Program, as well as the contribution agreement have presented challenges to creating international partnerships. This is an important matter to resolve as forging these international partnerships has the potential to increase capacity for brain research in Canada, promote researcher access to research platforms, and to share knowledge generated by Canadian-produced research more broadly.
Recommendation 2: To increase the likelihood of outcome achievement and to measure and communicate the Program's impact for Canadians and support decision-makers more effectively, consider consulting with partners, researchers, and institutions in the brain-related research ecosystem to identify optimal approaches to performance measurement and knowledge translation that recognize:
- a. the nature of a networked research ecosystem; and
- b. the complex nature of multi-disciplinary research.
Throughout the brain and neurological research ecosystem, challenges have been identified in capturing performance data that is meaningful to funders and administrators. As well, optimal approaches for translating research knowledge to achieve intermediate and ultimate impacts are not generally agreed upon, nor applied throughout the ecosystem. Notably, there is a lack of consensus as to how to address these challenges.
1.0 Evaluation purpose
The purpose of the evaluation was to examine the extent to which Health Canada's Canada Brain Research Fund Program (the Program) and Brain Canada Foundation (BCF) have achieved expected outcomes. Although the scope of the evaluation includes BCF activities during the period of 2016-17 to 2020-21, case studies that were initiated prior to this period were taken into consideration, owing to the long time that it takes for funded research to be translated into measurable outcomes.
This was a scheduled evaluation, as per Health Canada's Five-Year Evaluation Plan 2021-22 to 2026-27, in accordance with the requirements outlined in the Financial Administration Act.
2.0 Evaluation scope, approach, and design
The scope of the evaluation covered the period ranging from April 2016 to March 2021, and included an examination of the Program's contribution to support BCF objectives. The evaluation focused on BCF activities related to research funding and the funding model itself. This entailed an examination of the Program's outcomes, including the intermediate and long-term impacts of the BCF's funded research. The evaluation used multiple lines of evidence, such as literature and document reviews, performance data, and key informant interviews, to ensure triangulation of findings (see Appendix 3 for a detailed methodology, limitations, and mitigation strategy.)
3.0 Program profile
The Program is a named-recipient contribution program that provides support for neuroscience research through funding for the BCF. The BCF, which is the sole grants and contributions recipient of the Program, works in partnership with a range of donors from across the private and charitable sectors to provide competitively awarded funding for research across Canada. Program funding for the BCF supports the full spectrum of brain and mental health research, with a focus on "high-risk, high-reward" research through innovative collaborations. These types of projects come with a higher degree of risk, in that they propose new approaches that push the boundaries of current research paradigms. However, they also have the potential to result in significant discoveries and advancements.
The BCF is a national charitable organization that raises funds to foster advances in neuroscience discovery research, with the aim of enhancing the scientific community's understanding of brain health and improving care for those affected by brain health conditions, including neurological and mental health issues. The BCF receives annual contributions from the Program, as determined through federal budget allocations. Since 2011, this has totalled $130M in federal investments, for a total matched investment of over $260M. An investment by HC of $200M is projected by 2024-25. The BCF is required to raise funds to match, on a 1:1 basis, the federal investment in the Fund.
The BCF delivers its mandate through three main program lines, which reflect the types of funding competitions that award funding:
1. Team awards
Team awards support multidisciplinary research teams and aim to accelerate novel and transformative research that will change the understanding of nervous system function and dysfunction. These awards represent approximately 47% of funding disbursed over the period of the evaluation, with the average duration of each award being three years or more.
2. Platform support awards
The Platform Support awards are intended to facilitate and accelerate research by funding the maintenance and operation of major existing research platforms, providing the national or regional technical capability to multiple neuroscience investigators. These awards represent approximately 39% of funding disbursed over the period of the evaluation, with the average duration of each award being three years or more.
3. Capacity building awards
To promote the next generation of researchers, capacity-building awards support outstanding early and mid-career researchers undertaking original research related to brain health. These awards represent approximately 14% of funding disbursed over the period of the evaluation, with the average duration of each award being two years or more.
As well, the BCF undertakes a variety of Knowledge Translation activities in support of its mandate.
4.0 Findings: alignment with priorities
4.1 Current and projected need for brain health research
There is a continued need for research in the area of brain and neurological health, given the current and growing burden for brain-related diseases and disorders. With respect to mental health specifically, there is a strong link between poverty and addiction issues, along with neurological conditions affecting mental health. Brain and neurological health disorders include mental health diseases and disorders, neurodegenerative disorders such as dementia, Alzheimer's, Amyotrophic Lateral Sclerosis (ALS), Multiple Sclerosis, brain and spinal cord injuries, brain cancers, strokes, Parkinson's Disease Syndrome, trauma, and stress disorders, among others. Canadians are increasingly living with or caring for someone with one or more of these disorders, and the systemic and individual costs associated with prevention, treatment, and care are increasing over time. Estimates of the burden have not been completed consistently across all disease and disorder types for brain and neurological health disorders; the last time such an exercise was undertaken was in 2013Footnote 1, and the estimated total economic burden was calculated at $193B annually.
This burden is expected to increase. The diagnosis of mental health disorders, including mood disorders, such as depression, and psychotic disorders, among others, increased by 55% between 2007 and 2017. In 2016, it was estimated that the prevalence of Canadians living with a mental health disorder would increase between 2011 and 2041, from 4 million Canadians to 4.9 million.
With respect to neurological disorders, the growing population of aging Canadians is expected to increase the prevalence of neurodegenerative disorders, including dementia. By 2031, 25% of the Canadian population will be seniors, a significant milestone in this context, as aging is the most important risk factor associated with dementia. In 2016-17, more than 432,000 Canadian seniors (7%) aged 65 years and older were living with diagnosed dementia. Furthermore, as this number does not include those under age 65 who may have a young-onset diagnosis, nor those that have not been diagnosed, the true scope of dementia in Canada may be somewhat broader.
The increasing number of Canadians living with neurodegenerative conditions such as dementia creates a societal challenge, resulting in drastically increased costs for society, governments, and individuals. It is estimated that by 2036, 62% of the Canadian health care budget will be dedicated to senior care. Other rising costs associated with neurological/ and neuromuscular disorders include strokes ($3.6 billion) and Parkinson's Disease ($1.2 billion); other diseases and disorders, such as brain cancers, spinal cord injuries, and ALS, have had cost burdens calculated at a patient-level only, rather than at a systems level.
The BCF is in a position to help address this burden by being a focal point for investment in public, private, and voluntary research across the brain research ecosystem.
4.2 Addressing research priorities
4.2.1 Commitment to research in the ecosystem
Owing to the growing burden of disease associated with brain and neurological diseases, the Government of Canada (GoC) has made commitments to the brain and neurological health ecosystem to address the spectrum of issues in this area. Investments are being made in key areas, such as mental health and neurodegenerative diseases, to foster research within the ecosystem. For instance, the GoC has allocated specific amounts to support brain and neurological health research: in Budget 2021, $996M over five years was allocated to address mental health issues, and in Budget 2019, $50M over five years was allocated for the implementation of the national Dementia Strategy. From the perspective of addressing these GoC priorities, the BCF has funded research into the specific areas of mental health and neurodegenerative diseases, complementing the research interests of the GoC.
Another GoC commitment is the ongoing funding of the Canadian Institutes for Health Research's Institute of Neurosciences, Mental Health and Addiction (INMHA), which is finalizing a mental health strategy to address a gap in mental health research, and the CIHR Institute on Aging, which includes a focus on dementia and other neurodegenerative disorders. HC also continues to fund the Mental Health Commission of Canada ($14.25M per year) and the Canadian Centre on Substance Use and Addiction through contribution agreements. The Centre for Aging and Brain Health Innovation was funded with $123.5M over five years, from 2015 to 2020; however, funding has not been renewed. As well, the 2018 document "A Common Statement of Principles on Shared Health Priorities," focuses partly on mental health as a priority area of federal funding to provinces and territories to address an increasing need in that area.Footnote 2
It has been reported that Canada's science funding is the second lowest in the G7Footnote 3. Only 7.2% of its health budget is dedicated to mental health care, which may be considered as proportionally low in the context of the magnitude of the burden of disease that this represents in CanadaFootnote 4. Finally, as noted by the Canadian Brain Research Strategy, there is currently no national policy or coordination of brain and neurological health research to achieve collective goals within the ecosystem.Footnote 5
4.2.2 Addressing identified research and policy priorities
Key informant interviews, document review, and comparative analysis (Appendix 1) show that owing to its national mandate to fund research while working within the brain research ecosystem, the BCF's research priorities align with those of other groups operating within the that same ecosystem. This is because BCF establishes its research priorities in consultation with donors, partners, researchers and institutions. As well, the terms and conditions of the BCF's contribution agreement provide it with the flexibility to fund projects outside of its stated priorities, to meet emerging areas of research.
Importantly, the BCF supports the research conducted by other stakeholders in the research ecosystem through its mandate to match funding with partners on projects of mutual priority. In this way, the BCF is able to address the vast majority of research priorities in the ecosystem directly through co-funding. Furthermore, owing to the open call structure of its competitions, projects reflect the research interests of researchers and their institutions, which ensures that research interests and priorities are complementary or aligned between the BCF, its co-funding partners, and researchers who are awarded funding.
The only gaps in research priorities that were identified are priorities that address system issues, such as patient-oriented research, establishing sector-wide research and policy standards, or sector-wide strategic alignment within the ecosystem. Notably, addressing these issues would require sustained and broad-based engagement between willing key partners that would need to include a range of government and non-governmental entities. Otherwise, BCF funded research is inherently complementary to the research of its partners, as they only engage in partnerships in areas of mutual interest. This adds value to the ecosystem by ensuring that research priorities are addressed.
Research in brain health, especially in mental health and neurodegenerative disorders continues to be a funding priority for the GoC and HC. Through the BCF, the Program is addressing identified research and policy priorities across all brain research stakeholder groups, either as a shared priority or by engaging with partners to conduct research in that area. A small number of gaps have been identified through key informant interviews and document reviews.
4.3 Added value to the brain research ecosystem
Owing to its national mandate to conduct research while working with an extensive network of partners, the BCF adds value through its matching funding model and expertise in the sector. In key informant interviews, BCF funding partners noted that the value the BCF brings to the research ecosystem is largely due to its matching funding model and its knowledge and expertise in the sector. Among those partners operating in the not-for-profit sector and those representing private foundations, some key informants stated that they engaged in a funding partnership because the 1:1 matching funding model permitted them to undertake research funding that they would not have otherwise been able to. Key informants indicated that these funding partnerships helped them undertake great research, enabled in-depth research review, provided infrastructure for researchers, gave them access to many people with experience and expertise, as well as to the resources of other institutions. Also, key informants who were researchers noted that the leveraged, matched funding model allowed them to undertake more complex research projects than they could without the additional matched funding. Furthermore, other key informants noted that, although they were interested in funding research in the area of brain and neurological health, their lack of knowledge of the research ecosystems was a barrier to accessing it; they did not know who to partner with, nor how to navigate the research area. However, they found that the expertise and experience of the BCF gave them the confidence to create a meaningful funding partnership with a trusted organization. As one representative from a not-for-profit partner remarked:
"… and because we are not mental health experts and we do not want to do RFPs and run our own selection processes, we see them (the BCF) as a trusted partner with the expertise in their scientific advisory groups, and their board, and all of their different groups to actually ensure that what we fund is the best."
All of the key informants who were recipients of Team awards identified the leveraging of matching funding with the BCF as being the awards' main value. Furthermore, the recipients of Platform awards also noted that the leveraging of funding was the main value to them, as platforms require long-term investment with multiple partners to achieve sustainability over time. One key informant from a research institute provided an example of how the initial matched $1.2M funding from the BCF from 2015 to 2018 to create a research platform had attracted an additional $17M in funding from 2015 to 2020. More specifically, BCF funding contributed to the creation of the research platform, which drew interest from other funding sources, such as the pharmaceutical industry. The initial matched funding has a number of benefits, such as the means to hire additional researchers, including students. A further benefit identified by key informants is that both Team and Platform funding partnerships play a role in capacity building, as projects and platforms require teams of researchers, including early career researchers and students, to be successful.
On the subject of early career researchers, key informants also noted that the BCF Future Leaders awards in particular are unique within the funding ecosystem. Among the benefits identified, it was noted that this funding is dedicated to building capacity specifically in the area of brain and neuroscience research. To put the BCF awards into context, the following chart presents the only known neuroscience-specific awards for early career researchers:Footnote 6
Award | $ Amount and # of Grants (where available) |
---|---|
The BCF – Future Leaders in Canadian Brain Research Program | $100,000 for each of 20 awards, $2M total |
CIHR – Brain Star awards (with the Canadian Association for Neuroscience) | $1,500 to 15 trainees who published research relevant to CIHR-INMHA's mandate in 2020. |
Canadian Association for Neuroscience | Conference registration and $1000 for one candidate |
Not only is this dedicated funding for early career researchers significant, it was also noted by key informants that dedicated funding for brain and neurological health researchers and researcher teams is especially valuable since the overall number of researchers being funded by CIHR has been in decline over the past several years.Footnote 7
Although the vast majority of key informants believed that the leveraged funding model brought value to the ecosystem, three key informants believed that the BCF did not bring new funding into the brain research sector. Rather, they felt that the funding leveraged from other research partners would have entered into the ecosystem regardless of the BCF's matching model, although they were not specific as to how this funding would have made its way into the brain health ecosystem without the BCF. However, even these informants recognized that all funding dedicated to brain research is welcomed.
Another identified value of the BCF was its dedication to funding "high-risk, high-reward" research. For not-for-profit and private foundations, this approach was attractive to them as they sought not only to leverage their funding, but also to ensure that the research they fund achieves the highest level of impact possible, even in new or emerging areas where this level of impact was not guaranteed (the BCF approach matched the risk profile for research of its co-funders). Key informants from research teams noted that, apart from the BCF Team awards, there was little funding for high-risk research in the brain and neuroscience sector. Finally, the comparative analysis illustrates that the BCF focus on "high-risk, high-reward" research is complementary to the comparator institutions' focus on health systems, clinical and commercial translation, and public health aspects of research.
5.0 Performance: achievement of expected outcomes (leveraging resources, building partnerships, and knowledge translation)
5.1 To what extent have the outcomes been achieved?
Outcome #1: Leveraged resources
The BCF has been successfully leveraging funding according to the 1:1 matching model. This leveraging was not affected by the COVID-19 pandemic. Over the course of the period covered by the evaluation, the BCF has more than doubled the federal funding provided through donations leveraged from private and non-governmental sources, surpassing the 1:1 matching model. Thus, the BCF was able to match federal funds with sources outside of the federal government in line with expectations. Moreover, during the evaluation period, the number of partners and donors that have pledged funds to the BCF has increased by 311 %, from 101 in 2016 to 456 in 2020.Footnote 8 In total, 456 donors and partners committed funds to the BCF over the evaluation period. This increase in partnerships demonstrates the BCF's continued progress in attracting investment in high-risk, high-impact brain research. Thus, in addition to more than doubling the federal investment, the number of partners being engaged in the brain research ecosystem has been expanded.
Key informants saw the matching of donations by the federal government across stakeholder groups as a powerful incentive for those interested in investing in brain research, and taps into new sources of funding for this particular field. In particular, foundations associated with not-for-profit organizations have been attracted to the matching model that leverages resources, and the BCF's expertise in helping them navigate the research sector. In one notable example of a Platform award, a key informant noted that the matched funding from the BCF to create a research platform also attracted other funders who could sustain it.
"…and so there was some funding that followed on the Brain Canada grant. But if we hadn't gotten that first grant, there wouldn't be a platform to continue funding, so it would be a problem"
In addition, the current funding model and approach may have contributed to the BCF's resilience during the COVID-19 pandemic. Contributions from the Government of Canada can be understood as a 'floor', a minimum amount of funding available to the BCF to match with other organizations. Thus, BCF funding from matching partnerships remained consistent with that of previous years, even as donations from many other not-for-profit organizations fell dramatically in 2020-21Footnote 9. Indeed, for the BCF, partnership funding from corporations increased, while partnership funding from private donors decreased.Footnote 10
Beginning in fiscal year 2019, the BCF began to expand its fundraising activities, and in 2020, it engaged a full-time Director of Marketing and Communications. This is seen as a way to diversify sources of funding, increase the overall level of funding, and possibly have funding available beyond that provided by the Program. This further demonstrates the added value that the BCF brings to the ecosystem, in that it is continuing to find ways to attract funding from diverse sources to fund "high-risk, high-reward" research sustainably.
Of further note, the COVID-19 pandemic in early 2020 had a significantly negative impact on the funding landscape for not-for-profit organizations in Canada. The major impact on organizations in the not-for-profit sector was a dramatic decrease in donations, which resulted in a release of staff owing to funding shortfalls.Footnote 11Footnote 12 However, according to key BCF staff, the federal contribution provided the BCF with a funding floor, which allowed it to continue to engage in 1:1 matched-funding partnerships throughout the pandemic and continue disbursing awards. As well, it was able to increase its staffing levels during the pandemic. Although research funded by BCF partners was often delayed owing to, for instance, researchers being unable to access their labs, partnership building continued relatively uninterrupted during the evolving pandemic.
Case study #1: Human Inducible Pluripotent Stem Cells
As demonstrated in Case Study #1, Human Inducible Pluripotent Stem Cells (hiPSC) Platform, below, is an example of how the establishment of a diverse range of partners over a prolonged period was integral to leveraging additional funding. Over time, this support provided by this ongoing leveraging is leading towards translating the initial findings into a potential clinical practice.
Case study highlight #1: Human Inducible Pluripotent Stem Cells (hiPSC) Platform
Studies of genetic disorders are limited because of the lack of appropriate tissue samples from patients. A promising avenue to overcome this deficit is the novel approach of converting adult cells into inducible pluripotent stem cells (hiPSCs), and reprogramming them into desired cell types to support research.
Led by a research team from Université Laval and McGill University's Montreal Neurological Institute-Hospital, the hiPSC platform received a $1.5M Platform Support Grant from the BCF in 2014 to create a facility dedicated to producing cost-effective, high-quality hiPSCs. Additionally, the platform created hiPSCs from patients that would allow researchers to study the underlying biological mechanisms of different brain disorders, and test new therapies. This type of platform provides support for new and experienced researchers to conduct basic and translational research, as well as a training opportunity for students. Key project partners and donors included the Marigold Foundation, le Réseau de médecine génétique appliquée, McGill University, and the Quebec Pain Research Network. The platform contributed to the following outcomes:
Immediate outcome: Donors and partners invest in brain research
There were $4M in additional research grants following the completion of the 2014 Platform Support Grant, including several international research projects involving over 30 investigators, including Oxford University (UK) and the Karolinska Institute (Sweden).
Immediate outcome: Canadian researchers are equipped to undertake collaborative research on the brain and brain diseases and disorders
Not only did the hiPSC platform leverage funds, but also by providing a platform needed by researchers from a diversity of sectors, it enabled new collaborations that aligned academic research with pharmaceutical and biotechnology industries. The platform evolved from being a local platform in Quebec, to now serving researchers across Canada. Now known as the Early Drug Discovery Unit (EDDU), the platform has stimulated national and international collaborations with 45+ team members, has 60+ academic collaborators, has trained 200+ users, and has a growing number of industry partners.
Intermediate outcome: Stakeholders use knowledge to inform the development of prevention, diagnostic, therapeutic, clinical, technological, and health system solutions for brain diseases and disorders
By removing the obstacles many universities and companies face in accessing and sharing patient samples, the platform accelerated translational medical research and the development of new therapies for neurological disorders.
Outcome #2: Building partnerships and capacity building
The BCF has been able to foster added value in the brain research ecosystem by leveraging research partner funding, which then attracts other funders and partners to help sustain long-term projects beyond their original funding period.
Building research partnerships is at the very core of the BCF's research strategy, as collaboration and multidisciplinary research support the engagement of non-traditional disciplines as partners in the area of brain sciences. As well, they provide the opportunity to address research problems with multiple areas of expertise, offering different ways of conceptualizing and addressing the research problem.Footnote 13 Additionally, through these partnerships and collaborations, researchers can more efficiently pursue research topics of mutual interest, by reducing overlap and duplication of effort.
Moreover, partnerships between organizations have made it possible to avoid program duplication, and to scale up locally successful programs. Examples of research projects that were successful at the local and provincial levels, and were "scaled up" to the national level in partnership with the BCF include:
- The Quebec Parkinson Network, who partnered with the University of Calgary to develop the Canadian Open Parkinson Network, which is supported by the BCF.
- The researcher from the Induced Pluripotent Stem Cells (IPSC) platform first applied at the provincial level and then turned to the BCF for funding. The BCF partnership made it possible to develop and expand the platform. This platform is now available throughout the Canadian research community.
More than 35 institutions from across Canada have partnered with the BCF and their resources have been leveraged to conduct research. Many of the key informants found the collaborative approach of the BCF to be a positive approach and one of the fund's distinguishing features. Key informants unfamiliar with the brain research sector felt that the BCF shared scientific and operational expertise that enabled their organizations to engage in partnerships in the sector.
To achieve its objective of securing federal funding through a 1:1 matching structure, the BCF has received funding from the following donor groups: research institutions (36%), health charities (33%), other organizations (11%), provincial agencies (8%), public and private foundations (7%), corporations (3%), research networks (1%), and private donors (1%).Footnote i
Furthermore, a few key informants viewed the BCF as having a convening role by bringing researchers and funders together. Their role in working with stakeholders in 2017 to establish mental health research priorities is one example, and in March 2020, they convened a Youth Mental Health Workshop.
As noted in Table 4 below, the BCF has met two of its performance targets related to capacity building, and is on track to meet the other one in the next fiscal year based on the trend in performance data collected under the Brain Research Performance Information Profile (PIP). The BCF's priority efforts to facilitate collaboration between researchers, especially inter-departmental and interdisciplinary collaboration, have contributed to an increase in capacity building.Footnote 14 Note that performance information is not collected for all indicators in each year.
Indicator | Target Value | Actual Data | |||||
---|---|---|---|---|---|---|---|
2015-16 | 2016-17 | 2017-18 | 2018-19 | 2019-20 | 2020-21 | ||
Number of researchers using platforms enhanced by the BCF funding | 1340 researchers | Not collected | Not collected | 180 | 1061 | Not collected. | 1369Table 2 Footnote * |
Percentage of trainees (or trainee supervisors) reporting that the BCF contributed to the trainees' careers | 90% | Not collected | Not collected | 97% | Not collected | 99% | Not collected |
Percentage of new or mid-career highly qualified personnel retained in areas related to brain research two years after the BCF training opportunity | Maintain PhDs at 80%+ and post-docs at 80%+ | Not collected | Not collected | PhDs – 88% Post-docs – 93% | Not collected | PhD's – 86% Post-doc – 96% | Not collected |
Percentage of participants in capacity building activities who report that the activities catalyzed new research collaborations and provided opportunities to build relationships and develop collaborations with fellow participants | Maintain at 80%+ | Not collected | Not collected | 81% | Not collected | 89% | Not collected |
|
Some of the key informants specifically mentioned the importance of the BCF's Future Leaders program in capacity building. In general, they felt that the capacity-building program helps support the development of a new generation of researchers who currently are without experience. From this experience, they can later apply for other grants that require experience (e.g., CIHR grants) and continue with their careers. As one key informant noted:
"I think where those awards are effective is that they are seen by the community as being very prestigious and I think that is why people apply for them. And you know, they are also good funding for very early career researchers."
Case study #2: Learning in Machines and Brains
The Learning in Machines and Brains Program case study below is an example of how recourse leveraging leads to building collaborations and capacity in the research area. Specifically, it demonstrates how BCF funding attracts additional partners and collaborations, which in turn supports learning and capacity building throughout the ecosystem.Footnote ii
Case study highlight #2: Learning in Machines and Brains Research Program
The Learning in Machines and Brains research program, co-directed by researchers from l'Université de Montréal and New York University, funded with a total of nearly $6.5M, is revolutionizing the field of artificial intelligence (AI) by examining how artificial neural networks could be inspired by the human brain, and developing the powerful technique of deep learning. The number of researchers funded varied from year to year, and the highest number of researchers was 47 in 2017. These researchers were located in seven countries representing 31 institutions. The key partner for this project was the Canadian Institute for Advanced Research (CIFAR).
Immediate Outcome: Donors and Partners Invest in Brain Research
This project brought together five researchers and nine interdisciplinary collaborations, and the BCF's funding of approximately $3.8M helped leverage at least an additional $13M from Google and $9M from Facebook.
Immediate Outcome: Canadian researchers advance knowledge on the brain and brain diseases and disorders
Some highlights of the knowledge translation results of this program include:
- 421 program-related publications, including one highly cited publication (Deep Learning), which holds 9195 citations.
- Citation analysis from the Deep Learning publication shows three clinical studies and 28 citations for patents using knowledge generated from this Program.
- Twelve CIFAR program meetings where fellows in each of the research programs came together to share research and exchange ideas.
- Six (Deep Learning and Reinforcement Learning Summer School sessions that bring together graduate students, post-doctorates, and professionals to cover the foundational research, new developments, and real-world applications of deep learning and reinforcement learning.
Intermediate Outcomes: Research stakeholders use research findings to inform future brain research (funding, agenda, community)
Researchers from this program published findings on deep artificial neural networks (ANNs), which represent today's most accurate models of the brain's visual stream. In turn, other researchers have used this research to demonstrate how today's ANN models could be used in non-invasive clinical settings to study the brain in detail. This work has the potential to influence clinical applications for various diseases, including depression.
The ability to attract partners from a variety of backgrounds and research sectors, who collaborate across various disciplines, is one way in which the BCF has encouraged research that has expanded beyond its original scope.
Outcome #3: Knowledge translation
The BCF has demonstrated immediate and intermediate impacts of its funded research. The intermediate impacts are demonstrated by the case studies presented in this report. The BCF funds projects across the entire spectrum of research, although only a small portion of its funds is dedicated to funding knowledge translation and exchange projects. Funding that supports knowledge translation activities is put towards projects that encourage knowledge users, such as health care professionals and institutions, governments, and non-governmental organizations (NGOs), to translate research findings into policy, program, and practice changes. Knowledge translation is partly accomplished through the long process of generating and disseminating knowledge from research, influencing new research, and finding applications for the research. Stakeholders use knowledge to inform the development of prevention, diagnostic, therapeutic, clinical, technological, and health system solutions for brain diseases and disorders. Over time, BCF funded research partnerships have resulted in a number of patents, licences, and intellectual property rights registrations.
With respect to achieving its immediate and intermediate outcomes, the BCF has performed very well against its performance targets, as noted in the table below:
Indicator | Target Value | Actual Data | |||||
---|---|---|---|---|---|---|---|
2015-16 | 2016-17 | 2017-18 | 2018-19 | 2019-20 | 2020-21 | ||
Immediate Outcomes | |||||||
Number of distinct Canadian BCF-funded authors publishing in the area of brain research | 1,100 authors | Not collected | Not collected | 963 | 980 | 1053 | 1111 |
Number of publications resulting from the BCF investment | 1550 publications | Not collected | 409 | 739 | 1053 | 1251 | 1413 |
ntermediate Outcomes | |||||||
Number of citations of BCF-funded publications by other authors, both from Canada and from other countries | 45,000 citations | Not collected | Not collected | 9,981 | Not collected | 35,938 | Not collected |
Number of patents, licences, and intellectual property rights registrations resulting from BCF-funded research | 50 | Not collected | Not collected | 27 | 36 | 39 | 48 |
|
While these findings show the advancement of knowledge about brain and brain disorders from research, key informants mentioned that more needs to be done to capture the broader impact of the BCF's added value on the research ecosystem. In order to increase knowledge translation, some external key informants made the following suggestions: include board members from a broader range of research backgrounds (e.g., social scientists, engineers), provide more business and commercial knowledge translation support, integrate knowledge translation strategies into funding applications, and have specific knowledge translation awards. To implement some or all of these activities in a systematic way, the BCF could look to the various approaches adopted by similar research organizations that have funded knowledge translation business lines, such as those presented in the comparative analysis (see Appendix 1).
Furthermore, the BCF has generated 59 knowledge products to support knowledge translation efforts. According to the BCF survey sent to their funding recipients, 89% of the participants reported that capacity-building activities catalyzed new research collaborations and provided opportunities to build relationships and develop collaborations with fellow participants.
Case study #3: Identification of Aβ Plaques in Human Retina for the Diagnosis of Alzheimer's Disease
The Identification of Aβ Plaques in Human Retina for the Diagnosis of Alzheimer's Disease (AD) case study below is an example of how resource leveraging leads to building collaborations and capacity in the research area. Specifically, it demonstrates how BCF funding of research platforms attracts additional partners and collaborators. These partnerships and collaborations are necessary to sustain the financial viability of platforms, as well as to encourage capacity building throughout the ecosystem.
Case study highlight #3: Identification of Aβ Plaques in Human Retina for the Diagnosis of Alzheimer's Disease (AD)
Led by a research team at McGill University's Montreal Neurological Institute-Hospital, in collaboration with Optina Diagnostics, this neurodegenerative research project has resulted in a new eye scan that could revolutionize the early detection and diagnosis of AD. The research team was awarded $1.5M in 2015 through a Focus on Brain grant from Brain Canada and the Consortium de recherche biopharmaceutique (CQDM). CQDM and OBI co-funded the project with Brain Canada. The project was supported by CQDM through Merck, Pfizer, MESI, and BL-NCE.
Immediate outcome: Donors and partners invest in brain research
As with all research supported by the BCF, this was a collaborative effort between several partners who not only contributed funding and expertise, but attracted additional donors and partners.
Immediate outcome: Canadian researchers advance knowledge on the brain and brain diseases and disorders
Additionally, the partnership between the research team and Optina Diagnostics, a company that specializes in the development of non-invasive technologies to detect disease, accelerated the project from research into development.
Intermediate outcome: Stakeholders use knowledge to inform the development of prevention, diagnostic, therapeutic, clinical, technological and health system solutions for brain diseases and disorders
The technology received Breakthrough Device Designation from the U.S. Food and Drug Administration in 2019, which will streamline the market clearance and approval process to put the device to use in clinics, as well as a 510(K) clearance from the FDA in 2020. The technology is also nearing its final commercial approval for use in Canada. These early approvals will enable partnerships with eye clinics that can conduct these eye scans in at-risk populations.
With the BCF's focus on building funding partnerships to support collaborations in high-risk, high-reward areas of research, value can be created by expanding on the fields that are traditionally part of the brain research ecosystem. This has the potential to create new knowledge that can, in turn, generate new applications over time.
5.2 Translation of research to clinical application
With an average time horizon of 17 years to translate basic research into clinical applications and other intermediate impacts, BCF case studies are demonstrating promising progress in achieving these impacts.Footnote 15 Below are three case studies that are demonstrative of the long-term horizon of achieving outcomes for the BCF-funded research. In total, eight case studies were examined for this evaluationFootnote 16, including case studies of research awards that precede the evaluation period, reflecting the long time horizon of translating research into applications.
Project | Help for Kids with Injured Brains Led by a team from the Hospital for Sick Children in Toronto, which received a 2012 grant totalling $1.5M. The purpose was to determine whether metformin, a type 2 diabetes drug, and physical exercise could stimulate the recruitment of healthy brain cells, and in doing so, promote repair in the injured brain. To enhance the translational potential of their research, the studies were conducted in both mice and humans, with a particular focus on brain repair in children with acquired brain injury. |
Partnership for Stroke Recovery Advances in stroke treatment have increased survival, but have resulted in more people living with chronic disability. Research emphasis has shifted to treatments to enhance recovery. Community-based exercise programs for those living with stroke have been identified through prior research as being a promising approach to treatment. This project is intended to scale up the implementation of sustainable and evidence-based community exercise programs for those living with stroke, and to measure the impact of uptake. The initial grant award, in partnership with the Heart and Stroke Foundation and the Canadian Partnership for Stroke Recovery, was $1.4M. |
Early Years Program The Early Years (EY) Program is a project funded by Brain Canada in partnership with the Martin Family Initiative (MFI) that aims to improve outcomes for pregnant Indigenous women and their children. The Early Years program is providing families with the tools to uplift their children in ways that promote resiliency and attachment, foster early language development, and establish overall wellness. The initial grant amount provided to the Martin Family Initiative by the BCF was $3.5M. |
---|---|---|---|
Immediate Outcomes | In 2016, the research led to a number of notable findings from both their animal and human studies, resulting in 13 publications in several high-impact journals, including Science, Cell Stem Cell, and Neuron. Their findings that metformin could enhance brain repair and functional recovery in mice, and exercise could affect brain structure and cause a significant increase in cognitive function in children with acquired brain injury, set the stage for their human work with metformin. |
Involving more than 300 Canadians in 10 cities, the study found that using technology to delivery post-stroke therapy exercise programs can be as effective as traditional care. This is especially important for delivering post-stroke therapy to people in rural and remote parts of Canada. |
The project has the potential to demonstrate how to effectively support Indigenous child well-being in a holistic and outcome-driven way. Preliminary qualitative and quantitative data from the first 18 months of this project already shows evidence of participant and community engagement, social and health service navigation and access, as well as parental support and mitigation of stress. |
Intermediate Outcomes | In 2019, three years following the completion of the grant, findings were published in the open-access journal, Science Advances, showing that metformin promoted cognitive recovery in a mouse model of neonatal stroke, and had sex-dependent effects. The research continues to make an impact: currently, three clinical trials have stemmed from the original discovery, including one exploring the impact of metformin on recovery in children with acquired brain injury caused by tumour radiation as part of treatment. In 2021, the Multiple Sclerosis Society of Canada announced $400,000 in funding to support a pilot clinical trial to investigate the use of metformin as a therapy for children and young adults with multiple sclerosis (MS). In partnership with Stem Cell Network and Ontario Institute for Regenerative Medicine, this investment increases the funding for that trial to $1 million. Although the BCF is not involved in this project, this demonstrates how the knowledge is now being taken further toward clinical treatments. |
These insights have helped inform the rapid increase of tele-rehabilitation during the pandemic. As noted by a key informant, two collaborating studies are currently underway (with one in its initial phase) and four more have been identified. All studies will continue to test new approaches, therapies, therapeutics, and technologies to improve stroke recovery. These four studies will start once research resumes post-pandemic. |
Early successes have been recognized by other Indigenous groups, with plans to expand the program to one community in Nunavut in 2020, as well as to four communities in Yukon and nine in British Columbia in 2021. So far, additional funding of $12M has been raised. |
6.0 Findings: Delivery model
6.1 Best practices and lessons learned
6.1.1 Equity, Diversity, and Inclusion
The BCF has been incorporating Equity, Diversity, and Inclusion (EDI) best practices into its processes through policy actions, which are having an impact on the diversity of its applicants. Its approach to EDI practices is in line with acknowledged best practices of the Canada Research Coordinating Committee; however, it should be noted that EDI is still an emerging administrative practice, and it will be necessary for the BCF to remain informed of emerging developments.Footnote 17
In 2018, the Government of Canada began to prioritize equity, diversity, and inclusion (EDI) practices in the broader research ecosystem through the Canada Research Coordinating Committee.Footnote 18 Furthermore, the goals for EDI practices have been articulated through CIHR's work in its Sex and Gender-Based Analysis Action Plan, and through Health Canada's Sex and Gender Action PlanFootnote 19. Overall goals for the EDI action plans entail supporting equitable access to funding, promoting EDI considerations into research design, increasing equity and diversity in research teams, collecting appropriate performance, and undertaking the data analyses needed to include EDI considerations into decision making. Another best practice not included in the Action Plans is promoting institutional accountability. This practice is a characteristic of the Canada Research Chairs Program.Footnote 20
Since 2018, the BCF has taken specific actions to address EDI concerns at the pre-award, peer review, and post-award phases of its program competitions. In 2018, 75% of BCF-funded programs had EDI considerations incorporated into their processes; by 2019, this had increased to 100%.
The longer-term changes resulting from the internal policies have not yet been captured through performance measures, which have only recently been developed; however, data is already being collected and reported.
The following table captures acknowledged best practices from CIHR's Action Plan for EDI, as well as the Canada Research Chairs program accountability practices, and the actions that the BCF has undertaken to address these practices:
Best Practice | BCF EDI Practices |
---|---|
Support equitable access to funding. |
|
Promote EDI considerations into research design. |
|
Increase equity and diversity in research teams – ensuring that research teams are diverse. |
|
Peer review |
|
Institutional accountability |
|
6.1.2 Length of agreement
Starting in 2017, the funding period for the BCF was reduced from five years to three years; this was further reduced to two years in subsequent contribution agreements. The change in the duration of the contribution agreement was in anticipation of the creation of the Strategic Science Fund (SSF), which was announced in Budget 2019 as the new approach to funding third-party science and research organizations. Although the fund launched in summer 2021, its first disbursements are not expected to begin until 2024-25.
The current two-year agreement length has created administrative challenges that were not present under the five-year agreements. As shown in Appendix 2, challenges have arisen in allocating funds for programs, owing to the one-year period that it takes to approve a new contribution agreement and the one-year period that it takes to design and implement a research program partnership. To some extent, the Program and the BCF have been able to work around this limitation. For instance, staggering program and partner matched funding so that the BCF's funds are spent for the first two years of a research program's duration, and then donor funds are subsequently spent. However, the two-year funding timeline may still lead to lapsed funding. Another consequence of the two-year agreements is that the length of project duration appears to have been reduced. As demonstrated in Appendix 2, this is reflected in program data, in which the length of research programs has declined since 2017; partnerships lasting five years or more are no longer being undertaken, suggesting that projects of a certain scope are no longer being considered for funding.
Furthermore, as revealed in interviews and other documents, the two-year horizon has limited the opportunity to pursue some partnerships with private foundations or registered charities, owing to their legal obligations under the Income Tax Act that restrict how their donations can be used (i.e., preventing the donors from donating in the outer years of the agreement).
Recent findings and recommendations from evaluations of other science-based research-related initiatives, highlighted below, have noted that even a five-year time horizon for funding in the research environment may not be adequate to support either fundamental or translational research properly through to the knowledge translation phase of a project. This also has implications from a performance measurement perspective, as the shorter timeframe of projects means that data collection and reporting focus on outputs and immediate outcomes, rather than intermediate and ultimate outcomes. This presents a clear challenge in measuring these impacts and assessing the long-term successes of the Program. For example:
- In 2020, the evaluation of the Genome Research and Development Initiative, notes that the "Five-year lifecycle of SPPs [Shared Priority Projects] is too short to demonstrate socioeconomic outcomes of the scientific research," and therefore poses challenges for long-term measurement and translationFootnote 21; and
- In 2019, an evaluation of the Futurpreneur research program concluded that "ISED should examine longer-term funding contribution options to support program stability and Futurpreneur's progress towards intermediate and ultimate outcomes."Footnote 22
6.1.3 Funding parameters
Key informants have noted that the funding mechanism has been beneficial for the BCF. However, there are improvements that can be made to promote the BCF's successes further. As discussed previously in this report, the matching funding model is particularly appealing to its range of funding partners and donors, including those from the private sector and not-for-profit foundations. As noted in key informant interviews, it is this matching funding that helped to attract them to the BCF and the sector. In particular, the matching model is also a way of supporting other not-for-profit organizations to undertake research funding on a larger scale than they could have through their own funding.
Key Informants in this study note that Program funding provides a measure of stability to the BCF.
Key informants have also noted that the funding provided by the Program provides a measure of stability to the BCF. First, as noted by researchers from other institutions, the Program's contribution represents "fenced funding" specifically dedicated to brain research in Canada. This ensures that there is a minimum level of brain-related research, and its related benefits to capacity building, available annually to the ecosystem.
Although refinements to the Program's parameters have been made over the past several years, two notable areas for improvement include:
- International collaborations: Although establishing funding partnerships is a priority for the BCF to build capacity, leverage resources, and translate its knowledge, progress in this area has proven challenging when it comes to partnering with international organizations and their donors. This is due to a lack of clarity in the Program's terms and conditions that prevents the allocation of funds for research conducted outside of Canada. The BCF also faces challenges with tracking and monitoring these funds with the same standards and practices as applied to Canadian donors. As noted by key informants, including researchers, forging international partnerships has the potential to increase capacity building for brain research in Canada, to promote researcher access to research platforms, and to share knowledge generated by Canadian-produced research more broadly.
- Knowledge translation: Although the BCF has partnered to fund some knowledge translation (KT) programs, and does engage in certain KT activities, these are ad hoc efforts and are not organized around a central approach or strategy. KT, however, is a key factor that supports the achievement of higher-level impacts, such as the development of intellectual property from basic research. During key informant interviews and in the comparative review, it was noted that a few other research institutions active in the brain sector and broader research sectors specifically fund a KT business line. These business lines exist to assist systematically with the translation of research into clinical and patient-oriented results or intellectual property. An expanded or more systematic approach to KT by the BCF could support the achievement of wider KT impacts. An initial approach to adopting a more formal KT strategy could be to embed KT throughout BCF research competitions and corporate processes. Another approach may be to fund a separate business line. However, current program parameters only allow 10% of funds to be dedicated to operational costs, so additional funding or changes to program parameters may be needed to fund a dedicated business line. This approach of funding a KT business line, however, may not be attractive to the BCF matching funding partners, which engage with the BCF to fund basic research.
Thus, certain elements of the Program's terms and conditions for the funding agreement, especially the short duration and provisions for collaborations between the BCF and potential partners, may have affected the BCF's ability to achieve their desired outcomes. Apart from these challenges, the matching funding model has helped the BCF to add value to the brain research ecosystem.
6.1.4 Performance measurement
Interviews with key informants from other government agencies, academic and other institutions, foundations, and not-for-profit organizations have demonstrated that the current approach to performance measurement and reporting undertaken by the Program with the BCF is appropriate in scope, and similar to the outcomes measured in their own organizations. Likewise, interviews and reviewed documents demonstrate that the approach to encouraging and measuring Equity, Diversity, and Inclusion (EDI) outcomes is matching practices being applied by other not-for-profit and research institutes. However, interviewees expressed a degree of dissatisfaction with the overall robustness of performance measurement in the sector as a whole. The general sense is that current methods and approaches are not capturing the full value of collaborations and impacts achieved through brain-related research and its translation to higher-level impacts. However, key informants had limited solutions to offer.
The current approach to performance measurement taken by the Program and organizations operating in the ecosystem is that knowledge creation and its translation into new knowledge is what creates and sustains the relationships between the multiple stakeholders in the research ecosystem. Indeed, this is consistent with the Program's current Logic Model. While dissatisfied with their respective organizations' current set of performance measures, three of the interviewees indicated that their current approach to developing new measures involves broad-based consultations with participants in the ecosystem: researchers, donors, funders, clinicians, patients, and so on. Just as research priorities for these organizations are established through these types of consultations, it appears there is a growing acknowledgement that partners and stakeholders who are involved in setting priorities should also be engaged in establishing performance measures.
This approach has the potential to capture and reflect the values and interests of those who are consulted, from patients to researchers to funders. In this way, it may be possible to capture performance and impact information that is relevant to the decision-making needs of the different groups involved in the consultation. This further reflects the nature of working in an ecosystem composed of different networks of individuals and institutions that are creating and disseminating knowledge in ways that are not necessarily being captured by current approaches to performance measurement.
7.0 Conclusion
The BCF's matching funding model and focus on supporting "high-risk, high-reward" basic research have produced research that is recognized for its excellence in the brain-related research ecosystem. Furthermore, this research addresses the research priorities of its funding partners and donors, as well as the funding priorities of the GoC. This is especially important as the financial burden of disease associated with brain and neurological health conditions reaches into the tens of billions of dollars annually, and is only increasing over time.
Furthermore, funding partners and donors have indicated that collaboration with the BCF has helped them support quality research that they may not have otherwise pursued, especially funding partners unfamiliar with brain-related research who lacked scientific and operational expertise. On top of this notable accomplishment, the BCF has met or exceeded most of its performance targets related to capacity building, demonstrating that the Program is contributing to the development of researchers in the ecosystem. Finally, some research that is being funded through BCF-funded partnerships is starting to be translated into intellectual property, demonstrating that intermediate outcomes are being achieved.
As noted in key informant interviews, the Program has developed a positive and constructive relationship with the BCF resulting in a well-managed contribution agreement. Given the maturity of the BCF, a review of the terms and conditions of the contribution agreement between the BCF and the Program is suggested to determine how to reduce process irritants, which will further support success, as well as measure and communicate impact more effectively in support of decision-making. Expanding the BCF's partnership reach internationally could result in more opportunities to fund research that increases capacity for brain research in Canada, promote the access of Canadian researchers to research platforms in other countries, and share knowledge generated by Canadian-produced research more broadly. As such, supporting international collaborations of BCF-funded researchers could both increase the expertise available to Canadian brain research efforts and broaden the dissemination and use of knowledge generated by the BCF funding partnerships at the international level.
8.0 Recommendations
The Canada Brain Research Fund Program's 1:1 matching funding model has added value to the brain research ecosystem, and has provided support for addressing the financial burden of brain and neurological diseases in Canada. The following recommendations focus on areas that could support continuous improvement in this area:
Recommendation 1: Where possible, consider any modifications to program parameters or their interpretation in order to enable greater opportunities for collaboration and potential international research partnerships.
The current terms and conditions of the Program, as well as the contribution agreement have presented challenges to creating international partnerships. This is an important matter to resolve as forging these international partnerships has the potential to increase capacity for brain research in Canada, promote researcher access to research platforms, and to share knowledge generated by Canadian-produced research more broadly.
Recommendation 2: To increase the likelihood of outcomes achievement and to measure and communicate the Program's impact for Canadians and support decision makers more effectively, consider consulting with partners, researchers and institutions in the brain-related research ecosystem to identify optimal approaches to performance measurement and knowledge translation that recognize:
- a. the nature of a networked research ecosystem; and
- b. the complex nature of multi-disciplinary research.
Throughout the brain and neurological research ecosystem, challenges have been identified in capturing performance data that is meaningful to funders and administrators. As well, optimal approaches for translating research knowledge to achieve intermediate and ultimate impacts are not generally agreed upon, nor applied throughout the ecosystem. Notably, there is a lack of consensus as to how to address these challenges.
9.0 Management Response and Action Plan
Recommendations | Response | Action plan | Deliverables | Expected completion date | Accountability | Resources |
---|---|---|---|---|---|---|
Recommendation as stated in the evaluation report | Identify whether program management agrees, agrees with conditions, or disagrees with the recommendation, and why | Identify what action(s) program management will take to address the recommendation | Identify key deliverables | Identify timeline for implementation of each deliverable | Identify Senior Management and Executive (DG and ADM level) accountable for the implementation of each deliverable | Describe the human and/or financial resources required to complete the recommendation, including the source of resources (additional vs. existing budget) |
Optimizing the funding agreement
|
Health Canada program management agrees with the recommendation. |
December 2021 - March 2022 – Building on previous engagement with Health Portfolio experts, the program will consult with Treasury Board Secretariat and assess how current Program parameters support the recipient's participation in international research partnerships and collaboration. December 2021 - March 2022 – The Program will engage relevant functional and subject matter areas within the Health Portfolio to validate potential provisions to be included in the current funding agreement, which is set to be extended until 2024-25. |
Funding agreement reflects implementation parameters that allow international research partnerships. |
March 31, 2022 |
Director General, Health Programs and Strategic Initiatives |
This recommendation will be completed using existing SPB human and financial resources. |
Updating approaches to Performance Measurement and advancing Knowledge Translation
|
Health Canada program management agrees with the recommendation |
December 2021 - March 2022 – The Program will engage relevant federal stakeholders to determine the best approach for increasing the program and BCF's focus on and support for knowledge translation (KT). January - March 2022 – Using information gathered through stakeholder engagement, the Program will assess and introduce appropriate Program measures to modify/expand KT efforts. April - November 2022 – In collaboration with BCF, engage relevant subject matter experts (i.e., PM and KT research sector, broader neuroscience sector, health portfolio) to inform strategies for PM and KT suitable for adoption over the final three years of the program (from 2022-23 until 2024-25). 2022-23– For improved measurement and communication of impact, the Program will collaborate with BCF in updating the program's Performance Measurement Strategy to include approaches identified through stakeholder engagement. |
Funding agreement which includes parameters to increase the Program's focus on and support for KT Summary of engagement findings on PM and KT with guiding recommendations Revisions to Performance Measurement Strategy |
March 31, 2022 November 30, 2022 May 30, 2023 |
Director General, Health Programs and Strategic Initiatives |
This recommendation will be completed using existing SPB human and financial resources. |
Appendix 1: Comparative analysis chart
Brain Canada Foundation | Institute of Neurosciences, Mental Health and Addiction (INMHA – CIHR) | Institute of Aging (IA – CIHR) | Ontario Brain Institute | Michael Smith Foundation for Health Research | Hotchkiss Brain Institute | |
---|---|---|---|---|---|---|
Funding Source |
|
On average, investments in research relating to INMHA's mandate represented 24% of total CIHR investments over the period from 2000-01 to 2014-15. Total investments in 2014-15 in this area were $229.6M. | Over the past 17 years, the average annual percentage of CIHR investment in IA mandate was 11% of CIHR's total annual investment. | Provincially funded by the Government of Ontario:
|
Funded by the Government of British Columbia:
|
In 2017-18, the Institute received more than $47M in total research funding. |
Research Priorities |
|
|
|
|
To inform and respond to emerging population health and health system issues. No specific brain or neurological focus. | Research goal will be reached with the introduction of a "NeuroDiscovery Framework", which aligns research within three themes of Brain and Behaviour (epilepsy, mental health and stress, neurodevelopment), Neural Injury and Repair (multiple sclerosis, spinal cord and nerve injury, and concussion research) and Healthy Brain Aging (dementia and cognitive disorders, movement disorders, and stroke). |
Scientific Direction (Mandate) | To understand the brain, in health and illness, to improve lives and achieve societal impact by:
|
To support research that:
|
To support research, to promote healthy aging and to address causes, prevention, screening, diagnosis, treatment, support systems, and palliation for a wide range of conditions associated with aging." Its goal is to improve the quality of life and health of older Canadians by understanding and addressing or preventing the consequences of a wide range of factors associated with aging. | Collaborative approach to research aims to:
|
|
Mission is to inspire discovery and apply knowledge towards innovative solutions for neurological and mental health disorders. The primary strategic research goal of the HBI is to achieve internationally recognized key discoveries and transformative clinical research in the neurosciences and mental health. |
Knowledge Management/ Knowledge Translation (KM/KT) Activities (Checklist) |
No specific KT Program, although it has funded projects with a KT focus. | No specific KT Program | No specific KT Program | The GEEK (Growing Expertise in Evaluation and Knowledge Translation) program provides funding, evaluation expertise, and support to community-led programs and services for people living with brain disorders. GEEK supports the sustainability, scaling up, and spread of these programs, to improve the quality and quantity of evidence-based care in the community. | Fully developed KT strategy, paired with an evaluation framework, and based on an understanding of complex systems developed to increase the effectiveness and efficiency of research. | Has a Knowledge Engagement business line, with a focus on Indigenous Research Support. This business line is offered to support community organizations to help establish and find research partners, support collaborations, build capacity, and mobilize knowledge. |
Appendix 2: Impacts of two-year funding for contribution agreements
Figure 1: Text description
This image depicts the various steps of the contribution agreements for project funding over the course of 6 years. At the bottom of the image, there is a timeline divided into 6-month segments, with numbers of months from 0 to 72. Under this timeline are the various steps of the funding process, including GoC Funding Announcement, GoC Funds Available, Research Program funds commence, Accountability/funding milestone (3 times), and Project conclusion. At the top of the image, above the timeline is a series of horizontal coloured bars covering various periods on the timeline, including, from right to llest a light purple bar for "Contribution Agreement approval process" (12 months), a sky blue bar for "Research program partnership; design; project review/approval" (12 months), a pale green bar for "Research Project Duration (3-year example)", and a green bar for "Optional 1-year extension". Above the 24-month and 60-month marks of the timeline is the text "Agreement disbursement deadline".
Figure 2: Text description
The image is comprised of a series of multi-coloured vertical bars that depict the length of research projects as they relate to the year of the funding competition.
- For 2016, according to the number of funding competitions launched that year, the length of research projects was, in descending order, 3 years, 4 years, 2 years, 5 years, 1 year, 6 years, and 7 years.
- For 2017, according to the number of funding competitions launched that year, the length of research projects was, in descending order, 2 years, 2 years, 3 years, 1 year, and 4 years.
- For 2018, according to the number of funding competitions launched that year, the length of research projects was, in descending order, 3 years, 2 years, 1 year, 4 years, and 5 years.
- For 2019, according to the number of funding competitions launched that year, the length of research projects was, in descending order, 3 years, 2 years, 4 years, and 1 year.
- For 2020, according to the number of funding competitions launched that year, the length of research projects was, in descending order, 2 years, 1 year, 3 years, and 4 years.
- For 2021, according to the number of funding competitions launched that year, the length of research projects was, in descending order, 2 years, 3 years, 1 year, and 4 years.
Appendix 3: Evaluation methods and limitations
The evaluation used multiple lines of evidence, such as a literature and document review, performance data, and key informant interviews to ensure triangulation of findings.
- Key Informant Interviews
OAE interviewed 39 key informants, including:
- Academic and Research Institutions (8)
- BCF Staff and Executives (7)
- Federally-funded research institutes (6)
- NGO funding partners (5)
- HC Program representatives (5)
- Private funding partners (5)
- Provincial research institutes (3).
- Document Review
-
OAE reviewed over 75 internal and public documents related to program delivery, including, but not limited to:
- BCF Annual Reports
- Briefing and Planning Documents
- Proposals and other foundational documents.
- Comparative Analysis
A comparison of the BCF to seven other brain-related research institutes (six Canadian and one international) was conducted based on eight factors. The analysis was based on a review of publicly available program documents and three key informant interviews.
- Administrative and Performance Data Review
A review of administrative and performance data was completed to assess progress towards stated objectives. Administrative data included program delivery information from the BCF databases, and performance data included a review of case studies and PIP data covering the evaluation period.
- Literature Review
Two reviews of relevant peer-reviewed and grey literature published between 2014-15 and 2020-21 were conducted with support from Health Canada's Health Library Information Branch (HLIB), in addition to open-source searches. A total of 41 articles were selected for inclusion.
As with many evaluations, there were some limitations encountered during the implementation of the selected methods that may have had implications for the validity and reliability of evaluation findings and conclusions. The following table provides a summary of the limitations, impact, and mitigation strategies to ensure the findings could be used with confidence to guide program planning and decision making.
Limitation | Impact | Mitigation Strategy |
---|---|---|
The sample size of key informants was relatively small, considering it was divided across several groups of stakeholders. | Key informant views may not be equally representative across groups due to the different distribution of key informants in each of the groups. | Information from key informant interviews was triangulated with key documents containing performance data. |
Key informant interviews are retrospective in nature and are sometimes limited to specific points in time (i.e., the duration of a specific project). | Interviews tend to provide recent perspective on past events. In some cases, key informants had been involved with the Program for a limited amount of time, such as new hires. | Triangulation with other lines of evidence substantiated or provided further information on data captured in interviews. Document review provided corporate knowledge. |
A few of the staff key informants either had left their previous postings or had recently joined, which affected their level of corporate knowledge to answer interview questions. | In a few cases, key informants did not have the same corporate knowledge and were not able to respond to some of the questions in the interview guide. | Key informants with corporate knowledge were included to balance the information received adequately. Triangulation with other lines of evidence such as the document review also provided corporate knowledge information. |
Endnotes
- Footnote i
-
Other Organizations are the Canadian Institute for Advanced Research (CIFAR) and Les Grands Ballets Canadiens. CIFAR contributed 99.8% of the funds to this category.
- Footnote ii
-
Note that HC's program authorities for funding of third parties mean that BCF can only distribute funding to principal investigators (PIs) in Canada; however, the PI is responsible for how their research team is composed and this may include some international researchers. In these circumstances, some researchers outside Canada may receive funding from the PI.
- Footnote 1
-
The National Population Study Health Study on Neurological Conditions was initiated in 2009 and completed in 2013. A similar survey has not been undertaken since. Retrieved from: https://www.canada.ca/en/public-health/services/chronic-diseases/neurological-conditions/national-population-health-study-neurological-conditions.html
- Footnote 2
-
Government of Canada. A Common Statement of Principles on Shared Health priorities. Retrieved from: https://publications.gc.ca/collections/collection_2017/sc-hc/H14-227-2017-eng.pdf
- Footnote 3
-
Canadian Association for Neuroscience. Science Funding in Canada. Retrieved from: https://can-acn.org/science-funding-in-canada-statistics/
- Footnote 4
-
Centre for Mental health and Addiction. Ending the Health Care Disparity in Canada. September 2018, p 4. Retrieved from: https://cmha.ca/wp-content/uploads/2021/07/CMHA-Parity-Paper-Full-Report-EN.pdf
- Footnote 5
-
Canadian Brain Research Strategy. Mission and Vision. "Canada's neuroscience community envisions a unified, national brain initiative: The Canadian Brain Research Strategy (CBRS). CBRS will bring together researchers and Canadians living with brain conditions from across the country to address a fundamental question." Retrieved from: https://canadianbrain.ca/mission-vision/
- Footnote 6
-
Sources include: https://braincanada.ca/news/2020futureleaders/; https://can-acn.org/brain-star-awards/; https://can-acn.org/awards/young-investigator/
- Footnote 7
-
Canadian Association of Neuroscientists. Science Funding in Canada. From: https://can-acn.org/science-funding-in-canada-statistics/
- Footnote 8
-
Brain Canada Foundation – Performance Measurement Report
- Footnote 9
-
Imagine Canada. Press Release: House of Commons calls on government to include support for charitable sector in budget 202. Retrieved from: https://www.imaginecanada.ca/en/360/house-commons-motion-charitable-sector-budget-2021 Note that this was repeated in other KII.
- Footnote 10
-
Brain Canada Foundation – Performance Measurement Report
- Footnote 11
-
Imagine Canada. Sector Monitor Report. Retrieved from: https://imaginecanada.ca/en/360/New-study-shows-challenging-recovery-for-some-Canadian-charities
- Footnote 12
-
Charity Village. https://charityvillage.com/new-data-reveals-stark-picture-of-covid-19-impacts-on-canadian-nonprofit-and-charity-staff/
- Footnote 13
-
CBF. https://www.canada.ca/en/health-canada/corporate/transparency/corporate-management-reporting/evaluation/2011-2012-2015-2016-contribution-brain-research-fund.html#a44
- Footnote 14
-
Brain Canada Foundation – Performance Measurement Report
- Footnote 15
-
Munro CL, Savel RH. Narrowing the 17-Year Research to Practice Gap. Am J Crit Care. 2016 May; 25(3):194-6. doi: 10.4037/ajcc2016449. PMID: 27134218.
- Footnote 16
-
Case studies sourced from the BCF and supplemented by key informant interviews, where available.
- Footnote 17
-
Government of Canada. See: https://www.canada.ca/en/research-coordinating-committee/priorities/equity-diversity-inclusion-research.html and https://cihr-irsc.gc.ca/e/50837.html
- Footnote 18
-
Government of Canada. Strengthening equity, diversity and inclusion in research. Retrieved from https://www.canada.ca/en/research-coordinating-committee/priorities/equity-diversity-inclusion-research.html
- Footnote 19
-
https://cihr-irsc.gc.ca/e/50837.html and https://www.canada.ca/en/health-canada/corporate/transparency/corporate-management-reporting/sex-gender-based-analysis-action.html
- Footnote 20
-
Note that the Canada Research Chairs program has established targets for equity goals, publicly reports on them by institution, and lists consequences for not meeting targets. Targets: https://www.chairs-chaires.gc.ca/about_us-a_notre_sujet/statistics-statistiques-eng.aspx#a3 Reporting: https://www.chairs-chaires.gc.ca/program-programme/equity-equite/Institutional-etablissements-eng.aspx Accountability: https://www.chairs-chaires.gc.ca/program-programme/equity-equite/consequences-eng.aspx
- Footnote 21
-
Goss Gilroy Inc. 2019-2020 GRDI Evaluation of Shared Priority Projects. Retrieved from: https://nrc.canada.ca/sites/default/files/2021-04/grdi-evaluation-report-2019-2020-e.pdf. P 24
- Footnote 22
-
Government of Canada. Evaluation of Futurepreneur Canada. Retrieved from: https://www.ic.gc.ca/eic/site/ae-vde.nsf/vwapj/future.pdf/$file/future.pdf P.4.
Page details
- Date modified: