Evaluation of the Centre for Aging + Brain Health Innovation (CABHI) Contribution Program 2015-16 to 2018-19

Figures and Tables

Figure 1

CABHI's Innovation Pipeline

Table 1

Activities and Achievements Produced from CABHI-funded Projects - 2015 to 2019

Table 2

Survey Results with Target Stakeholders re: Improvements of their Resources and Capabilities Attributed to CABHI

Table 3

Planned Spending and Expenditures of PHAC Contributions to CABHI 2015-16 to 2018-19 ($)

Table 4

Core Evaluation Issues and Questions

Table 5

Limitations and Mitigation Strategies

Table 6

Comparative Analysis

List of Acronyms

BHFP

Brain Health Foundations Program

CABHI

Centre for Aging + Brain Health Innovation

CCNA

Canadian Consortium on Neurodegeneration in Aging

CDKTN

Canadian Dementia Knowledge Translation Network

CICP

Canada-Israel Collaboration Program

GoC

Government of Canada

IA

Institute of Aging

IAC

Innovation Advisory Council

I²P²

Industry Innovation Partnership Program

KMP²

Knowledge Mobilization Partnership Program

MC²

Mentorship, Capital, and Continuation Program

O&M

Operations and Management

PMEF

Performance Measurement and Evaluation Framework

PHAC

Public Health Agency of Canada

RCP²

Researcher-Clinician Partnership Program

SAP

Seniors Advisory Panel

SCSIF

Senior's Care Strategic Innovation Fund

SGBA+

Sex- and Gender-Based Analysis Plus

SOF

Strategic Opportunity Fund

TEN

The Evidence Network

Executive Summary

Program Profile

The Centre for Aging + Brain Health Innovation (CABHI) is a collaboration of over 40 partners in the healthcare, science, industry, not-for-profit, and government sectors. It has funded projects in seven provinces and worked in collaboration with innovators, companies, and seniors' care organizations across five countries. Established in 2015, CABHI has received a total of $123.5 million over five years. These funds came from the Government of Canada (GoC), through the Public Health Agency of Canada ($42M); the Government of Ontario, through its Ministry of Research, Innovation and Science ($23.5M); the Baycrest Foundation ($25M); and other commercial and not-for-profit partners.

CABHI seeks to accelerate the development, validation, commercialization, and adoption of innovative products and new services that support brain health and aging, including dementia. CABHI provides funding and support for a variety of projects at various stages in the innovation pipeline (design, development, testing, and mobilization) through the following four main programs: Spark; Researcher-Clinician Partnership Program (RCP²); Industry Innovation Partnership Program (I²P²); and Mentorship, Capital, and Continuation Program (MC²). In addition, CABHI has a number of other strategic and regional programs that they fund, such as the Brain Health Foundations Program (BHFP), the Canada-Israel Collaboration Program (CICP), the Quebec Program, and the Saskatchewan Program.

Conclusions

In our examination of CABHI's first four years of operation, it is clear that CABHI has achieved notable success with respect to its short- and medium-term goals of improving the resources, capabilities, and performance of target stakeholders (i.e., healthcare providers, researchers and developers, companies, partner organizations). Not only has it exceeded performance targets in a number of areas, a number of new products, processes, services and new, refined, or improved care practices were developed and introduced into practice or brought to market as a result of CABHI.

CABHI's accomplishments and success to date can be traced back to a number of key factors:

• a multi-sectoral approach;
• a robust governance model;
• easy access to research experts co-located at Baycrest and a broad network of test facilities, which maximizes the ability to move projects through the various stages of the innovation pipeline (i.e., design, development, testing, mobilization);
• the leveraging of funding;
• a well-designed and fully implemented performance measurement framework; and
• a demonstrated commitment to continuous improvement.

Evaluation evidence in this report demonstrated that, in the aging and brain health communities, the work of CABHI is generally complementary rather than duplicative of the work of other players in these fields. Moreover, with its focus on dementia and the use of an innovation pipeline ending with commercialization, CABHI is very different, and according to many internal and external key informants, quite unique.

It is also worth noting that the evaluation found that CABHI has been quite cost-effective in its operations, in large part due to their affiliation with Baycrest, which has allowed them to make good use of resources, maintain low administrative costs, and leverage significant amounts of funding, both in-kind and private.

While CABHI has recently taken additional steps to adapt programs to accommodate the needs of more vulnerable populations, such as Indigenous and rural communities, there was no available data on the number of these projects or their impact. In addition, given the early stages of this program's implementation, it has been challenging to demonstrate the tangible health impacts of CABHI-funded projects. These issues should be addressed if PHAC's relationship with CABHI continues beyond its original five-year agreement. However, it should be noted that efforts have already begun in these areas.

CABHI's achievements to date have been accomplished while facing a few notable challenges that were highlighted by key informants across all groups. Of these various challenges, CABHI and funding recipient key informants commented that the perceived rigidity of the federal funding model was the most relevant and noteworthy for PHAC.

In the end, CABHI has clearly been meeting its stated objectives over the first four years of its existence. Its approach and funded projects are helping to address issues related to the growing population of seniors in Canada. Furthermore, CABHI appears to be on its way toward its goal of improving the quality of life for those affected by aging and brain health issues, especially dementia.

Areas of Consideration

With the noted success of CABHI to date, and given that a decision on whether or not federal funding will continue beyond its original five-year allocation has not yet been made, the evaluation did not identify any recommendations. However, the evaluation has identified a few areas of consideration for PHAC, should its relationship with CABHI continue.

1. PHAC should explore any possible flexibility in its funding agreement with CABHI.
2. CABHI has started to engage with diverse populations, such as Indigenous populations, and is focusing on diversity more broadly to include geographic differences and rural communities. However, PHAC should encourage CABHI to make more of an effort to gather specific data on how the projects they fund target and benefit vulnerable populations.
3. CABHI's performance measurement framework and data collection is exemplary and of clear benefit to the administration and management of the program. However, considering they have vastly exceeded nearly all of their original targets, PHAC should encourage CABHI to review and update these targets and find the right balance in setting targets that are both challenging and realistic. As well, they should ensure that longer-term health impacts are sufficiently represented in the collection of future performance data.

1.0 Evaluation Purpose

The purpose of the evaluation was to assess the relevance and the performance of the Centre for Aging + Brain Health Innovation (CABHI) Contribution Program for the period from April 2015 to March 2019.^{Footnote a} The evaluation was required under the Financial Administration Act (FAA) and the Treasury Board of Canada's Policy on Results (2016).

For details on the evaluation scope, approach, and design, including the CABHI logic model, evaluation questions, and a description of the data collection and analysis methods used, please consult appendices 1, 2, and 3.

2.0 Program Profile

Established in 2015, CABHI has received a total of $123.5 million over five years. These funds came from the Government of Canada (GoC), through the Public Health Agency of Canada ($42M); the Government of Ontario, through its Ministry of Research, Innovation and Science ($23.5M); the Baycrest Foundation ($25M); and other commercial and not-for-profit partners. CABHI is a collaboration of over 40 partners in the healthcare, science, industry, not-for-profit, and government sectors, and has funded projects in seven provinces and in collaboration with innovators, companies, and seniors' care organizations across five countries. Overall, CABHI seeks to accelerate the development, validation, commercialization, and adoption of innovative products, services, and best practices to support brain health and aging.^{Footnote b} For more background information on CABHI and its services, please see Appendix 4.

Based on CABHI's logic model (see Appendix 1) and associated program theory, CABHI's ultimate impact is to improve health system effectiveness and quality of life for aging adults, those living with complex brain health issues (e.g., dementia), and their friends, family, and caregivers. CABHI aims to accomplish this by first improving the resources and capabilities (e.g., technical capabilities, linkages to businesses or researchers, access to specialized facilities or data) of target stakeholders (i.e., healthcare providers, researchers and developers, companies, partner organizations). This improved capacity would then be translated into performance improvements, which result in the development and introduction of new or improved products, processes, or services to end-users, as well as the creation of new jobs and increased revenues for CABHI-supported companies.

As stated in their 2015 application to request funding, ^{Footnote 1} CABHI's initial objectives aimed to:

• Improve the lives of millions across Canada and around the world at risk for, or experiencing, age-related cognitive decline;
• Help to reduce the direct and indirect economic costs of dementia;
• Support the development of strong Canadian companies with recognized expertise in this field;
• Work to overcome barriers and ensure that new knowledge and technologies are translated into excellent care and successful enterprises through education and demonstration; and
• Accelerate the adoption and scalability of new and validated solutions across Canadian provincial healthcare systems and beyond.

According to CABHI officials, during the initial setup, CABHI's programs purposely applied a broader scope of looking at developing, testing, validating, and disseminating innovative products and services designed to support brain health and aging as a whole. However, through the implementation of their programs and direct interaction with innovators, healthcare professionals, and expert leaders across the brain health and aging ecosystem, CABHI realized there was a greater gap in the area of dementia. Therefore, they decided to also include a targeted focus on dementia.

Overall, CABHI's activities are performed in support of and through the funding of a variety of projects at various stages in the innovation pipeline (design, develop, test, and mobilize) through four main funding programs:

• Spark, which supports the development of innovative early-stage solutions led by point-of-care staff;
• Researcher-Clinician Partnership Program (RCP²), consisting of projects co-led by researchers and clinicians and support for the evaluation of innovative solutions in real-world settings;
• Industry Innovation Partnership Program (I²P²), which identifies promising solutions, connects vendors with trial partners, and supports them to pilot, validate and ultimately adopt these solutions; and
• Mentorship, Capital Continuity Program (MC²), whose goal is to sustain, spread, scale up, and adopt the most promising projects beyond their initial CABHI-funded pilot.

Figure 1 - CABHI's Innovation Pipeline

The graphic shows CABHI's innovation pipeline where innovations can enter and exit at various points of development stages, from design development to validation and refinement to mobilization and adoption. Through the various programs CABHI offers, proposed projects are vetted against well-defined selection criteria and are classified as being in a specific stage in the innovation pipeline. Some of the selected projects falling in the above three stages will be completed by the end of the fiscal year while many would be continuing into the subsequent year. Overall CABHI supports strategically selected projects with the potential for significant healthcare or economic impact.

The first column of the innovation pipeline shows how an innovative idea, generated from point of care workers, can move to become a project that is further designed and developed that can then generate an output such as a new product or service.

In turn, the second column shows how this new products or services can then be adopted by start-ups or established companies to be validated and refined into products or services that are validated in real-world settings.

This, then, can lead to the third column where the most promising and validated solutions are mobilized and adopted and then spread and scaled up to then achieve broader impacts such as those depicted the fourth column, of health and well-being, health systems, economic growth and a culture of innovation.

Of note, the Knowledge Mobilization Partnership Program (KMP ²), which was previously part of the core programs, was discontinued based on stakeholder feedback, and funding was redirected towards integrating knowledge mobilization into all existing projects. CABHI also has a number of other strategic and regional impact programs. These include:

• Senior's Care Strategic Innovation Fund (SCSIF) (2017-present): CABHI's first Regional program formed in support of a partnership agreement with the New Brunswick Health Research Foundation (NBHRF) to fund innovations made by New Brunswick-based researchers and clinicians, focused on aging adults;
• Brain Health Foundations Program (BHFP) (2017-present): supports validation and testing to accelerate innovation by providing consultation, facilitation, and implementation support from clinicians, researchers, and residents from Baycrest and the Rotman Research Institute;
• Canada-Israel Collaboration Program (CICP) (2017-present): co-funded and cooperatively operated by CABHI and the Israel Innovation Authority (IIA), the program provides Israeli and Canadian innovators with robust scientific evidence to inform decisions on long-term successful commercialization of products and services;
• Other regional programs, such as the Quebec Program, the Saskatchewan Program, the Manitoba Program, the Alberta Program, and the Nova Scotia Program (2018-present). These programs combine provincial and CABHI funding to focus on shared priorities and regional innovations related to aging and brain health; and
• In addition, from 2015 to the present, there is the Strategic Opportunity Fund (SOF), which supports large, strategic initiatives, projects, and partnerships that are aligned with CABHI's mission to improve health outcomes and quality of life for older adults, and drive economic growth and investment in the aging and brain health sectors on a large scale.

3.0 Supporting Canada's Aging Population

3.1 Impact of Aging and Brain Health

There is a continued need for research in the area of aging and brain health, as well as a need to accelerate movement from research to applications that will improve the quality of life for those affected by brain health issues, especially dementia.

Canada's aging population and the increasing burden of brain health issues

Canada's aging population and the increasing incidence of dementia and other neurodegenerative conditions pose a significant challenge for Canadian families, caregivers, and the healthcare system. In 2015, Statistics Canada indicated that, for the first time, the number of persons aged 65 years or older exceeded the number of children aged 0-14 years.^{Footnote 2} Data from the Canadian Institutes of Health Research projects that, by 2031, 25% of the Canadian population will be seniors.^{Footnote 3} While dementia is not an inevitable part of aging, age is the most important risk factor. As a result, with a growing and aging population, the number of Canadians living with dementia is expected to increase in future decades.

In 2016- 2017, more than 432,000 Canadians (7%) aged 65 years and older were living with diagnosed dementia.^{Footnote 4} As this number does not include those under age 65 who may have a young onset diagnosis, nor those that have not been diagnosed, the true picture of dementia in Canada may be somewhat larger.^{Footnote 5}

The increasing number of Canadians living with neurodegenerative conditions such as dementia creates a societal challenge, resulting in drastically increased costs for society, governments, and individuals. It is estimated that by 2036, 62% of the Canadian healthcare budget will be dedicated to the care of seniors. In addition, people living with dementia are staying in their homes longer and placing increased demands on informal caregivers, who spend an average of 26 hours per week providing care.^{Footnote 6}

These issues are not unique to Canada as the worldwide number of people living with dementia and the costs associated with dementia are increasing. According to the World Health Organization, 50 million^{Footnote 7} people have dementia, which is projected to reach 82 million by 2030.^{Footnote 8} In 2015, the total global societal cost of dementia was estimated to be $818^{Footnote 9} billion USD, and is expected to rise to $2 trillion USD by 2030.^{Footnote 10}

Populations are affected by dementia differently

In applying an SGBA+ lens, we found that different segments of the population are affected by dementia at different rates. For example, women make up two-thirds of the Canadian population living with dementia. As women live longer than men, they are more likely to experience age-related cognitive decline.^{Footnote 11} However, women may be more likely to be affected by dementia because of other factors, such as hormonal changes at menopause.^{Footnote 12} Regardless of the reason, women are more likely to develop Alzheimer's disease, the most prevalent form of dementia, than men.^{Footnote 13} Furthermore, women are more likely than men to be caregivers, with 80% of caregivers being female.^{Footnote 14}

Indigenous people in Canada are affected by, and respond to, dementia differently than non-Indigenous Canadians. Indigenous older adults, both on reserve and off reserve, often face complex health issues stemming from socioeconomic marginalization and a legacy of colonialism, and their ability to access healthcare is affected by a host of barriers related to poverty, cultural and linguistic differences, racism, and geography.^{Footnote 15 Footnote 16} The rate of dementia among Indigenous people in Canada is expected to grow more rapidly than among non-Indigenous people, due to higher rates for several risk factors for dementia, such as diabetes, physical inactivity, low education, depression, and family history or genetics.^{Footnote 17} Furthermore, the lack of available information about the incidence and prevalence of dementia in Indigenous populations is proving to be a critical challenge for federal and provincial healthcare systems that seek to provide services to this population.^{Footnote 18} In addition, some populations face barriers to equitable care and access to care for dementia, such as ethnic and cultural minority communities, LGBTQ2 individuals, official language minority communities, rural and remote communities, and those diagnosed with early onset dementia.^{Footnote 19}

As well, some studies have shown that risk factors are related to low socioeconomic status, such as income, financial adequacy, education, and literacy.^{Footnote 20 Footnote 21 Footnote 22 Footnote 23 Footnote 24} While the relationship between socioeconomic status and dementia is not yet well understood, low socioeconomic status is associated with worse physical and mental health, which are in turn associated with an increased risk of cognitive impairment.^{Footnote 25} Other factors that may increase the risk of developing dementia include lifestyle factors, such as physical inactivity, obesity, unbalanced diets, tobacco use, and harmful use of alcohol, as well as diabetes and mid-life hypertension.^{Footnote 26} Mid-life depression, low educational attainment, social isolation, and cognitive inactivity can also have an impact. Having reduced access to care and support services, unsafe living environments, and living in low-income or rural and remote communities also tend to increase the risk of developing dementia.^{Footnote 27}

Innovative Solutions are Needed

With the increasing burden and cost for both societies and families, fostering innovation related to aging and dementia is necessary, because an effective treatment or cure for dementia remains elusive. There are currently no effective drugs to cure or to stop the progress of dementia. As such, non-pharmaceutical approaches have been, and continue to be developed.^{Footnote 28} These innovations aim to stimulate the brain, provoke memories, and induce calm and peacefulness. They can also support caregivers in ensuring the safety and wellbeing of their family and friends.

According to internal PHAC documents, while investments in research on treatments, prevention, and a cure for dementia continue, there is also a need to support interventions and movement from research to applications that will help Canadians living with dementia. At the same time, some challenges have been noted in the area of innovation and healthcare (e.g., required infrastructure, human capital and resources to develop, test and scale applications for broader adoption), which ultimately also present barriers to the adoption and diffusion of innovations in the area of aging and brain health. For example, turning research into practical solutions is particularly complex, with challenges in connecting to the right people and resources, in order to advance valuable ideas to market.^{Footnote 29} Furthermore, according to PHAC internal documents, and confirmed by several key informants, there is risk aversion in the healthcare system related to the early adoption of innovative solutions, which also affects new solutions in this area, as there is sometimes reluctance to try new approaches, especially those reliant on new technologies.

CABHI's activities support these growing needs

CABHI addresses the growing needs of those with aging and brain health issues, such as dementia, by accelerating innovations to develop and refine products, practices, and solutions to help ease this public health challenge, and generally encourage healthier aging. CABHI's work seeks to address current needs to improve health outcomes and the quality of life for those currently living with dementia, especially in the absence of readily available treatments or cures. CABHI aims to address challenges to integrating innovation into healthcare systems by bridging the gaps between research and development, and commercialization by bringing together innovators, the business community, care providers, and consumers. This helps foster research and innovation and can make healthcare system administrators more comfortable with new products because they have already been tested and verified.

In addition, CABHI is beginning to address the needs of those disproportionately affected by dementia. For example, CABHI's Spark program specifically mentions Indigenous and rural populations among their target populations.

3.2 Aging, Seniors, Dementia, and Innovation are Government Priorities

The Government of Canada, through investment and other commitments, has indicated that aging, seniors, and dementia are a priority. Additionally, innovation is a priority for creating a better quality of life for Canadians and stronger economic growth. The work of the Centre for Aging + Brain Health Innovation (CABHI) is aligned with these priorities.

Recent budgets from the GoC demonstrate a record of support for aging Canadians, as well as for people living with dementia and other neurodegenerative diseases. For example, in 2014, the Canadian Consortium on Neurodegeneration in Aging (CCNA) was announced, with an investment of $31.5 million.^{Footnote 30} In 2015, $42 million was invested over five years to support CABHI. In 2018, $75 million was provided to New Brunswick's Healthy Seniors Pilot Project to fund a range of applied research initiatives to support healthy aging for seniors at home or in a long-term-care setting, with an additional $20 million over five years to support people living with dementia and their caregivers through community-based projects. Budget 2019 announced $50 million over five years to support the implementation of the dementia strategy. As well, in 2019, the GoC contributed $31.6 million for five years in support of Phase II of the CCNA.^{Footnote 31}

Over the last five years, the GoC has also engaged in a number of other activities that demonstrate their commitment to supporting seniors and those affected by dementia. For example, the GoC has adopted the World Health Organization's Global action plan on the public health response to dementia (2017-2025) and, in 2018, PHAC hosted a national conference to inform the development of a national dementia strategy.

Recently, PHAC released A Dementia Strategy for Canada: Together We Aspire (the Strategy), which sets out a vision for the future, and identifies common principles and national objectives to help guide actions by all levels of government, non-governmental organizations, communities, families, and individuals. The Strategy's vision is to achieve "a Canada in which all people living with dementia and caregivers are valued and supported, quality of life is optimized, and dementia is prevented, well understood, and effectively treated". CABHI was one of the many active contributors in the development of the Strategy.

In addition to seniors and dementia being a priority, the GoC supports innovation that leads to a better quality of life for Canadians and stronger economic growth. The GoC has demonstrated this by supporting initiatives like Innovation Canada, which helps Canadian innovators navigate government-led innovation programs. Also, in 2019, the GoC invested $763 million over five years to provide tools to researchers through the Canada Foundation for Innovation.^{Footnote 32} Recent mandate letters (2015 and 2017) for the Minister of Health and the Minister of Innovation, Science and Economic Development reinforced research and technological advancements that support the economy and encourage the adoption of digital health technology as priorities.

CABHI's objectives, which include helping healthcare innovators translate their ideas into prototypes to improve aging and brain health for Canadians, are clearly linked to the GoC's priorities related to aging, dementia, and innovation. According to PHAC officials, CABHI, along with other dementia-relevant research and innovation networks, supports the Strategy's objective to advance therapies and find a cure, as well as its research and innovation pillar. Also, as outlined in this investment, the GoC is supporting the outcome area of Healthy Canadians, which strives to develop program areas that support a responsible, accessible, and sustainable healthcare system.

3.3 Through CABHI, PHAC is playing a role in addressing aging and brain health, with a focus on dementia

The Public Health Agency of Canada's (PHAC) investment in CABHI contributes to the Agency fulfilling its mandate and its roles, as outlined by a number of authorities. While there are other Canadian initiatives that work to address aging, brain health, and dementia, CABHI's work is complementary, rather than duplicative.

CABHI helps PHAC fulfill its federal role of improving the quality of life of Canadians, and is in line with the Agency's and the Minister of Health's authorities, as outlined in the Public Health Agency of Canada Act (2006) and the Health Canada Act (1996). Internal documents indicate that PHAC aims to support healthy living and chronic disease prevention in a number of ways, including working with partners like CABHI to identify public health risks, trends, and emerging issues, and take action on them.

There are a number of Canadian initiatives addressing aging, brain health, and dementia. As stated in the contribution agreement for CABHI:

"no one sector alone can meaningfully address the causes of chronic diseases like dementia. The combined resources and expertise of a wide range of partners are required to identify and generate sustainable solutions needed to improve the health of the population".

As part of this evaluation, a review of seven of these initiatives was conducted to determine if there was overlap, duplication, or complementarity with CABHI's activities. General descriptions of the initiatives, as well as their vision, mission, general activities, funding sources, and population served were examined, in order to better understand the similarities and differences to CABHI (see Appendix 4 for more details). Overall, while the programs examined had some overlap with CABHI in terms of focus, mission, and population served, none of them duplicated the work of CABHI. CABHI is unique within Canada in that it supports an innovation pipeline in the brain health and aging domains, giving it the ability to support projects at, and potentially through, the different stages of the pipeline (e.g., research, development, testing, commercialization). The most similar initiative is AGE-WELL, which, like CABHI, was also established in 2015 through federal funding. While there is some overlap between CABHI and AGE-WELL in that both work in the field of aging and dementia, dementia is an important focus for CABHI, while it is just one of the many areas of work for AGE-WELL.^{Footnote 33}

Most key informants across multiple groups also agree that, while there is overlap between CABHI's work and the work of other various initiatives, the overlap is seen as complementary rather than duplicative, because they address different aspects of an immense issue. The majority of internal and external key informants also reported that CABHI is unique in a number of ways. For example, these key informants contended that, unlike many other organizations, CABHI funds research along a continuum, from ideas to commercialization, with a focus on developing products for the market, and they provide access to real-life testing situations and expertise through their association with a growing network of healthcare organizations, including Baycrest.

4.0 What has CABHI achieved?

4.1 CABHI's Activities

During the first four years of its existence, CABHI's funded projects have consisted of a variety of different types of activities (e.g., learning modules, educational events and conferences, creation of new partnerships and collaborations, development and implementation of new programs and projects) and have vastly exceeded their original set targets across the board.

For activities that did not have set targets such as "engagement of older adults and caregivers", the numbers increased exponentially over the four-year period, starting with 4,552 in the second year, 7,568 in the third year, and 25,514 in the fourth year. Public engagement is a key feature of CABHI's programming, where they seek input from individuals engaged with CABHI. For example, community members participating in research studies as part of an intervention trial or technology deployment trial are engaged to provide input on the design, testing, and validation of new products and processes. Furthermore, with respect to the engagement of partners and collaborators, CABHI branded themselves globally as innovation accelerators, forming national and international partnerships and networks in seven Canadian provinces, and with long-term care organizations and global innovators in Israel, Japan, the UK, and the US. In total, CABHI has launched 210 innovation projects (see the text box above that highlights two of these projects with a bit more detail).

As explained in documented interactions between PHAC and CABHI, the original performance targets were selected based on a methodology whereby 10 to 15 projects would receive funding per year. However, post-CABHI launch, this approach was adjusted as there was a higher level of interest in CABHI's programs from the innovation community as well as an increased level of support to match and leverage funding.This allowed CABHI to fund a larger number of projects (210 over a four-year period) and resulted in a greater number of activities and achievements, exceeding expectations and the original performance targets. While contractual targets were not formally adjusted over the years, despite overachievements, revised stretch goals were set through Annual Operating Plans.

4.2 Improvements to the Resources and Capabilities of Target Stakeholders

Across a broad set of performance indicators, CABHI is exceeding targets to improve the resources and capabilities of target stakeholders. Clear majorities of stakeholders report that CABHI has had a positive impact in a variety of areas.

As stated earlier, CABHI's short-term goals are to improve the resources and capabilities of target stakeholders (healthcare providers, researchers and developers, companies, and partner organizations) by:

• increasing access and linkages to industry, researchers, older adults and caregivers, healthcare providers, test environments, and data;
• improving knowledge or awareness of brain health issues, market and client needs, or care related information;
• improving the ability to develop new practices, and to promote and disseminate research; and
• improving project and product design, research, and development of techniques and theories.

Based on surveys with target stakeholders (i.e., healthcare providers, researchers and developers, companies, partners organizations) conducted by external consulting company The Evidence Network (TEN)^{Footnote d} in 2017, 2018, and 2019^{Footnote 34}, and as indicated in the table above, in most cases respondents reported positive impacts on their resources and capabilities due to their involvement with CABHI. Most of their performance targets were exceeded, in some cases by as much as 30 percent.

Below, we highlight some of the more notable findings from the previous table:

• Stakeholders attributed the highest impact of CABHI's involvement to new practice development (97% to 100% reporting a positive impact), developing new knowledge (83% to 94%), their ability to conduct evaluations and validations with end users (76% to 86%), and being able to access test environments, facilities, resources, or data (75% to 78%).
• Relatively lower positive impact was seen with respect to access to non-profit organizations over the three years (50% to 59%). This was the only area where CABHI did not reach its target for the three years that had been evaluated. However, the level of reported impact has been increasing each year and, in 2019, fell just below the target. Furthermore, it was noted in a follow-up document to explain these results that access to non-profits already existed, to some degree, across CABHI stakeholders and this was not seen as a critical added value to accelerate innovations.
• While two indicators did not meet targets in 2018 (impact on technology development and impact on intellectual property), they exceeded targets in the preceding and following years. The drop in 2018, as referenced in documentation, was partly due to the status of the projects when the survey was conducted.
• In the 2018 Impact Assessment Report, stakeholders reported using a variety of support services provided by CABHI (e.g., networking and linkages, design services, post-award project management support, exposure to expertise, knowledge mobilization, facilities, exposure to the CABHI catalogue of innovations and how they may be leveraged). The CABHI support services used most frequently by various target stakeholder groups included:
  • Networking and linkages were most popular among companies (91%) and partner organizations (84%);
  • For healthcare providers, the highest level of support was in being exposed to CABHI's catalogue of innovations (94%); and
  • Research and developers (82%) relied most on post-award project management support services.

Not only did the majority of each stakeholder group (healthcare providers: 100%, researchers and developers: 92%, companies: 91%, partner organizations: 53%) agree that it would be at least somewhat difficult to access similar support services elsewhere, additional analysis indicated that all stakeholders who used any of the support services, and those who used them with higher intensity, attributed greater-than-average impact on their performance. In their report, The Evidence Network (TEN) found CABHI to have a positive association between healthcare providers' knowledge base and their ability to engage in new clinical practices, with their ability to develop and adopt new tools and technologies, as well as influence healthcare guidelines and policies. Furthermore, it was found that a greater use of CABHI's support services also increased their ability to develop new tools or technologies.

The value of CABHI's support was also mentioned during key informant interviews. As one external key informant stated:

"I have had the opportunity to meet with several CABHI client companies and they told me that CABHI was crucial in their development, because without the support from CABHI, they would not have been able to overcome some of the hurdles to things such as lack of money, or lack of 'know-how', particularly in evaluating their products and connections."

4.3 Improvements to the Performance of Target Stakeholders

Performance data suggests that CABHI is making good progress towards improving the performance of target stakeholders, including introducing new products, services, and practices into market or practice.

5.0 Demonstration of Efficiency and Economy

The program has undertaken its activities in an efficient and economical manner. CABHI's operational approach is well structured with a robust governance model, strong project management oversight, a comprehensive approach to performance measurement, and a demonstrated commitment to continuous improvement.

5.1 CABHI's Governance Model and Strong Leadership Enhances Efficiency

CABHI is "housed" within Baycrest, which allows it to minimize administrative costs by using Baycrest facilities, equipment, and resources while operating as a small team. Baycrest also provides care for older adults and is recognized as one of the top research centres in cognitive neuroscience in Canada. Therefore, by being housed at Baycrest, CABHI can readily access the expertise of researchers, healthcare professionals, and those with 'lived experience' to test and validate initiatives as they move through the innovation pipeline.

CABHI has a governance model made up of a board of directors and two advisory councils, all of whom volunteer their time, and provide advice and feedback on processes and management. It is also worth noting that CABHI's two advisory bodies include a diverse representation of members. For example, their Seniors Advisory Panel is comprised of older adults with lived experience in aging and caregiving that can help to provide valuable insights from the perspective of user needs for innovations. CABHI's Board of Directors is also gender diverse and has a good mix of geographic and interdisciplinary representation, resulting in the inclusion of multiple perspectives.

According to CABHI staff and funding recipients, CABHI makes use of its governance structure, project monitoring, and the collection of data to regularly discuss, identify, and focus on the most promising innovations, as well as those that need more support. The following are some examples of advice, support, or evidence provided:

• The Innovation Advisory Council (IAC) recommended having more balance in the funding of a wide range of projects, while at the same time prioritizing projects with the most pertinent goals, as well as refining key performance indicators (KPIs), creating a dashboard to monitor project status, and focusing on more 'promising' projects for higher impact and efficiency.
• The IAC also provided feedback to companies who presented their commercial and clinical plans, helping them to further revise and field-test their plans.
• The Seniors Advisory Panel (SAP) is currently made up of 16 to 18 older adults with lived experience in aging and caregiving help to direct and focus on the practical aspects of innovations from the outset, rather than waiting for the later stages of development to determine if these innovations are indeed practical and useful for end users. Of note, this approach will further be strengthened by 2025 as CABHI plans to transform the SAP into a nationwide Lived Experiences Advisory Panel (LEAP) of 1,000,000 members by engaging seniors, older adults living with dementia, and caregivers. LEAP will also have a much wider reach to vulnerable populations and a greater diversity of older adults.

Of note, one external key informant raised the potential perception of a conflict of interest related to the distinction between Baycrest and CABHI. Under the current governance structure, and as stipulated in the contribution agreement, Baycrest receives the funding for CABHI, who then funds certain projects, some of which are led by Baycrest. However, CABHI's board, their two advisory councils, and their use of external reviewers, all help to mitigate any potential conflict of interest. For example, during the review of submissions to their Calls for Proposals, CABHI uses a third party review process to help maintain their independence.

5.2 Project Management and Oversight

Both PHAC and CABHI showed strong project management and oversight that was characterized by regular interactions via email or phone, status report updates, monitoring of internal processes using standardized templates, and a regular focus on reviewing metrics and data for accuracy.

PHAC

PHAC uses a number of tools for the planning and monitoring of contribution agreements, including following Standard Operating Procedures (SOPs) and using a Business Management Model with clear accountabilities between the program and the Centre for Grants and Contributions (CGC) at PHAC. Some of the tools and methods used to monitor CABHI's activities include financial audits, budget justification templates, a work plan based on risk assessments, and project evaluation templates, to name a few.

A review of documents showed that PHAC had regular contact with CABHI and requested clarification when needed through formal and informal communication mechanisms, such as emails, phone calls, or more formal quarterly meetings. PHAC key informants also noted that the performance data collected from CABHI was assessed based on the work plans established in the contribution agreement. PHAC also conducted a risk assessment at the outset of the contribution agreement and made efforts to synchronize the reporting requirements with other CABHI contributors (e.g., the Government of Ontario) to alleviate the overall reporting burden.

One key informant from CABHI suggested that PHAC's oversight could have been further improved with more timely and formal guidance on eligible expenditures and financial templates.

CABHI

Key informants across all groups felt that CABHI's leadership and oversight were exemplary. In addition, a review of key documents demonstrated use of some of PHAC and other funding organizations' monitoring tools for greater overall accountability (e.g., contribution agreements, funding agreements, travel expenses directive for Ontario). Similarly, CABHI was proactive and timely in complying with the requirements associated with the contribution agreement set out by PHAC. For instance, CABHI offered to provide quarterly reports on key performance indicators and initiated the development of an accompanying executive summary report to provide more in-depth explanations of their performance numbers. CABHI has also been proactive in clarifying information with PHAC to ensure compliance with Agency guidelines, such as asking to receive PHAC's updated guidance on eligible expenditures to ensure adequate compliance with the stated requirements.

The document review demonstrated CABHI's rigorous monitoring process for its projects. The vast majority of CABHI staff, funding recipients, and PHAC key informants agreed that CABHI places a great deal of emphasis on project management and oversight, as well as offering "tips to be more timely and efficient" in project delivery. Every project funded by CABHI has a detailed project charter that outlines project requirements, both in terms of eligible expenditures and performance data collection. Projects are required to report on key performance indicators on a quarterly basis, and provide milestone updates to CABHI management, including financial updates, every six months and upon completion of the project. Final reports, with a summary of successes and failures are also provided at project closures. CABHI staff and funding recipient key informants mentioned the strong and collaborative relationship they share with each other. Both groups mentioned that CABHI not only follows up with project leads for clarification on the quarterly reporting, but also provides support to recipients in meeting reporting requirements.

Key informants from all groups felt that CABHI's strong governance structure also contributes to the success of its project oversight, as it includes various advisory bodies (e.g., Seniors Advisory Panel, Innovation Advisory Council) and a board of directors of individuals with diverse and extensive expertise in the areas of aging, brain health, and innovation. These governing bodies raised questions and provided feedback on CABHI's projects at different stages of review and throughout the innovation pipeline process.

A couple of stakeholder key informants suggested that CABHI could increase efficiency by disseminating information on the social and financial impacts that companies are making with their products once on the market, in order to further educate the private and public sectors on the long-term benefits that can be achieved. It should be noted that CABHI is in the process of publishing the findings of its projects.

5.3 Collection and Use of Performance Measurement Data

CABHI has a detailed, well-developed, and fully implemented performance measurement framework, with indicators and targets, that has been used for program refinements and decision making. CABHI hired an external consultant to develop a detailed performance measurement and evaluation framework (PMEF), with a logic model and indicators with targets over the five-year funding period, including information on performance target determination and methodology. As part of the PMEF, CABHI committed to providing quarterly updates to its funders, tracking inputs and outputs, as well as to yearly impact assessments^{Footnote 36 Footnote 37} on improvements to target stakeholders' resources and capabilities, and performance.

CABHI also proactively decided to provide additional information to PHAC in the form of executive summaries of key performance indicators with detailed explanations on their achievements. In addition to the PMEF, CABHI collects internal data from the individual funded projects that are set out in project charters, including performance indicators, targets, and expected results.

Evidence retrieved from documents, and later confirmed by CABHI staff and PHAC key informants, indicated that performance data is used for planning, determining next steps, decision making, and evaluating impact. For example, projects that are performing well, showing evidence of impact, and demonstrating potential for scale and spread may become eligible to enter into the Mentorship, Capital, Continuation (MC2) Program. On the other hand, projects that are not performing as intended, as indicated by their performance data, are not meeting project timelines, or are not achieving key milestones enter a phase of detailed review and oversight. CABHI works with the project team to assess if changes in schedule, scope, and resources are required to course correct and realign the project for success.

As previously stated, CABHI also meets accountability requirements to funders by providing regular updates and reports to demonstrate what is working or not, as well as identify any potential gaps. In turn, PHAC uses this information for internal briefings and to inform its work. For example, PHAC shared information with colleagues focused on injury prevention among seniors on innovations that can help with the prevention of falls or reduce the number of patients admitted to acute care hospitals or emergency departments.

Going forward, CABHI's performance data collection efforts could be further enhanced by the collection of SGBA+ data to better illustrate the impact of a program on vulnerable populations. In addition, while it will take more time to be able to demonstrate the health impacts of CABHI-funded projects, PHAC key informants expressed their interest in receiving this information. As stated in the logic model and provided in preliminary documentation, CABHI is expecting to be able to demonstrate longer-term health benefits of their funded projects. In fact, it has already started collecting and compiling some of these health impacts from funding recipients, including improved client experience, improved quality of life, improved client access to care and support, improved client access to aging at home, and improved cost-effectiveness to the healthcare system.

5.4 A Culture of Continuous Improvement

CABHI's culture of continuous improvement has led to the adaptation and revision of their programs and application processes, with some of the most notable examples below:

• CABHI created the MC² Program to reinvest resources into projects that demonstrate high potential for spread and scale.
• Based on stakeholder feedback, CABHI eliminated the Knowledge Mobilization Partnership Program (KMP²) to integrate knowledge mobilization as part of all projects at earlier stages of the innovation cycle, by developing a knowledge mobilization roadmap and offering resources and tools to build further capacity in this area.^{Footnote 38}
• While Indigenous and rural populations were not actively initially targeted, an Indigenous and rural populations lens was later applied to calls for innovations in the Saskatchewan Program, the Research Manitoba Program, Nova Scotia Health Research Foundation Program, the Quebec Program, and the Spark Program. This was done to address aging and brain health specifically in Indigenous populations and those in rural and remote areas. Similarly, CABHI provided funding for projects targeted at marginalized groups, such as those in the LGBTQ2 community, in an effort to improve the culture of care for LGBTQ2 seniors. In addition, more recently, a diversity and inclusion adjustment scale was created and will be applied formally to ensure a broader, more diverse selection of projects. The exact number of CABHI-funded projects that target vulnerable populations, and their impact, remains unknown.
• After reviewing a number of applications from Indigenous organizations and discussing the process with them, CABHI realized that these groups conceptualize research differently; as such, they adjusted their assessment process to include various cultural implications by scoring these applications differently to enable a more adequate demonstration of their potential merit.
• Although funding recipients undergo a rigorous application process with clear guidelines and requirements, both the document review and many CABHI, funding partner, and PHAC key informants specifically pointed out that further refinements had been made to this process to make it more efficient. The refinements included streamlining applications' intake, review, and selection process for projects under the Industry Innovation Partnership Program (I²P²), and inclusion of a 'pitch day' that allows funding recipients to present their projects and receive feedback early in their development stages. Overall, according to select funding recipients and CABHI key informants, these refinements to the application process created a better understanding of the process, thereby eliminating redundancies and reducing application times, enabling them to focus on marketing opportunities earlier, and accelerating the shortlist process, with support from external subject matter experts.

5.5 Use of Resources

The program has been cost-effective. Through their affiliation with Baycrest, CABHI has made good use of resources, maintained low administrative costs, and leveraged significant amounts of funding, both in-kind and private funding. Overall, they have spent their allocated budget and none of the funding has lapsed.

Proactive and ongoing monitoring of the program budget ensured appropriate spending from year-to-year, with no variances between overall planned spending and actual expenditures; 100% of the planned budget was used in each of the first four years of CABHI's existence. Based on the table below, in the first two years of operation, most spending was on personnel and Operations and Management (O&M), as CABHI was being set up for implementation, while in the third and fourth years, the two main categories of expenses were personnel and the funding of programs.

Overall O&M was spent as planned; however, there were some variations. For instance, CABHI experienced some overspending with respect to travel (128% of planned budget spent), materials (112%), and underspending on performance measurement and evaluation (77%). It should be noted that these areas within O&M constitute only about 20% of the budget they receive from PHAC. Eighty percent of the budget was allocated to personnel costs and funding initiatives, and CABHI spent 99% of the funds allocated for these two priority areas. In the end, CABHI spent 100% of the $32M they received from PHAC in its first four years of operation. In addition, a recipient financial audit conducted by PHAC in 2018 also showed CABHI to be in a good financial standing, with only minor miscalculations that were clarified with the recipient (e.g., employee benefits, bonuses).^{Footnote 39}

As stated earlier, evidence showed that through its affiliation with Baycrest, CABHI was able to minimize its operational costs (13.8% over the first four years ^{Footnote e} and 9.2% for the most recent fiscal year: 2018-19). In addition, as mentioned previously, CABHI has leveraged a significant amount of private and public funding. Based on their 2018-19 annual report, CABHI has been able to leverage more than $33 million to date. ^{Footnote f} Furthermore, while initially their I²P² Program was the only one that required matching funding from applicants, in subsequent calls for proposals there was such overwhelming support for CABHI's projects that matching funding was offered by applicants for other CABHI programs as well.

5.6 Program Success Factors and Challenges

5.6.1 Elements and Factors that Contributed to Program Success

CABHI's multi-sectoral approach linking innovators and investors through partnerships and collaborations, the leveraging of funds, its affiliation to Baycrest and robust governance model, and the use of an innovation pipeline design that moves initiatives from idea generation to development, testing, commercialization, and market dissemination that is supported by strong project management and oversight, have all contributed to program success.

Although some of these factors were also mentioned in the section on efficiency, the following are the key themes identified by key informants as crucial factors that contributed to program and project success.

Creation of Partnerships

As stated in the document review, and confirmed by the majority of key informants across groups, CABHI created a multi-sectoral approach with organizations in both the public and private sectors to address a very complex problem. Overall, many key informants felt that CABHI increased opportunities in the innovation sector by bringing together different perspectives from the private and public sectors, and linking frontline staff and end users with researchers, healthcare professionals, and industry. This allowed these groups to work together towards a common solution, as well as build stakeholders' knowledge through workshops, conferences, and learning events to better understand the innovation market and needs of older adults and caregivers.

Leveraging of Funds

As stated in the Use of Resources section (6.2), CABHI leveraged a significant amount of private and public funding above the funding allocated in the contribution agreement with PHAC. This additional funding allowed CABHI to expand the breadth of its operations, increase the number of its programs and projects, and expand the overall reach of activities and engagements in the aging and brain health ecosystem.

Affiliation with Baycrest and Governance Structure

CABHI's reputation and credibility are closely linked to its affiliation with Baycrest. What makes this relationship so important to the work of CABHI is its ability to access "top notch" research expertise, while also accessing frontline workers and those with lived experience to test and pilot initiatives. For example, as mentioned previously, they have extensively involved tens of thousands of individuals, often older adults and caregivers, in their activities as participants in the testing of innovations or provision of feedback; they have also received advice through their Seniors Advisory Panel. This feedback helps focus on the practical aspects of innovations, and determine their value and usability for end users.

Program Design and Project Management and Support

As previously stated, CABHI was designed to accelerate the movement of innovations through various stages of the innovation pipeline, from idea generation to development, testing, commercialization, and market dissemination. As one funded project key informant noted, this model allows for the right combination of flexibility and nimbleness to support innovations in becoming sustainable from concept design to widespread adoption.

In addition, CABHI has an extensive performance measurement system with built-in checkpoints through their boards and regular project monitoring, which helps them focus on getting maximum results by quickly identifying those projects that need further support, and those that are most successful or have the most potential.

5.6.2 Challenges Facing the Program

Aligning the dual mandates of the private and public sectors, including demonstrating both health benefits and economic impacts, continuing to fund a variety of projects while also focusing on the most promising ones to achieve the most impact, and having a more stable and flexible funding model were identified as challenges to the continued success of CABHI.

Dual Mandates of Public and Private Sectors

While CABHI's public and private partnerships are one of its successes, aligning the mandates between private and public sector stakeholders, who sometimes have competing perspectives, needs, and agendas, can lead to different goals and definitions of success (e.g., financial vs. health outcomes). Also, CABHI is still in its early stages of operation, as it has only been four years since its inception, and it takes time to see health impacts of projects. However, it can be easier to demonstrate more immediate financial successes in terms of increases in company revenues and job creation.

Culture of Innovation

Creating a culture of innovation in the health sector, and especially in remote senior care facilities, not only takes time (e.g., training staff in research and in various components of project management), but also requires a significant mind shift by those in the healthcare sector to being open to new ideas, practices, and innovative solutions that can replace traditional and more familiar ways of doing things. As one of CABHI's council members stated:

"Creating a culture of innovation is really hard in healthcare. It is risk averse, highly regulated, and not very technologically savvy."

Project Focus

According to CABHI officials, obtaining a balance between having a diverse number of projects, encouraging a wide range of innovations, and focusing on areas or projects with the most potential is challenging. A similar comment was made by an external key informant, who noted the potential challenge of trying to accomplish "too much", with the risk of potentially diluting efforts within the broad environment of aging and brain health and between all the different players, instead of using resources on those projects that have the highest potential impact (i.e., health and economic) for Canadians.

Funding Model

External funders, funding recipients, and CABHI key informants noted a number of challenges associated with the application of the funding model associated with the funds CABHI receives from PHAC. These include the relatively short time frame for funding (five years), the fact that CABHI does not receive money upfront, but rather gets reimbursed for approved spending, and the inability to carry over money from one fiscal year to another.

Many key informants across the various groups interviewed suggested that the uncertainty in extending funding beyond the original five years, along with a heavy reliance on government funding for continued success, could affect the likelihood of future private investments and the retention of top-level expertise from the innovation sector. It should be noted that CABHI is currently looking at a more self-sustainable model that will have some level of federal funding, but should diminish over time. As well, the nature of the funding model also limits the length for which projects are funded (CABHI has set a maximum of 18 months), which could affect the kind of work that can be accomplished as projects have to fit within these relatively short timelines. As one funding recipient said:

"innovations don't follow fiscal cycles".

In addition, the reimbursement model, where expenses are reimbursed only after being spent, presents a challenge to organizations like CABHI in the healthcare sector that do not have extensive funds at their disposal to incur these upfront costs. Another shortcoming noted by CABHI officials and funding recipients was the inability to carry money over from one fiscal year to another. In some cases, this can lead to forced spending within specific timeframes so as not to "lose" the allocated funds. These key informants suggested that, if possible, a funding model that provides money upfront and the flexibility to carry money over from one fiscal year to another would help alleviate some of these challenges.

6.0 Conclusions

In our examination of CABHI's first four years of operation, it is clear that CABHI has achieved notable success with respect to its short- and medium-term goals of improving the resources, capabilities, and performance of target stakeholders (i.e., healthcare providers, researchers and developers, companies, partner organizations). Not only has it exceeded performance targets in a number of areas, a number of new products, processes, services and new, refined, or improved care practices were developed and introduced into practice or brought to market as a result of CABHI.

CABHI's accomplishments and success to date can be traced back to a number of key factors:

• a multi-sectoral approach;
• a robust governance model;
• easy access to research experts co-located at Baycrest and a broad network of test facilities, which maximizes the ability to move projects through the various stages of the innovation pipeline (i.e., design, development, testing, mobilization);
• the leveraging of funding;
• a well-designed and fully implemented performance measurement framework; and
• a demonstrated commitment to continuous improvement.

Evaluation evidence in this report demonstrated that in the aging and brain health communities, the work of CABHI is generally complementary rather than duplicative of the work of other players in these fields. Moreover, with its focus on dementia and the use of an innovation pipeline ending with commercialization, CABHI is very different and, according to many internal and external key informants, quite unique.

It is also worth noting that the evaluation found that CABHI has been quite cost effective in its operations, thanks in large part to their affiliation with Baycrest, which has allowed them to make good use of resources, maintain low administrative costs, and leverage significant amounts of funding, both in-kind and private.

While CABHI has recently taken additional steps to adapt programs to accommodate the needs of more vulnerable populations, such as Indigenous and rural communities, there was no available data on the number of these projects or their impact. In addition, given the early stages of this program's implementation, it has been challenging to demonstrate the tangible health impacts of CABHI-funded projects. These issues should be addressed if PHAC's relationship with CABHI continues beyond its original five-year agreement. However, it should be noted that efforts have already begun in these areas.

CABHI's achievements to date have been accomplished while facing a few notable challenges highlighted by key informants across all groups. Of these various challenges, CABHI and funding recipient key informants commented that the perceived rigidity of the federal funding model was the most relevant and noteworthy for PHAC.

In the end, CABHI has clearly been meeting its stated objectives for the first four years of its existence. Its approach and funded projects are helping to address issues related to the growing population of seniors in Canada. Furthermore, CABHI appears to be on its way toward its goal of improving the quality of life for those affected by aging and brain health issues, especially dementia.

7.0 Areas of Consideration

With the noted success of CABHI to date, and given that a decision on whether or not federal funding will continue beyond its original five-year allocation has not yet been made, the evaluation did not identify any recommendations. However, the evaluation has identified a few areas of consideration for PHAC, should its relationship with CABHI continue.

1. PHAC should explore any possible flexibility in its funding agreement with CABHI.
2. CABHI has started to engage with diverse populations, such as Indigenous populations, and is focusing on diversity more broadly to include geographic differences and rural communities. However, PHAC should encourage CABHI to make more of an effort to gather specific data on how the projects they fund target and benefit vulnerable populations.
3. CABHI's performance measurement framework and data collection is exemplary and of clear benefit to the administration and management of the program. However, considering they have vastly exceeded nearly all of their original targets, PHAC should encourage CABHI to review and update these targets and find the right balance in setting targets that are both challenging and realistic. As well, they should ensure that longer-term health impacts are sufficiently represented in the collection of future performance data.

Appendix 1 – Simplified CABHI Logic Model

Appendix 2 - Evaluation Issues and Questions

The evaluation covered the period from April 1, 2015 to March 31, 2019. This is the first evaluation of contributions to CABHI and was an impact evaluation that focused on how well CABHI has performed in its first four years. Of note, given that CABHI has only been in operation for four years, the evaluation focused on the short- and medium-term outcomes and did not examine the program's long-term outcomes, as they typically take more time to be achieved. Specifically, the evaluation examined the following:

• the continued need for a program like CABHI, and alignment with Government of Canada priorities and roles;
• progress towards achieving expected results (see CABHI logic model in Appendix 1) of:
  • improvements in the resources and capabilities of target stakeholders (i.e., healthcare providers, researchers and developers, companies, partner organizations); and
  • improvements in the performance of target stakeholders;
• demonstration of efficiency and economy, as well as collection and use of performance measurement data in decision making; and
• the effectiveness of PHAC's management and oversight activities.

The evaluation was also designed to examine the factors that contributed or impeded program success. An SGBA+ lens was also applied to the evaluation. Specific questions in the evaluation matrix looked at priority populations and how CABHI-funded projects took into account sex, gender, and other sociodemographic factors. The specific evaluation questions used are listed in Table 4 below.

Appendix 3 - Data Collection and Analysis Methods

Evaluators collected and analyzed data from multiple sources. Data collection began in February 2019 and ended in June 2019. Data was analyzed by triangulating information gathered from the different methods listed below. The use of multiple lines of evidence and triangulation were intended to increase the reliability and credibility of the evaluation findings and conclusions.

Literature review

A review of academic, peer-reviewed publications and grey literature was conducted to support evaluation findings. Findings from the literature review informed questions related to relevance of the program. Approximately 25 articles were reviewed.

Program document and file review

Approximately 225 documents were reviewed to obtain information on all aspects of the CABHI Program and inform findings related to relevance, effectiveness, and efficiency.

Financial data review

A review of financial data from 2015 to 2019 was conducted, including a comparison of the planned budget and actual expenditures.

Comparative analysis to other players in the aging and brain health communities

A review of seven of these initiatives^{Footnote g} was conducted to determine if there was overlap, duplication, or complementarity. The programs' vision, mission, general activities, budget, and population served were examined, in order to better understand similarities and differences to CABHI.

Key informant interviews

Interviews were conducted with 24 (30) ^{Footnote h} internal and external key informants:

• PHAC: n=5 (7)
• Funders other than PHAC: n=2 (3)
• CABHI: (i.e., Staff and Management, Board, Council Members): n=8 (10)
• Funding Recipients: n=4 (4)
• Other stakeholders (i.e., organizations involved in the fields of aging and brain health): n=5 (6)

Limitations and Mitigation Strategies

As with many evaluations, there were some limitations encountered during the implementation of the selected methods that may have some implications for the validity and reliability of the evaluation findings and conclusions. The following table provides a summary of the limitations, impacts, and mitigation strategies to ensure the findings could be used with confidence to guide program planning and decision making.

Appendix 4 - Program Profile

About CABHI

The Centre for Aging + Brain Health Innovation (CABHI) is a solution accelerator dedicated to driving innovation in the aging and brain health sector, with a targeted focus on dementia. CABHI facilitates collaborations among innovators, healthcare providers, private sector partners, governments, and consumers. Through its funding programs, CABHI helps global innovators gain access to key user groups in order to test, develop, validate, and accelerate their solutions in the field of aging and brain health.

The leadership team at CABHI involves the president and CEO of Baycrest, the Managing Director of CABHI, the Executive Director of Health Innovations, the Director of Business Development, Marketing and Communications, the Director of Finance, Performance and Operations, and the Director of Research, Innovation and Translation. This team is supported by approximately 15 full-time employees and a number of other casual and part-time employees who provide project-specific support as required.

Calls for Innovation

With the exception of the Mentorship, Capital and Continuation Program (MC²), potential funding recipients must respond to a public Call for Innovation to seek funding. In general, the various Calls for Innovation are structured to align with CABHI's Innovations Themes (e.g., Aging in Place, Caregiver Support, Care Coordination and Navigation, Cognitive Health, and Aging Indigenous and/or Rural Populations), and operate with defined eligibility criteria, funding rules, funding limits per project, project duration limits, application process and timelines, and the roles and responsibilities for innovators, collaborators, and host institutions. In addition, specific selection criteria are outlined and used to evaluate each application received.

MC² is an invitation-only program where only project teams already receiving CABHI funding are eligible. Innovations demonstrating significant capacity to improve patient and health system outcomes, generate economic returns or achieve significant health system savings, create jobs, as well as scale up and spread across the health system, are invited to participate in MC². Again, specific criteria are used to determine which projects will be eligible to participate in the MC² Program.

CABHI Support Services

In addition to the funding it provides, CABHI also offers a set of core and acceleration services to its funding recipients to aid them throughout the project life cycle.

These core services include:

• Project Management - CABHI-funded projects are assigned to one of the project managers within CABHI's Innovation Office to ensure projects proceed in an efficient manner, meeting milestones and timelines. Assigned project managers are also the point-of-contact for accessing CABHI's acceleration services.
• Marketing and Communications - CABHI's Marketing and Communications Department helps with project's outreach and knowledge mobilization strategies by disseminating their work through the media and their broad network of stakeholders.
• Meeting and Event Space - CABHI headquarters offers state-of-the-art technology, meeting rooms, and event space to its successful applicants.

Acceleration services include:

• End User Engagement - CABHI provides innovators with direct access to end-users (including older adults, caregivers, and clinicians) across diverse settings through its Seniors Advisory Panel (SAP) and a network of trial sites across North America.
• Design Thinking - CABHI provides design services to help assist project teams define their users' unmet needs to optimize product and service development and implementation, usability testing, and participatory design development.
• Trial Design, Research Ethics, and Implementation - CABHI helps projects to design evaluation and validation trials, using the appropriate research methods and ethics frameworks. CABHI can also facilitate the strategic planning and execution of an implementation plan.
• Knowledge Mobilization - CABHI offers knowledge mobilization and implementation science services to projects that are looking to scale up, as well as drive and sustain adoption. Services available include a knowledge mobilization toolkit with tools and templates to support dissemination and implementation of innovations, coaching, and consultation.
• Health System Navigation - CABHI assists innovators in navigating the Canadian healthcare system by identifying system pain points, connecting innovators with clinicians and healthcare organizations to facilitate points of insertion for implementation and testing, developing economic models to add value to the healthcare system, and facilitating procurement in a complex ecosystem.
• Business Development - CABHI provides innovators access to its partners and networks to build key relationships. It provides both online and in-person support for innovators to prepare them for commercialization. For example, some services that are available include market research support, strategy and business development advising, brokering of connections within the healthcare innovation ecosystem and to potential customers, as well as intellectual property (IP) and legal support.

Appendix 5 - Comparative Analysis

Table 6: Comparative Analysis

General Description
Centre for Aging + Brain Health Initiative (CABHI)	Established in 2015, CABHI is a unique collaboration of healthcare, science, industry, not-for-profit, and government partners whose aim is to help improve quality of life for the world's aging population, allowing older adults to age safely in the setting of their choice while maintaining their cognitive, emotional, and physical well-being.
Canadian Institutes of Health Research - Institute of Aging	CIHR established the Institute of Aging (IA) in 2001 "to support research, to promote healthy aging, and to address causes, prevention, screening, diagnosis, treatment, support systems, and palliation for a wide range of conditions associated with aging."
Age Well	Established in 2015, AGE-WELL is Canada's technology and aging network. AGE-WELL is dedicated to the creation of technologies and services that benefit older adults and caregivers. Its aim is to help older Canadians maintain their independence, health, and quality of life through technologies and services that increase their safety and security, support their independent living, and enhance their social participation.
Brain Canada / The Canadian Brain Research Fund	Since 1998, Brain Canada (the Canadian Brain Research Fund was established in 2012) has played a unique and valuable role as the national convenor of a community of those who support and advance brain research. Brain Canada seeks a greater understanding of how the brain works.
Canadian Dementia Knowledge Translation Network (CDKTN)	CDKTN is a network for knowledge translation (KT) and exchange (KE) of research in Alzheimer's disease and dementia that was established in 2008.
Alzheimer's Society (Canada)	Established in 1977, the Alzheimer Society is Canada's leading nationwide health charity for people living with Alzheimer's disease and other dementias. Alzheimer Societies across the country offer programs and services, research funding, public education and awareness campaigns, and advocacy for people affected by all forms of dementia, including Alzheimer's disease.
Hotchkiss Brain Institute	The Hotchkiss Brain Institute (HBI) is an internationally recognized centre of excellence in brain and mental health research and education that was established in 2004.
Brain Health Network	Established in 2015, the Brain Health Network is a body of 16 neurological organizations that represent individuals with brain diseases, disorders, and injuries. Working together, the Brain Health Network supports development of shared programs as well as the identification of common needs and gaps in current services for brain health.
Mandate / Vision / Mission / Objectives
Centre for Aging + Brain Health Initiative (CABHI)	To accelerate the development, validation, commercialization, dissemination, and adoption of innovative products, services, and best practices to support brain health and aging.
Canadian Institutes of Health Research - Institute of Aging	To support research, promote healthy aging, and address causes, prevention, screening, diagnosis, treatment, support systems, and palliation for a wide range of conditions associated with aging. The fundamental goal of IA is the advancement of knowledge in the field of aging to improve the quality of life and the health of older Canadians.
Age Well	To develop a community of researchers, older adults, caregivers, partners, and future leaders that accelerates the delivery of technology-based solutions that make a meaningful difference in the lives of Canadians.
Brain Canada / The Canadian Brain Research Fund	To understand the brain, in health and illness, in order to improve lives and achieve societal impact. Brain Canada is achieving its vision by: Increasing the scale and scope of funding to accelerate the pace of Canadian brain research; Creating a collective commitment to brain research across the public, private, and voluntary sectors; and Delivering transformative, original, and outstanding research programs.
Canadian Dementia Knowledge Translation Network (CDKTN)	Ultimately, this initiative will: Increase patient and caregiver access to information about dementia; and Increase the uptake and application of research findings to dementia care. By doing this, the Network looks to improve the quality of life and care for people with dementia and their caregivers.
Alzheimer's Society (Canada)	The Alzheimer Society of Canada's vision is a world without Alzheimer's disease and other dementias. The Alzheimer Society of Canada identifies, develops, and facilitates national priorities that enable its members to effectively alleviate the personal and social consequences of Alzheimer's disease and other dementias, promotes research, and leads the search for a cure.
Hotchkiss Brain Institute	To inspire discovery and apply knowledge towards innovative solutions for neurological and mental health disorders.
Brain Health Network	To improve the lives of those living with a brain condition, their families, and caregivers. To provide leadership in collaboration, advocacy, research, and education. To elevate awareness of brain health through community collaboration, advocacy, research, and education.
Activities / Area of Focus
Centre for Aging + Brain Health Initiative (CABHI)	CABHI is a solution accelerator focused on driving innovation in the aging and brain health sector. CABHI has funded projects in seven provinces and in collaboration with innovators, companies, and seniors' care organizations across five countries.
Canadian Institutes of Health Research - Institute of Aging	IA focuses on all areas of aging and its research priorities include: The life course as a determinant of active and satisfying aging; Adding life to the later years; Interventions appropriate to the complexity of older people's state of health; Healthcare and services that integrate continuity, innovation, and efficiency; and Ensuring the conditions for a positive impact on older people's health and wellness.
Age Well	Funding research under the following themes: What are the needs of older adults and caregivers and how could technology be used to meet those needs? What technology-based systems and services should be used to enhance the health and well-being of older adults and support independent living? How can innovation be fostered in the short and long term to benefit older people, healthcare providers, and Canadian Industry?
Brain Canada / The Canadian Brain Research Fund	Brain Canada develops and implements transformative, original research programs that address areas critical to advancing brain research. Types of research funded include Basic science, Preclinical research, Clinical research, Translational research, Population health, and Public health. Areas of Research include brain cancer, mental illness, neurodegenerative diseases, brain and spinal cord injuries, pain, and neurodevelopment disorders.
Canadian Dementia Knowledge Translation Network (CDKTN)	CDKTN is made up of four research themes that together create an integrated network of stakeholders: Education and Training, Patient and Caregiver, Knowledge Exchange, and Policy.
Alzheimer's Society (Canada)	Active across Canada, the Alzheimer Society offers programs and services to help people living with dementia, caregivers, and healthcare professionals. They provide information, resources, guidelines, and support. The Society raises awareness of issues that matter most to those living with dementia. They work with politicians, policy makers, and other community and healthcare organizations to advocate for change in legislation, policies, and programs at all levels of government. They also fund Canadian researchers working to find ways to improve care, learn the causes of dementia, and ultimately find a cure.
Hotchkiss Brain Institute	Research - Neurodiscovery Framework includes brain and behaviour, injury and repair, healthy brain aging, and neurotechnologies. Recent research examined issues such as the impact of child abuse on brain development, stress, drug therapy for stroke, and preventing, delaying, reducing, and managing age-related dementias. Education - working to become the destination of choice for mental health and neuroscience education, and provide opportunities for trainees to become successful leaders. Community - looking to become a hub in the community for knowledge and engagement, including engaging community members and organizations, and building impactful partnerships.
Brain Health Network	The Brain Health Network supports the development of shared programs, as well as the identification of common needs and gaps in current services. Activities include providing leadership in collaboration, advocacy, research, and education surrounding brain health. The Network provides information and learning tools to increase public awareness of brain health and wellness. (e.g., centralized access to brain-related education materials, resources, and event information, establishing a network of neurological and mental health organizations in Southwestern Ontario to facilitate information sharing).
Population Served
Centre for Aging + Brain Health Initiative (CABHI)	Innovators, researchers, healthcare providers, private sector partners, government, older adults, and consumers
Canadian Institutes of Health Research - Institute of Aging	The IA community is made up of researchers, scientists, students, community groups, policy makers, and individuals from Canada and around the world that share an interest in the field of research on aging.
Age Well	researchers, older adults, caregivers, partner organizations, and future leaders
Brain Canada / The Canadian Brain Research Fund	researchers
Canadian Dementia Knowledge Translation Network (CDKTN)	Brings together researchers, students, and practitioners from centres across Canada.
Alzheimer's Society (Canada)	General public, healthcare professionals, researchers through grants
Hotchkiss Brain Institute	Direct: academics, researchers, Calgary community members Indirect through research: those suffering brain injuries, trauma, or neurodegenerative disorders, medical community, general population
Brain Health Network	General public, caregivers, healthcare and non-governmental organizations
Funding (e.g., source, model)
Centre for Aging + Brain Health Initiative (CABHI)	CABHI has received funding from the Government of Canada through the Public Health Agency of Canada, the Government of Ontario through the Ministry of Research, Innovation and Science, and the Baycrest Foundation and other commercial and not-for-profit partners.
Canadian Institutes of Health Research - Institute of Aging	CIHR was created by an Act of Parliament on June 7, 2000, bringing together existing government activities. CIHR's annual budget is approximately $1 billion. CIHR is a Departmental Corporation listed in Schedule II of the Financial Administration Act.
Age Well	AGE-WELL is being funded through the NCE program from March 2015 to February 2020.
Brain Canada / The Canadian Brain Research Fund	The Government of Canada initially committed up to $100 million to fund Brain Canada's Canadian Brain Research Fund. It was later supplemented with an additional $20 million through the 2016 federal budget. Access to the full amount of federal funding is determined by Brain Canada's ability to raise equivalent funds from non-federal governmental sources, based on a 1:1 matching model.
Canadian Dementia Knowledge Translation Network (CDKTN)	The Canadian Dementia Knowledge Translation Network was established in 2008 through support from a Canadian Institutes of Health Research (CIHR) grant.
Alzheimer's Society (Canada)	Funding sources include individual, corporate, and government donors. They also rely on communities and healthcare partners to carry out their work.
Hotchkiss Brain Institute	The Institute was enabled by a foundational gift from the Hotchkiss family. Funding sources include individual and other and community investors.
Brain Health Network	Grants provided by the Sifton Family Foundation and through support from various partners in the healthcare and non-government organization fields.

Footnote References