Evaluation of Health Canada’s Territorial Health Investment Fund 2017-18 to 2022-23

Final report
October 2023

Prepared by the Office of Audit and Evaluation
Health Canada and the Public Health Agency of Canada

Table of contents

• Executive summary
• Context
• Evaluation approach
• Findings
• Conclusion and recommendations
• Management Response and Action Plan
• Annex A: Overview of THIF funding allocations
• Annex B: Methodology
• Annex C: Performance indicators
• End Notes

List of acronyms

EMR

Electronic medical record

NIHB

Non-Insured Health Benefits

THIF

Territorial Health Investment Fund

Executive summary

Context

This report presents the findings of the evaluation of the Territorial Health Investment Fund (THIF).

The purpose of the evaluation was to provide guidance and information to Health Canada by focusing on THIF’s relevance and effectiveness. The evaluation covered activities from April 2017 to September 2022.

What we found

THIF funding has enabled the territories to respond to local health needs and has been used differently by each territory to fund the planning, development and execution of a variety of health services and initiatives. This funding flexibility enabled territories to innovate their health system and service delivery without negatively affecting existing services. Identified health care needs were addressed by THIF through innovative system changes and projects. THIF also provided funds for medical travel in each of the territories; supplementing funding received from other federal and territorial programs. Unmet needs related to the social determinants of health, such as housing, drinking water and food security, were outside of the scope of THIF.

THIF has been successful in achieving its short-term outcome of improving Northerners’ access to health care services, and it allowed territories to offer direct health care services, recruit and train staff and adopt new technologies to improve service delivery. The short-term nature of THIF funding presented a challenge to each of the territories in attracting and retaining qualified personnel, which has led to delays in implementing THIF activities. There has not been sufficient data and elapsed time to assess achievement of THIF’s long-term outcome.

Although THIF did not require territories to respond to the Truth and Reconciliation Commission of Canada’s Calls to Action nor the Inuit Nunangat Policy, multiple examples of alignment were found between THIF-funded projects and these key documents. This was because the territories leveraged established mechanisms to engage with Indigenous partners to identify health needs and used THIF funding to address those needs. The funding enabled them to plan and execute system-wide initiatives to address and improve Indigenous health on multiple fronts, including the development and provision of cultural competency and anti-racism courses, and the hiring of Indigenous health care professionals. Early and ongoing engagement with Indigenous partners will continue to be essential to ensure advancements in addressing the health needs of Indigenous peoples.

Recommendations

Recommendation 1: Maintain THIF’s flexibility in addressing the unique needs identified by the territorial governments.

THIF is a unique source of funding that allows the territories to innovate and transform their health care systems while addressing Northerners’ current and emerging health needs. The program’s current flexibility was highlighted as a positive factor contributing to the achievement of outcomes in both the document review and during interviews with territorial partners. This flexibility enables each territorial government to determine how to use these funds based on its priorities and needs. It is recommended to maintain THIF’s flexibility and the understanding that the territories will use these funds appropriately based on their priorities and focus areas.

Recommendation 2: Support each territory to develop context-specific performance measurement strategies that would be of most value for both the territories and Health Canada.

Some territorial partners have faced issues gathering and tracking data on identified and relevant short-term indicators, and measuring longer term outcomes related to program impact. As a result of limited human resources, project delays and difficulties with data collection in remote settings, there were often downstream delays in reporting, with internal interviewees reporting significant challenges in receiving timely reports. To streamline reporting and improve data relevance for both parties, it is recommended that Health Canada partners with each territorial government to develop context-specific performance measurement strategies. This would include working together to identify specific short- and medium-term indicators that are feasible for project timeframes, are easily collected and would help determine the achievement of anticipated outcomes. Given that THIF-funded projects may evolve over time, consideration may be given to reviewing the relevance and usefulness of project indicators in collaboration with the territories as needed.

Context

Program context

The Territorial Health Investment Fund (THIF) delivers funding through grant agreements to each of the 3 territories in recognition of the unique financial, structural and geographical challenges of delivering health care in the North. Structured as 2 streams, the Medical Travel stream and the Innovation Funding stream, THIF funding supports access to health services and territorial initiatives that build sustainable health care systems.

The Innovation Funding stream is directed at strengthening health systems and improving health outcomes through system innovation. The fund supports a maximum of 4 projects per year in each territory.

The Medical Travel stream is used to off-set eligible medical transportation costs of the territories, as defined by each territorial government’s medical transportation policies, which may include transportation for clients and eligible escorts, accommodation and meals.

The tables in Annex A show funding allocations by funding stream for each territory. The Medical Travel stream funding is used as a block transfer, for which data is not tracked. Expenditures from the Innovation Funding stream are tracked and reported annually.

Other federal programs that fund local health care needs in the territories include the Health Services Integrated Fund (HSIF)^{Footnote 1} and the Non-Insured Health Benefits (NIHB) program,^{Footnote 2} which funds supplementary health benefits for First Nations and Inuit clients. Unlike THIF, these funds provide coverage only to registered First Nations and recognized Inuit populations.

Evolution of THIF

Figure 1 provides an overview of THIF’s evolution over the years. The implications from the 2023 Budget announcement are discussed in the Conclusion section.

Figure 1 - Text description

Figure 1 shows the evolution of THIF from 2005 to 2023. Each year is represented by a coloured block with connecting lines. A brief description accompanies each year and is displayed on the right side of each connector.

Here is the order of the coloured blocks, from left to right, by year:

• 2005 in dark red
• 2014 in red
• 2017 in dark blue
• 2018 in royal blue
• 2021 in olive green
• 2023 in dark green

Northern context

Together, Nunavut, Yukon, and the Northwest Territories, also referred to collectively as the North, encompass almost 40% of the land area of Canada, while making up only 0.3% of the Canadian population. The vast land area, inclement weather conditions and limited infrastructure within the territories have a significant impact on, and limit access to health and social services. This limited access to locally available care often results in people with health needs having to travel to larger health centres that may be located far from their communities. Though each territory has programs to help residents with the travel costs associated with medical treatment (medical travel), traveling out-of-territory is often difficult and can become a significant barrier to accessing care.

Nunavut is the largest territory in northern Canada, with 20% of Canada’s land base and with approximately 37,000 residents^{Footnote 3}. The majority of the population (85%) is Inuit, and live in 25 remote communities only accessible by plane. The Government of Nunavut’s Department of Health administers and delivers health care services to all residents via the territory’s only acute care hospital and 24 community health centres, that provide out-patient and 24-hour emergency nursing services. Priority health issues for the territory include mental health and suicide prevention, oral health among children, sexually transmitted and blood-borne infections and tuberculosis^{Footnote 4}.

The Northwest Territories is the only territory with a significant population of all 3 groups of Indigenous peoples: First Nations, Inuit and Métis. Close to 42,000 residents^{Footnote 3} reside in 33 communities, of which a third are only accessible by plane or winter ice roads. Settled and outstanding land claims and self-government agreements in the territory have resulted in a complex internal governance structure. The Northwest Territories Health and Social Services Authority administers and delivers health and social services to 27 communities. The remaining communities are served by the Tłı̨chǫ Community Services Agency and the Hay River Health and Social Services Authority^{Footnote 5}. Residents can access health care at 1 territorial hospital, 1 regional hospital and numerous community health centres. Priority health issues for the territory include mental health and suicide, substance use, sexually transmitted and blood-borne infections and supporting healthy lifestyles^{Footnote 4}.

Yukon’s nearly 41,000 residents live in 18 communities, all of which have road access, except 1. 23% of the territory’s population has Indigenous heritage, with the majority (86%) identifying as First Nations. Almost 80% of Yukoners live in the territorial capital, Whitehorse, with the remaining residing in the 17 rural communities. Residents are provided health care services through 14 community health centres and 4 mental wellness and substance use hubs operated throughout the territory by the Department of Health and Social Services. Additionally, there are 3 hospitals that are managed by the Yukon Hospital Corporation.

Mental health and suicide, substance use, and sexually transmitted and blood-borne infections are identified health priorities for the territory^{Footnote 4}.

Evaluation approach

The purpose of the evaluation was to examine the relevance and effectiveness of Health Canada’s THIF. In addition, the alignment of THIF with Calls to Action for improved Indigenous and Inuit health were examined to identify lessons for program improvement.

Evaluation scope

The scope of this evaluation covered THIF activities from April 2017 to September 2022 (2017-18 to 2022-23). Multiple lines of evidence were used (see Annex B) to address the following questions:

Program relevance

1. What past, current, and emerging needs does THIF exclusively address?
2. How has COVID-19 impacted the implementation of THIF-funded projects and what steps were taken to address any challenges?

Program effectiveness

1. To what extent has the program achieved its expected short-, medium- and long-term outcomes?
   • What factors influenced or hindered achievement of these outcomes?
2. How has THIF supported innovation and health care transformation in the North?
3. How does THIF align with, and can contribute to, the Calls to Action for improved Indigenous and Inuit health?

Findings

Program relevance

THIF funding has enabled the territories to respond to local health needs and was used differently by each territory to fund a variety of initiatives and services, including mental and oral health, medical travel and cultural competency and safety training for health staff. Except for medical travel, which is also funded by other federal and territorial programs, identified health care needs were addressed by THIF through innovative system changes and projects. Other unmet needs related to the social determinants of health, such as housing, drinking water and food security, were outside of the scope of THIF.

Evaluation question 1: Past, current, and emerging health needs

Territorial health needs funded by THIF in fiscal year 2021-22 were considered current. Past needs pre-date this period and emerging needs are foreseeable future health service and system needs.

THIF funding has supported key past and current health service needs by "supplementing existing programs", including mental health services in Yukon and Nunavut, oral health in Nunavut and the Northwest Territories and tuberculosis in Nunavut. Local populations have been significantly burdened by these illnesses. For example:

• In Yukon, the rate of opioid-related deaths (per 100,000 people) was twice the national average in 2021^{Footnote 6}, while the rate of deaths by suicide (per 100,000 people) in Nunavut was nearly 15 times the national average in 2020^{Footnote 6}.
• In 2009-10, nearly 86% of surveyed preschool First Nations children aged 3 to 5 in Yukon and the Northwest Territories experienced dental caries (cavities) in the primary dentition^{Footnote 7}. Pre-school children in Nunavut exhibit a similar burden of oral disease (85.3%)^{Footnote 8}.
• Among the Inuit population in Inuit Nunangat, the 3-year rate of tuberculosis from 2014 to 2016 was over 300 times the rate for the non-Indigenous Canadian-born population across Canada^{Footnote 9}.

Emerging needs identified by territorial partners include aging in place and support for chronic illnesses. Aging in place means having the health and social supports and services needed to live safely and independently in one’s home or community. These issues are being addressed though policies, like the Aging in Place Action Plan and chronic disease management initiatives in Yukon^{Footnote 10} and the 2019 to 2023 Mandate of the Government of the Northwest Territories^{Footnote 11}. In addition, there is a continued need for services related to mental health and addictions, oral health, and tuberculosis.

Health care system needs funded by THIF range from e-health in Nunavut to electronic medical records (EMR) and telehealth in Yukon, from cultural competency and safety training in Yukon and the Northwest Territories to health and human resource capacity building in Nunavut and advancing the health system in Yukon. THIF funding has also been used to address the issue of racism faced by Indigenous populations in their interactions with the health care system. All 3 territories have used THIF funding to strengthen cultural safety and cultural competency. This was carried out through training in Yukon, the hiring of Inuit paraprofessionals in Nunavut and the development of the vision and strategic plan for a culturally safe Health and Social Services system in the Northwest Territories (the NWT Caring for Our People, Cultural Safety Action Plan 2018-20).

THIF has addressed an essential health care system need in all 3 territories through its medical travel stream. As one of several medical travel programs operating in the territories^{Footnote 12}, THIF’s medical travel funding stream is a federal transfer of non-designated funds to offset territorial medical travel expenditures. This stream represented a significant portion of THIF allocations, ranging from 68% of total THIF funding allocated to medical travel in Nunavut, to 70% in the Northwest Territories, and 33% in Yukon (see Annex A). THIF’s medical travel stream played a particularly important role in Nunavut, where internal and external interviewees described flying-in as the only method of service provision in some remote locations. Yukon relied heavily on medical travel given the significant need for medical travel within and out of territory to access medically necessary health services.

All 3 territories have identified important emerging health care system needs, including having robust health care system and service delivery data to support planning, strategic investments in human resources that attracts and retains talent, and an expansion of the Indigenous workforce within the health care system. Collaboration with Indigenous governments to identify and prioritize health needs and address Indigenous-specific social determinants of health were highlighted in Yukon and the Northwest Territories. In Nunavut, regular engagement and collaboration with Inuit organizations took place.

Several external and internal interviewees identified broader social determinants of health, including housing, drinking water, food security, education and economic development that need to be addressed within the territories. Despite affecting access to health care and health outcomes, these issues are beyond THIF’s scope.

“If THIF were to disappear and these programs were to go away, the Department would have huge holes to fill. THIF funding helps bridge gaps, helps add capacity and helps us get closer to our clients.”

Evaluation question 2: Impact of COVID-19 on THIF project implementation

The implementation of THIF-funded projects was impacted by the COVID-19 pandemic. Some projects experienced delays and disruptions. Delays affected program administration, implementation of training sessions and scheduling. Disruptions included the suspension of community programs and health human resources shortages. There were, however, some opportunities gained. Some projects were successful in innovating in the delivery of their activities to help counter the delays and disruptions.

Project delays

During the pandemic, as with all other jurisdictions in Canada, the territories faced significant administrative capacity issues as staff were deployed to support COVID-19 relief efforts. This led to delays in meeting reporting requirements and transferring THIF funds from Health Canada to the territorial governments. The implementation of THIF-funded projects and initiatives across the territories was also pushed back. For example, in Nunavut, Mental Health and Addictions paraprofessionals continued to respond to crises, but all group prevention work was put on hold due to public health restrictions. In-person training sessions were also postponed due to COVID-19 restrictions. In Yukon, the COVID-19 pandemic forced the delay of the launch of the Yukon Midwifery Program by 6 to 12 months while the launch of the Centre de santé Constellation Health Centre was delayed by 9 months. The suspension of community-based oral health screening and preventative programs in Nunavut and the Northwest Territories limited service access during the pandemic and nearly doubled the number of children in need of dental treatment services in Nunavut.

Project disruptions

Travel restrictions and quarantine requirements made it difficult to staff positions and many potential employees preferred to pursue telework positions when offered. COVID-19 response efforts, including the vaccine rollout, diverted staff from THIF-funded projects, and disrupted program operations. The pandemic also exacerbated longstanding health human resources challenges in the North. In the Northwest Territories, travel restrictions and quarantine requirements made it challenging for oral health professionals to travel to communities and engage in oral health prevention and early intervention. School closures and public health restrictions also impeded access to some community oral health programs, such as the toothbrushing program and the fluoride varnish program. In Nunavut, COVID-19 disrupted the territory’s tuberculosis program, specifically, the direct observed therapy training for its community workers and the orientation for nurses working in the program.

Project solutions

Territorial partners developed various solutions to address some of the challenges experienced during the COVID-19 pandemic. A few examples are:

Distributing resources in the community

• The Oral Health Program in the Northwest Territories launched a new initiative to distribute oral health supplies at various locations, including schools, health centres, grocery stores, and vaccine clinics.
• The Department of Health and Social Services in Yukon worked with First Nations governments, non-governmental organizations, and pharmacies to distribute harm reduction naloxone kits to households.

Virtual health care delivery

• Since COVID-19 started, territorial partners have seen an increase in virtual methods of providing care. For example, the Yukon Midwifery Program and the Centre de santé Constellation Health Centre both offer virtual care services to their clients.
• In Nunavut, COVID-19 led the territory to spearhead the use of virtual health care delivery. The territory is developing a long-term strategy to integrate virtual care into its system, which has the potential to improve health care access in remote locations.

Recruitment strategies

• To tackle staffing issues, territorial partners in Yukon engaged in new recruitment strategies, like using social media.

Community engagement

• Territorial partners in Nunavut found creative ways to conduct outreach and provide mental health services to the community, including through community radio shows, Facebook groups and socially distanced community-based contests.

Program effectiveness

THIF has improved Northerners’ access to health care services. THIF funding has allowed territories to offer direct health care services, recruit and train staff, and adopt new technologies to improve service delivery. Challenges remain in attracting and retaining qualified personnel, which has led to delays in implementing THIF activities. Availability of data on Northerners’ health would also be key in helping territories understand what services and programming are most needed, and to examine the impact of THIF-funded activities.

See Annex C for an overview of all key program short-, medium- and long-term performance indicators and their results.

Evaluation question 3: Achievement of Outcomes

Short-term outcomes

THIF has effectively achieved its intended short-term outcome of improving Northerners’ access to health care services. The program’s efforts align with the WHO Health Systems Framework’s building blocks (see Table 1), resulting in territorial initiatives that have strengthened the health systems and enhanced access to care.

Building block 1: Service Delivery

The territories have provided direct services and enhanced their health care systems. THIF has funded initiatives that have allowed greater access to practitioners, mental health and addictions services, oral health services, tuberculosis treatment and medical travel. For example, in the Northwest Territories, THIF was used to fund travel of 5 oral health professionals and 1 registered dental hygienist in communities to provide screening and preventative oral care services to children. Moreover, the territories have also developed and implemented new initiatives like virtual care to improve access to health care services.

“THIF has allowed us to put more health care services in the communities and on the traditional territories where people live.”

Building block 2: Human Resources

To increase access to care, the 3 territories used THIF funding to recruit and train health and human resource professionals. They focused their recruitment efforts on hiring various types of professionals, such as:

• Service providers, including 2 Nurse Practitioners in Yukon, 15 Inuit Community Oral Health Coordinators in Nunavut, and 18 Inuit mental health and addictions paraprofessionals in Nunavut, and
• Program and administrative staff, including a 5-member team for Nunavut’s Tuberculosis program and a project manager at Yukon’s Centre de santé Constellation Health Centre.

Their training efforts included topics such as cultural competency in Yukon and Nunavut and cultural safety and anti-racism training in the Northwest Territories, oral health, mental health, integrated care and technology, as well as the training of para-professionals.

Building block 3: Information

The Information building block is divided into 2 domains: health promotion, and health system data and research. The 3 territories have implemented various activities to raise awareness of health issues and promote healthy behaviours. In addition, all territories recognize the importance of data-driven decision making and have undertaken efforts to advance their health systems’ performance data and research.

Across the North, multiple engagement opportunities were undertaken to understand the needs of different population groups, and addressing health needs remains a priority. Specific initiatives in the Information building block include the Northwest Territories’ oral health promotion and the creation of the Population and Public Health Evidence and Evaluation (PPHEE) Unit and 1Health in Yukon.

“We spent a lot of time planning and trying to understand community needs. We took the time to hear from community members and people that would access the services, and consider X or Y.”

Building block 4: Financing

Activities in this building block have strengthened the health system by assessing the use of funds to identify areas of operational efficiency and quality improvement, as well as transitioning projects from THIF’s temporary funding to sustainable core funding through the territorial governments. The Northwest Territories conducted reviews of health system costs and medical billing, and initiated the modernization of operations and business processes within the medical travel program. Yukon also used THIF funding to support a review of their medical travel program, including holding consultations with more than 2,300 Yukoners on affordability and administration of the program, among other topics. Recommendations from these engagement sessions, as well as from the Putting People First report, have informed several improvements to medical travel in the territory. Additionally, Yukon secured sustainable funding for 1Health Yukon and introduced more flexible supports for Yukoners through its Rural End-of-Life Self-Managed Funding Program.

“THIF has allowed us to advance strategic initiatives we would not be able to resource.”

Building block 5: Medicines and technologies

In this building block, THIF supported efforts related to medicines, supplies and equipment, technologies, and tools and databases.

To improve health care access to Northerners, the territorial governments have implemented various initiatives, including distributing medicines and oral health supplies, purchasing dental equipment, implementing EMR systems and increasing virtual care options.

Building block 6: Governance

All 3 territories undertook activities related to governance with the aim of strengthening their health systems and increasing access to health care. They created key policies, strategies, and action plans, including Yukon’s Putting People First Report, the Northwest Territories’ Cultural Safety Action Plan, and Nunavut’s Inuit Tuberculosis Elimination Framework. These governance documents and frameworks were used to create or modify their organizations, including Health and Wellness Yukon (in development), the Population and Public Health Evidence and Evaluation Unit in Yukon and the System Sustainability Office in the Northwest Territories.

“As of August 2022, 48 out of 76 Putting People First recommendations are partially or fully implemented. A total of 22 recommendations that are either completed or in progress are fully supported by THIF.”

The territories also built stronger leadership structures and partnerships, including Yukon’s cultural safety and humility training pre-planning table and the Northwest Territories’ Indigenous Advisory Body which provides guidance and advice on including Indigenous knowledge and healing within health service settings^{Footnote 13}.

Central pillar: People

The People pillar is a central part of the health system and influences each of the building blocks. Different groups of people played important roles as users, contributors, partners and collaborators in THIF.

As users and receivers of services, patients, their families, and the public are recipients of the different THIF-funded initiatives, such as the oral health program for children in Nunavut and the Northwest Territories. THIF funding has enabled the hiring of clinical and non-clinical staff for a variety of positions.

Each territory has engaged with a variety of partners and organizations to support policy development, and service design and delivery. These contributors and collaborators included the public, people with lived experience, community organizations representing Indigenous Peoples and Northerners, including Indigenous groups such as Nunavut Tunngavik Incorporated, along with self-governing First Nations governments in Yukon, who are main partners for the delivery of THIF-funded land-based programming in Yukon. Indigenous partners acknowledged these collaborative forums and emphasized their value in increasing involvement and engagement among partners.

Factors influencing achievement of short-term outcomes

Many factors positively influenced the achievement of improving Northerners’ access to health care services. All lines of evidence suggested that THIF’s flexibility enabled the territories to focus on local priorities. The territories also had the ability to carry over unused funds to the next fiscal year, which was described by interviewed territorial partners as being “vitally important to (project) sustainability”. The flexibility of THIF funding during the COVID-19 pandemic allowed territories to adopt innovative technology and communication, which along with beneficial collaborations, including with Indigenous partners, and effective hiring of Indigenous professionals that know the language and culture, supported effective service provision.

As reported by territorial partners and found in the document review, the main factor hindering the achievement of short-term outcomes has been the short-term nature of THIF funding. This has limited the capability of territories to fill identified health human resource positions through long-term contracting (or permanent employees), particularly for the 2-year extension. Territories have had to rely on casual or term employees, which has resulted in higher staff turnover. This affected continuity of services, and the territories’ ability to develop and execute plans. Furthermore, both lines of evidence suggest that the COVID-19 pandemic hindered the achievement of improving Northerners’ access to health care services given that pandemic-related staffing shortages impacted project implementation. Additional limiting factors reported by territorial partners and internal interviewees included the difficulty to track relevant short-term indicators, the lack of longer-term data to determine program impact and the challenges in timely report submission. While Indigenous partners reported room for improvement in their engagement, the evaluation noted several existing and operational forums for ongoing collaboration and dialogue in each of the territories. Unaddressed social determinants of health, including racism in health care, limited infrastructure and a housing shortage which, in turn, limit the ability to attract human health resources in the North, were also identified as hindering factors.

Medium-term outcomes

There is evidence of variable progress in meeting Northerners’ health needs among the 3 territories.

Medium-term outcomes are typically assessed over a 5-to-10-year timeframe, and THIF-funded projects primarily focused on immediate needs. As presented in Annex C, Yukon achieved both identified medium-term performance indicators, while Nunavut and the Northwest Territories reported variable progress toward achieving identified indicator targets.

Nunavut’s mental health and addiction paraprofessionals and Yukon’s midwifery programs, the Centre de santé Constellation Health Centre and community mental wellness hubs are addressing some health needs. For instance, the recent opening of the health centre resulted in 500 patients finding a primary care provider. The community mental wellness hubs, where clients receive integrated nursing and counseling services, are meeting the needs of more than 2,000 clients per month. However, meeting basic needs remains challenging due to factors such as social determinants of health and inadequate public health data. Specific longer-term system changes may be required to address some health needs.

Long-term outcome

More data and time are needed to determine whether THIF has improved the health status of Northerners.

Despite a slight increase in the percentage of Northern populations reporting good health status between 2017-18 and 2019-20^{Footnote 14}, there is a significant lack of recent and reliable data in the North due to small population sizes and unique challenges in collecting data in remote and isolated communities. This prevents the assessment of long-term outcomes. Additionally, as THIF funding represents a very small portion of the territorial government’s overall health budgets, some long-term outcome may not be directly attributable to the program.

Territorial partners attributed gaps in project data collection and reporting to “thinly resourced” project teams, with positions often being filled through temporary hiring. Movement of these temporary staff members negatively affected project activities, including data collection and reporting. Additional issues identified included the complexity of identifying and mapping long-term indicators for innovative projects and the inflexibility of performance framework requirements and reporting indicators, which territorial partners thought were non-responsive to the “realities of the fluidity on the ground”.

Evaluation question 4: System innovations and transformations

THIF has successfully supported innovation and health care transformation in the North. Innovations have ranged from enhancing collaborative care models and expanding technology and electronic health, while transformations have included larger-scale initiatives in the areas of health care planning and governance. According to some interviewees, THIF funding allowed the advancement of important work that otherwise would not have been resourced.

Innovations

THIF funding has supported a variety of innovations across the 3 territories. With Yukon’s strong focus on enhancing integrated and collaborative care models, THIF fully supported the foundational planning and development of the Centre de santé Constellation Health Centre (described in the previous section) and the Yukon Midwifery Program. The latter provides safe and integrated care during pregnancy, birth and postpartum. Prior to the launch of this program, there were limited options for Yukoners to have safe home births, especially for First Nations women and other vulnerable individuals who experienced trauma in clinical settings. THIF funding also helped Yukon adopt a new electronic medical record system that has improved the tracking of patient demographics, service needs, waitlists and intake volumes for better access and continuity of care.

Transformations

THIF funding has contributed significantly to system-wide transformations across the territories. The Department of Health and Social Services in the Northwest Territories is making large-scale changes to better integrate cultural safety and relationship-based care into health care service delivery through its Primary Health Care Reform Demonstration Projects. The territory is also leading system commitments to address cultural safety and anti-racism through the Indigenous Advisory Body. These initiatives are grounded in the 2018-2020 Cultural Safety Action Plan that seeks to promote an integrated system of care grounded in the cultures of Northern and Indigenous Peoples and implemented throughout the health and social services system.

Additionally, THIF funding fully supported the planning and development of Yukon’s Putting People First Report and its 76 recommendations. THIF funding is also supporting the report’s following actions:

• Creating a new health authority (Health and Wellness Yukon) to improve the delivery of health and some social services in the territory.
• Creating a holistic and expanded primary care system built on relationships between providers and patients.
• Establishing a Cultural Safety Team that is building a comprehensive approach to cultural safety and humility in the health care system, in collaboration with Yukon First Nations partners.
• Increasing the use of virtual care options for Yukoners.

Evaluation question 5: Alignment with Calls to Action

THIF did not require territories to respond to the Truth and Reconciliation Commission’s Calls to Action nor the Inuit Nunangat Policy; however, there are multiple examples of alignment between funded projects and those key Indigenous health guiding documents. THIF supported ongoing territorial initiatives to improve Indigenous health on multiple fronts, including through the development and provision of cultural safety, competency and anti-racism courses, and the hiring of Indigenous health care professionals.

The territories used THIF to plan and execute a variety of initiatives aimed at improving the health of their populations. Though THIF did not require territories to respond to the Truth and Reconciliation Calls to Action, it was noted that THIF-funded projects within each territory were aligned with the Calls to Action and the Inuit Nunangat Policy. The territories designed and implemented several system-wide initiatives to improve Indigenous health based on their knowledge of local needs and as determined through their collaborations with Indigenous partners. Additionally, the territories have adopted larger frameworks in their response to identify and address local needs, for example, the United Nations Declaration on the Rights of Indigenous Peoples, adoption of the Missing and Murdered Indigenous Women and Girls Calls to Justice in the Northwest Territories, and the Nunavut Land Claims Agreements.

Availability of appropriate health services

Each territory identified local health needs and adopted different initiatives to improve health care service availability for their populations at large. While health services were nor specifically targeted to Indigenous residents, deliberate enhancement of service delivery within communities, and response to identified needs of local populations are expected to benefit local Indigenous communities.

THIF funding for preventive oral health services in Nunavut and the Northwest Territories enabled staff to travel from local hubs to catchment communities to provide services. The Yukon Midwifery Program, described previously, provides comprehensive services to expectant mothers and those who have recently delivered, which may be particularly helpful for Indigenous women who may prefer to give birth at home.

Yukon and Nunavut collaborated with Indigenous partners to design mental health and addictions services that respond to local needs. THIF-funded initiatives included land-based mental wellness programs that provide mental health and substance use supports in Yukon. These programs use traditional teachings and practices along with clinical services to enhance healing among program participants. Similarly, traditional approaches to mental health and addiction services were incorporated within Nunavut’s Inuusivut, Mental Health and Addictions Division. Additionally, THIF funding supported the streamlining and standardization of mental health, addictions and suicide prevention services.

“It’s my understanding that the entire expert panel on the Putting People First report was only possible because of THIF. How important and transformational it’s going to be! And we thank the federal government for that!”

Increasing the number of Indigenous^{Footnote 15} professionals in health care

Some territories hired more Indigenous and Inuit health care and paraprofessional staff as a mechanism to support health care delivery in languages other than English and French, improve representation, ensure continuity of care and bridge the gap in the provision of culturally appropriate services.

THIF funding enabled Nunavut to recruit 15 Inuit Community Oral Health Coordinators and 18 Inuit Mental Health and Addictions paraprofessionals to help provide consistent and culturally appropriate care. The creation of the Centre de santé Constellation Health Centre and new mental wellness hubs in the Yukon led to the addition of approximately 15 new positions to be filled through preferential hiring and retention of staff with Indigenous ancestry.

Providing cultural competency training for all health care professionals

The 3 territories adopted training for health care staff to provide relationship-based, inclusive, safe and anti-racist care for Indigenous populations. In particular, the Department of Health and Social Services in the Northwest Territories developed mandatory and in-person territory-specific cultural safety and anti-racism training for all department staff. In Yukon, initial training sessions were developed, but all work was paused in response to concerns raised by Yukon First Nations. Since then, efforts are underway to develop and adopt an agreed upon, fully Yukon First Nations driven approach to project direction and development of a Yukon based and owned course.

Moving beyond training, primary health care reform initiatives in the Northwest Territories incorporated cultural responsiveness into the provision of front-line services. Interviewed territorial partners anticipate that relationship-based care provision by front-line staff would create “tangible shifts in work”.

Closing the gaps in health outcomes between Indigenous and non-Indigenous communities

Closing gaps in health outcomes is a longer-term outcome and there was limited evidence to clearly demonstrate how this Call to Action was addressed through THIF-funded projects, which were primarily focused on short-term outcomes. While each territory funded priority issues that could potentially help improve health outcomes in Indigenous communities, several barriers were identified that may limit the success of these initiatives. These include:

• Data insufficiency, particularly of disaggregated data that enables effective equity-based planning.
• Physical inaccessibility to centres of care and associated issues of out of community travel to access insured health care services for Indigenous communities.
• Health human resources shortages, with reliance on temporary or casual health staff, which affects continuity of care.
• Social determinants of health unique to Indigenous populations and perception of discriminatory service practices existing within the health care system, limit their access to health care services.

Conclusion and recommendations

THIF is a unique program that enables the territories to innovate and transform their health care systems as they see fit, and ensure Northerners have access to the health care they need. The program is greatly appreciated in the North because it provides the resources and time to focus on improving and strengthening health systems. THIF funding for medical travel also helps meet the health needs of Northern populations. Despite delays and disruptions caused by the COVID-19 pandemic, the territories implemented new solutions and continued their work using carry-forward funds once public health restrictions were lifted.

Improved access to health care services has been achieved, as illustrated through several building blocks, including service delivery, human resources, medicine and technology. However, insufficient data and time elapsed make it impossible to determine whether the long-term outcome of improved health status of Northerners has been achieved. Longer-term funding is needed to implement the innovative and transformational systematic changes being put in place in the 3 territories. Finally, THIF aligns well with the Truth and Reconciliation Commission’s Calls to Action and the Inuit Nunangat Policy, as THIF-funded projects addressed health needs identified in collaboration with Indigenous partners, with examples across the 3 territories showing work underway to improve the health of Indigenous people. Additionally, each of the 3 territories has established mechanisms to liaise with Indigenous partners which present an opportunity to initiate early engagement with Indigenous partners. This would effectively address concerns raised by a few Indigenous partners on their late inclusion in projects.

Budget 2023

On March 28, 2023, a ten-year renewal of THIF was announced in Budget 2023, with a commitment to “provide $350 million over 10 years in recognition of medical travel and the higher cost of delivering health care in the territories.”^{Footnote 15} This budget announcement aligns well with the results of this evaluation. The evaluation found that short-term THIF funding has significantly limited the territories’ ability to recruit and retain staff who are seeking permanent positions, to plan and implement funded activities on schedule, and to report on indicators. Considering the existing health and human resource crisis and limited infrastructure in the North, the budget announcement of more THIF funding over a longer period is expected to address some of these key issues.

Budget 2023 also made a commitment to “provide $505 million over 5 years, starting in 2023-24, to the Canadian Institute for Health Information, Canada Health Infoway, and other federal data partners” to “work with provinces and territories to develop new health data indicators and support provincial and territorial efforts to use data to improve the safety and quality of health care.”^{Footnote 17} This announcement is consistent with this evaluation’s findings. All lines of evidence suggested that long-term health data are limited in the 3 territories due to small populations and the challenges associated with data collection in remote and isolated communities. In addition, data need to be systematically collected over time to assess program impact. For this reason, it is not currently possible to determine whether THIF has improved Northerners’ health status. This new federal investment in health care data has the potential to address these important barriers, track the impact of investments over time and ultimately lead to improvements in the health of Northerners.

Considering these 2 budget commitments, the recommendations will focus on 2 additional areas for improvement.

Recommendation 1: Maintain THIF’s flexibility in addressing the unique needs identified by the territorial governments.

THIF is a unique source of funding that allows the territories to innovate and transform their health care systems while addressing Northerners’ current and emerging health service and health care system needs. The program’s current flexibility was highlighted as a positive factor contributing to the achievement of outcomes in both the document review and throughout the interviews with territorial partners. This flexibility enables each territorial government to determine how to use these funds based on its priorities and needs. It is recommended to maintain THIF’s flexibility and the understanding that the territories will use these funds appropriately based on their priorities and focus areas.

Recommendation 2: Support each territory to develop context-specific performance measurement strategies that would be of most value for both the territories and Health Canada.

Some territorial partners have faced issues gathering and tracking data on identified and relevant short-term indicators, and measuring longer term outcomes related to program impact. As a result of limited human resources, project delays, and difficulties with data collection in remote settings, there were often downstream delays in reporting, with internal interviewees reporting significant challenges in receiving timely reports. To streamline reporting and improve data relevance for both parties, it is recommended that Health Canada partners with each territorial government to develop context-specific performance measurement strategies. This would include working together to identify specific short- and medium-term indicators that are feasible for project timeframes, are easily collected and would help determine the achievement of anticipated outcomes. Given that THIF-funded projects may evolve over time, consideration may be given to reviewing the relevance and usefulness of project indicators in collaboration with the territories as needed.

Management Response and Action Plan

Recommendation 1:
Maintain THIF’s flexibility in addressing the unique needs identified by the territorial governments.

Management response
Health Canada program management agrees with this recommendation.

Action Plan	Deliverables	Expected Completion Date	Accountability	Resources
The program will ensure that terms and conditions and grant agreements maintain and improve flexibility to territories to determine how to allocate their respective THIF funding based on their unique priorities and needs.	Program authorities updated to allow for funding to be delivered through 1 stream only, instead of distinct streams for medical travel and innovation, to provide flexibility to territories as they continue to innovate within their health care systems and support the needs of their residents.	October 2023	Director General, Health Care Strategies Directorate, Strategic Policy Branch	This recommendation will be completed using existing resources.

Recommendation 2:
Support each territory to develop context-specific performance measurement strategies that would be of most value for both the territories and Health Canada.

Management response
Health Canada program management agrees with this recommendation.

Action Plan	Deliverables	Expected Completion Date	Accountability	Resources
The program will engage with each territory as their respective performance measurement plans are developed and implemented, with a goal of streamlining reporting and improving data relevance.	Grant agreements will identify short-term and medium-term indicators that can assess progress against program outcomes.	March 2024	Director General, Health Care Strategies Directorate, Strategic Policy Branch	This recommendation will be completed using existing resources.
	As-needed meetings with each territory to discuss performance measurement.	N/A (Ongoing)	Director General, Health Care Strategies Directorate, Strategic Policy Branch	This recommendation will be completed using existing resources.

Annex A: Overview of THIF funding allocations

Table 1: Allocation for Nunavut (2017 to 2023)

Fiscal Year	THIF Funding Stream (CAD $) - Innovation	THIF Funding Stream (CAD $) – Medical travel	Total Funding (CAD $)
2017-18	4,300,000 (31.85%)	9,200,000 (68.15%)	13,500,000
2018-19	4,300,000 (31.85%)	9,200,000 (68.15%)	13,500,000
2019-20	4,300,000 (31.85%)	9,200,000 (68.15%)	13,500,000
2020-21	4,300,000 (31.85%)	9,200,000 (68.15%)	13,500,000
2021-22	4,300,000 (31.85%)	9,200,000 (68.15%)	13,500,000
2022-23	4,300,000 (31.85%)	9,200,000 (68.15%)	13,500,000
Total	25,800,000	55,200,000	81,000,000

Table 2: Allocation for Northwest Territories (2017 to 2023)

Fiscal Year	THIF Funding Stream (CAD $) - Innovation	THIF Funding Stream (CAD $) – Medical travel	Total Funding (CAD $)
2017-18	2,100,000 (29.58%)	5,000,000 (70.42%)	7,100,000
2018-19	2,100,000 (29.58%)	5,000,000 (70.42%)	7,100,000
2019-20	2,100,000 (29.58%)	5,000,000 (70.42%)	7,100,000
2020-21	2,100,000 (29.58%)	5,000,000 (70.42%)	7,100,000
2021-22	2,100,000 (29.58%)	5,000,000 (70.42%)	7,100,000
2022-23	2,100,000 (29.58%)	5,000,000 (70.42%)	7,100,000
Total	12,600,000	30,000,000	42,600,000

Table 3: Allocation for Yukon (2017 to 2023)

Fiscal Year	THIF Funding Stream (CAD $) - Innovation	THIF Funding Stream (CAD $) – Medical travel	Total Funding (CAD $)
2017-18	4,300,000 (67.19%)	2,100,000 (32.81%)	6,400,000
2018-19	4,300,000 (67.19%)	2,100,000 (32.81%)	6,400,000
2019-20	4,300,000 (67.19%)	2,100,000 (32.81%)	6,400,000
2020-21	4,300,000 (67.19%)	2,100,000 (32.81%)	6,400,000
2021-22	4,300,000 (67.19%)	2,100,000 (32.81%)	6,400,000
2022-23	4,300,000 (67.19%)	2,100,000 (32.81%)	6,400,000
Total	25,800,000	12,600,000	38,400,000

Table 4: Overall THIF Funding Allocation by Territory (2017 to 2023)

Territory	THIF Funding Stream (CAD $) - Innovation	THIF Funding Stream (CAD $) – Medical travel	Total Funding (CAD $)
Nunavut	25,800,000 (31.85%)	55,200,000 (68.15%)	81,000,000
Northwest Territories	12,600,000 (29.58%)	30,000,000 (70.42%)	42,600,000
Yukon	25,800,000 (67.19%)	12,600,000 (32.81%)	38,400,000

Annex B: Methodology

The scope of the evaluation included THIF activities from April 2017 to September 2022. The evaluation was designed to assess the achievement of outcomes and provide insight on the evaluation questions.

The evaluation team collected data using various sources and methods, including the following:

Document Review

Program staff provided documents for evaluators to review. In total, the evaluation team reviewed 81 relevant documents. An additional 20 documents were added to the document review as a follow-up to the interviews.

Interviews

Evaluators conducted 27 interviews, including individual and group interviews. This included 3 interviewees internal to Health Canada, 2 interviewees internal to the Public Health Agency of Canada, 39 interviewees from the territories, 6 interviewees from other federal government departments and 8 other external interviewees (non-governmental organizations or experts). The evaluators used the QSR NVIVO qualitative analysis software to code all data and identify emerging themes from the interviews.

Data Analysis

Health Canada provided performance measurement data, which the evaluation team analyzed to identify key trends and assess outcomes.

The evaluation team used triangulation to analyze data collected by these various methods to increase the reliability and credibility of the evaluation findings and conclusions. Still, most evaluations face some constraints that may affect both validity and reliability of information.

The evaluation team used the modified WHO Health Systems Framework for Action, 2009^{Footnote 18}. Based on values of universal access, equity, participation, and intersectoral action, the systems framework promotes a common understanding of the elements of a health system and identifies what constitutes health system strengthening.

Figure 2 - Text description

Figure 2 shows the World Health Organization’s Health System Framework, 2009.

The figure is a series of circles. The dark red circle at the centre reads “People” which is the Central Pillar of the WHO Health Systems Framework.

The smaller circles represent each of the six building blocks of the Framework. Starting from the top of the circle and going clockwise, the smaller circles read: Governance (purple), Information (navy blue), Financing (brown), Service Delivery (royal blue), Human Resources (dark blue), and Medicines and Technologies (olive green).

Based on the functions that make up the health system, the Framework has 6 “building blocks” and a central pillar (Figure 2). Each of the building blocks and the pillar are discussed in Table 1.

Table 5: Description of the Components of the WHO Health Systems Framework

Health System Framework for Action	Description
Building Block 1: Service delivery	Effective, safe, and quality personal and non-personal health interventions to those who need them, when and where needed, with a minimum waste of resources.
Building Block 2: Human resources	A responsive, fair, and efficient distribution of human resources to achieve the best health outcomes possible, given available resources and circumstances. Available in sufficient numbers and with a mix of staff, and are fairly distributed, as well as competent, responsive and productive.
Building Block 3: Information	Ensuring the production, analysis, dissemination, and use of reliable and timely information on health determinants, health systems performance and health status.
Building Block 4: Financing	Adequate health funding that ensures people can use needed services and are protected from financial catastrophe or impoverishment associated with having to pay for them.
Building Block 5: Medicines and Technologies	Equitable access to essential medical products, vaccines and technologies of assured quality, safety, efficacy and cost-effectiveness, as well as their scientifically sound and cost-effective use.
Building Block 6: Governance	Ensuring strategic policy frameworks exist and are combined with effective oversight, coalition building, the provision of appropriate regulations and incentives, attention to system design and accountability.
Central Pillar: People	The role of people is central as mediators, beneficiaries, and actors driving health systems, including as individuals, civil society organizations, stakeholder networks and key actors influencing each of the building blocks, as health workers, managers, and policymakers.

Table 2 outlines the limitations encountered during the implementation of the selected methods for this evaluation, and the mitigation strategies put in place to ensure that evaluation findings were sufficiently robust.

Table 6: Limitations and Mitigation Strategies

Limitation	Impact	Mitigation strategy
Limited information found in annual reports submitted by the territories.	This can affect development of an understanding of program implementation, outcome assessment and identification of lessons learned.	The evaluation team conducted in-depth interviews to understand the context of program implementation, and identify achievements and lessons learned.
Interviews are retrospective in nature, providing only a recent perspective on past events.	This can affect the validity of assessments of activities or results that may have changed over time.	Triangulation with other lines of evidence substantiated or provided further information on data captured in the interviews. The document review also provided corporate knowledge.
There was limited data available and not enough time to assess long-term outcome.	Assessment of progress towards related outcomes is challenging.	The evaluation focused on other outcome areas and used triangulation with other lines of evidence to the extent possible. Insufficient data and time were also raised as issues in the report under Achievement of Outcomes, as well as in the Conclusions and recommendations.

The evaluation applied a Sex and Gender-Based Analysis Plus lens to its assessment of the THIF program in addition to follow carefully the actions identified under the Truth and Reconciliation Commission’s Calls to Action and the Inuit Nunangat Policy. Although an examination of the Sustainable Development Goals was not directly applicable for this evaluation, it was noted that THIF funding directly contributed to Goal 3 in ensuring healthy lives and promoting well-being for all at all ages, and particularly to Goal 3.3, which is to end the epidemic of tuberculosis by 2023, and Goal 3.5, which is to strengthen the prevention and treatment of substance abuse, including narcotic drug abuse and harmful use of alcohol. By funding locally planned and designed projects, THIF provided a platform for territorial governments to collaborate with Indigenous representatives to address local health needs. In this regard, THIF may be seen to contribute to Goal 16.7, which is to ensure responsive, inclusive, participatory, and representative decision-making at all levels.

In conducting the evaluation, a single window was identified with the Health Care Strategies Directorate within the Strategic Policy Branch, with whom the Office of Audit and Evaluation worked closely throughout the evaluation. The scope for this evaluation was shared with the Strategic Policy Branch’s Assistant Deputy Minister and Associate Assistant Deputy Minister, as well as with the Health Canada Deputy Minister to help guide the evaluation questions. The final report and Management Response and Action Plan developed by the Strategic Policy Branch were presented to the Health Canada Performance Measurement, Results and Evaluation Committee (PMERC) in June 2023.

Annex C: Performance Indicators

Legend

Target met

Performance target established by the territory was met.

In progress; data not yet available

While progress is being made on this indicator, the indicator cannot be fully assessed due to insufficient data and elapsed time to achieve target.

Unmet

Performance target established by the territory could not be met.

Table 7: Medium-term performance indicators (5 to 10 years)

Indicator	Target Met	In progress; data not yet available	Unmet
Northwest Territories
75% of Health and Social Services staff have received training in cultural competency and safety.			X
An operational review of management and governance structures in DHSS and HSSAs was completed.		X
8 primary care health reform demonstration projects were held by region.		X
20,000 patients were assigned to Integrated Care Teams.		X
Yukon
5% of patients connected to remote patient care technologies for their health care services.	X
96% of individuals who participated in THIF-funded training reported that they strongly agree or agree that they will use the knowledge and skills they learned to provide better client care.	X
300 staff received cultural safety and humility training.		X
300 Yukoners accessed primary health care at the Centre de santé Constellation Health Centre.	X
40 Putting People First recommendations were partially or fully implemented.	X
Nunavut
100% of communities with active tuberculosis cases had teams available to support a response.	X
75% of the total number of Community Oral Health Coordinator positions in the Oral Health Program were filled.			X
50% of communities had individuals access health care services via a new e-Health. Application	X
60% of Mental Health and Addictions paraprofessional positions were filled.			X
Northwest Territories
90% of individuals reported the care they received from their health care system as being good or excellent.			X
Yukon
100% of individuals receiving supports through THIF-funded initiatives reported that they strongly agree or agree that their needs were met.	X
80% of programs were engaged in the adoption of an interoperable EMR and data system by 2023Footnote *.	X
Nunavut
100% of individuals successfully completed their tuberculosis treatment plan.		X
Table 7 Notes Table 7 Note 1 This indicator was achieved ahead of target within the evaluation period. As of 2021, the indicator is no longer used in THIF reporting given that this work has transitioned to ongoing, core funding in Yukon. Return to table 7 note * referrer

Table 8: Long-term performance indicators (more than 10 years)

Indicator	Target Met	In progress; data not yet availableFootnote *	Unmet
Northwest Territories
10% reduction occurred in percentage of premature and potentially avoidable deaths.		X
64% of population reported their health as excellent or very good.		X
Yukon
Territory remained below the national average for rate of mortality from treatable causes.		X
5% reduction occurred in rate of mortality from preventable causes.		X
60% of population reported their health as excellent or very good.	X
Nunavut
6% reduction occurred in rate of premature and potentially avoidable mortality.		X
40% of population reported their health as excellent or very good.		X
Table 8 Notes Table 8 Note 1 Long-term data reporting is done by Statistics Canada through Canadian Vital Statistics, and this data was not available during the evaluation time period. Return to table 8 note * referrer