Evaluation of the Canada Student Loan Forgiveness for Family Doctors and Nurses Benefit - Canada Student Financial Assistance Program

On this page

Alternate formats

Evaluation of The Canada Student Loan Forgiveness for Family Doctors and Nurses Benefit [PDF - 1,17 KB]

Large print, braille, MP3 (audio), e-text and DAISY formats are available on demand by ordering online or calling 1 800 O-Canada (1-800-622-6232). If you use a teletypewriter (TTY), call 1-800-926-9105.

List of figures

List of tables

List of abbreviations

CA
Census agglomeration
CMA
Census metropolitan area
CSD
Census subdivision
CSFA
Canada Student Financial Assistance program
CSL
Canada Student Loan
ESDC
Employment and Social Development Canada
FPTCHW
Federal/Provincial/Territorial Committee on Health Workforce
HC
Health Canada
NAGSFA
National Advisory Group on Student Financial Assistance
PASRB
Public Affairs and Stakeholder Relations Branch
RAP
Repayment Assistance Plan

List of key terms and expressions

Designated communities
Under-served, rural or remote communities in Canada that are designated by the Program as meeting the eligibility requirements for doctors and nurses to receive the benefit.
Communities
Canadian census subdivisions, defined by Statistics Canada as an area that is a municipality or an area that is deemed to be equivalent to a municipality for statistical reporting purposes.
Beneficiaries
Doctors and nurses who received the loan forgiveness benefit.
Non-Beneficiaries
The non-beneficiaries consisted of 2 groups: 1. All the Doctors and nurses with Canada student loan of any amount in good standing at the inception of the benefit. 2. Doctors and nurses currently in training (students) across medical and nursing schools in Canada.
Doctors
Family physicians and family medicine residents.
Nurses
Registered nurses, registered psychiatric nurses and nurse practitioners.
Key stakeholder groups
Managers of healthcare services in provinces and territories, healthcare institutions, medical student groups.
Applicants
Everyone who applied for the benefit, regardless of the results. All beneficiaries are applicants but not all applicants are beneficiaries.
The benefit
The Canada Student Loan Forgiveness for Family Doctors and Nurses Benefit is an initiative that falls under the Canada Student Financial Assistance program. Both terms will be used throughout the report for the same meaning.

Introduction

The Canada Student Loan Forgiveness for Family Doctors and Nurses benefit is a Government of Canada initiative instituted in fiscal year 2012 to 2013 in accordance with the Budget 2011 announcement. The initiative falls under the Canada Student Financial Assistance (CSFA) Program which provides partial student loan forgiveness to eligible healthcare professionals, including family doctors, nurses and nurse practitioners, who practised in an under-served rural or remote community.

This is the first summative evaluation of the benefit's relevance and effectiveness since inception. It is based on 4 lines of evidence and covers the period spanning from fiscal years 2013 to 2014 and 2021 to 2022. The evaluation aims to determine the extent to which the benefit has achieved its objectives. To do so, 3 evaluation questions were developed to guide inquiry into the benefit. The full list of evaluation questions can be found in Appendix B.

Summary of findings (with recommendations)

Key findings

There are 11 main findings from the evaluation:

  1. intended beneficiaries learn about the benefit from different sources, with word-of-mouth being the most common source of information
  2. a low proportion of intended beneficiaries (non-beneficiaries) are aware of the details of the benefit. Additionally, there is varied awareness of the benefit among different professional and student sub-groups
  3. beneficiaries found the application process and requirements to be straightforward, manageable, and attainable. Among the small number of rejections, most were commonly due to application errors (35%) and resulting processing delays
  4. a large majority of key stakeholders in the Provinces and Territories have at least some awareness of the benefit. However, inadequate communication about the benefit was identified as a barrier to awareness
  5. the number of doctors and nurses receiving loan forgiveness increased between 2013 and 2021, particularly over the early years of the benefit
  6. non-beneficiaries and other stakeholders report that the amount of the benefit may not be sufficient on its own to incentivise doctors and nurses to relocate to designated communities
  7. factors other than the benefit, such as debt levels, cost of living, or personal motivations, contribute to influencing the decision to work in designated communities
  8. since benefit inception, beneficiaries provided services to an increasing number of communities (611 in 2019) across the country
  9. the majority (69%) of the beneficiaries surveyed continued to work in the designated communities after they were no longer eligible for loan forgiveness
  10. several factors, such as loan amounts, family status, distance from central metropolitan area, and province of work, contribute to how long doctors and nurses claim the benefit
  11. key informants, including those from provinces and territories, were unable to comment on the contribution of the benefit to the expansion of primary healthcare services. However, a majority of professionals and students held the perception that it will

Recommendations

Based on these findings, the evaluation provides the following 2 recommendations to the program:

  1. the program should explore outreach opportunities to increase awareness of the benefit among key stakeholders, especially among intended beneficiaries
  2. the program should explore the development of tools to help consistently measure and monitor the program’s benefit

Management response and action plan

Executive summary

Canada Student Loan (CSL) forgiveness for doctors and nurses is a federal benefit administered by the Canada Student Financial Assistance (CSFA) Program in the Learning Branch of Employment and Social Development Canada (ESDC). As a program component, this benefit forgives a portion of federal loans for family doctors and residents, nurse practitioners, and nurses that practice in under-served rural and remote communities. Its objective is to attract and retain health providers in rural and remote communities by offering financial incentives.

Overall management response

This management response addresses the evaluation recommendations, provides information about recent actions undertaken by the Learning Branch and outlines plans for improvements based on the evaluation findings and recommendations.

Some key findings from the evaluation of the CSL forgiveness benefit emphasize the importance of increasing the level of awareness among intended beneficiaries and that the benefit on its own may not be sufficient to incentivise doctors and nurses to relocate to designated communities.

While the evaluation reports that the CSL forgiveness benefit does reach rural communities in Provinces and Territories and is achieving its intended results, two areas have been identified where improvement may be possible to enhance awareness and continued monitoring and assessment of this benefit.

Recommendation 1: The program should explore outreach opportunities to increase awareness of the benefit among key stakeholders, especially among intended beneficiaries

Management response

The Learning Branch generally agrees with this recommendation and will explore opportunities to integrate CSFA Program information about the CSL forgiveness benefit into broader awareness activities of the Branch to ensure increased awareness of this benefit among key stakeholders. The Learning Branch actively consults stakeholders and Provinces and Territories on CSL forgiveness for doctors and nurses, through communications, engagement, and information sharing activities on an ongoing basis. The Department’s Public Affairs and Stakeholder Relations Branch (PASRB) will continue to support and complement outreach activities to increase public awareness of the program and its benefit.

The Program has leveraged the expertise of other government departments such as Health Canada (HC) in the context of policy development to ensure federal initiatives are aligned. The Program could also explore opportunities to continue leveraging HC’s provincial/territorial health authorities stakeholder networks on strategies to expand awareness of the benefit.

Management action plan

1.1 Explore ways to increase awareness of the CSL forgiveness benefit for doctors and nurses through work with the Public Affairs and Stakeholder Relations Branch to develop and implement a plan to raise awareness and outreach of all Learning Branch programs among potential beneficiaries.

1.2 Engage with federal partners (for example: Health Canada), and provincial and territorial partners to expand awareness of the benefit and its enhancements among health care providers in rural and remote communities.

Recommendation 2: The program should explore the development of tools to help consistently measure and monitor the program’s benefit

Management response

The Learning Branch agrees with this recommendation and will explore ways to measure and monitor the Program’s benefit.

The current evaluation recognizes that the CSL forgiveness benefit for doctors and nurses is one of the many intergovernmental initiatives that seeks to improve health care access in rural and remote areas. Therefore, measuring impacts and changes to health care access because of this benefit alone is challenging.

The Learning Branch recognizes the importance of data collection to more effectively track, measure and monitor the benefit. As such, there is an opportunity to explore the feasibility of conducting a survey of CSL forgiveness beneficiaries in order collect the necessary data to consistently measure and monitor the program’s benefit.

In addition, the feasibility of tracking additional variables could also be explored with provincial and territorial partners. The CSFA Program currently monitors the benefit through a number of internal key performance indicators. These indicators provide valuable information on the benefit, such as the benefit’s uptake, usage length, and the amount of loans forgiven. The collection of additional disaggregated variables, such as more detailed program of study information, would allow the program to better measure the performance of the benefit.

Management action plan

2.1 Explore the feasibility of conducting an annual survey of CSL forgiveness beneficiaries for the purpose of measuring and monitoring the effectiveness of the benefit.

2.2 Explore with provincial and territorial partners the feasibility of tracking additional variables, such as specific program of study variables, to better disaggregate intended beneficiaries.

Background

The Canada Student Loan Forgiveness for Family Doctors and Nurses benefit is a component of the Canada Student Financial Assistance Program developed by the Government of Canada initiative instituted in fiscal year 2012 to 2013 in accordance with the Budget 2011Footnote 1 announcement. The benefit provides partialFootnote 2 student loanFootnote 3 forgiveness to eligible healthcare professionals, including family doctors, nurses and nurse practitioners, who practised in an under-served rural or remote community.

The benefit's specific objectives are to:

Eligibility

Profile of beneficiaries

From 2012 to 2020, 15,390 unique beneficiaries applied for the benefit (2,785 doctors and 12,605 nurses):

Benefits amount

To incentivise healthcare professionals to relocate to and practice in designated communities, the benefit provides family doctors with student loan forgiveness of up to $8,000 per year to a maximum of $40,000 over 5 years. It also provides nurses and nurse practitioners with student loan forgiveness of up to $4,000 per year to a maximum of $20,000 over 5 years.

From 2013 to 2021, the benefit forgave over $140 million in Canada Student Loan debt.

Due to benefit design, the total amount forgiven in any given year is dependent on the number of eligible applications received.

Since the inception of the Canada Student Loan forgiveness for family doctors and nurses (up to March 31, 2022), there have been 17,921 beneficiaries and $172.2 million in loan amount forgiven.

In fiscal year 2021 to 2022, $25 million of loans were forgiven for almost 5,400 recipients through the Canada Student Loan forgiveness benefit for family doctors and nurses. In the same fiscal year, there were $12.4 billion loans in repayment. The expenditures for Canada Student Loan forgiveness for doctors and nurses represent 0.2% of the total loan portfolio within the CSFA Program.

Figure 1. Loan amount forgiven since inception of the benefit (in thousands of dollars) by profession
Figure 1. Loan amount forgiven since inception of the benefit (in thousands of dollars) by profession: description follows
Figure 1: Text description
Profession Fiscal year 2013 to 2014 Fiscal year 2014 to 2015 Fiscal year 2015 to 2016 Fiscal year 2016 to 2017 Fiscal year 2017 to 2018 Fiscal year 2018 to 2019 Fiscal year 2019 to 2020 Fiscal year 2020 to 2021
Doctors N/A N/A 3,631 5,388 6,346 6,988 7,000 6,135
Nurses N/A N/A 12,864 15,247 16,544 17,268 17,136 13,563
Total 6,671 12,364 16,495 20,635 22,889 24,256 24,135 19,689
Figure 2. Average loan forgiveness amount (in thousands of dollars) by profession
Figure 2. Average loan forgiveness amount (in thousands of dollars) by profession: description follows
Figure 2: Text description
Fiscal year Average loan: Doctors (in thousands of dollars) Average loan: Nurses (in thousands of dollars)
Fiscal year 2015 to 2016 7.611 3.811
Fiscal year 2016 to 2017 7.515 3.846
Fiscal year 2017 to 2018 7.618 3.793
Fiscal year 2018 to 2019 7.489 3.759
Fiscal year 2019 to 2020 7.543 3.756
Fiscal year 2020 to 2021 7.574 3.796

Most of the debt forgiveness granted has been directed towards nurses and nurse practitioners, even though their profession qualifies for a lower maximum amount of debt forgiveness compared to doctors. This is likely because the healthcare system employs a significantly larger number of nurses than doctors. For instance, in 2021, the count of physicians supplying healthcare services was 93,998, compared to 459,005 regulated nurses (according to CIHI data). Due to this higher representation, more nurses have the potential to relocate and thereby become eligible to receive the benefit of debt forgiveness.

Evaluation approach

This summative evaluation uses a mixed-method approach with 4 key lines of evidence to evaluate the extent to which the benefit is achieving its objectives. Evidence from the lines of evidence were triangulated to identify key evaluation findings. There were 3 questions developed to guide the evaluation. They can be found in Annex B.

The evaluation covers the period from fiscal year 2013 to 2014 until fiscal year 2021 to 2022.

Figure 3. Collected evidence was triangulated to arrive at evaluation findings
Figure 3. Average loan forgiveness amount (in thousands of dollars) by profession: description follows
Figure 3: Text description

Different lines of evidence are triangulated to arrive at the evaluation findings.

  1. Document and literature
    1. Examined key academic literature
    2. Reviewed internal program and external government documents
    3. Included a jurisdictional review of similar programming
  2. Key informant interviews
    1. Interviewed 29 key stakeholders
    2. Consulted internal, provincial, professional, and post-secondary stakeholders
  3. Surveys
    1. Targeted students and doctors and nurses who are non-beneficiaries
    2. Surveys completed by 1,913 students and 5,761 professionals
    3. Results compared with a previous survey of beneficiaries
  4. Data review
    1. Combined program administrative data with key health datasets. For instance, datasets from the Canadian Institute of Health Information (CIHI)
    2. Examined outcome trends of benefit participants
    3. Included geospatial mapping to show reach and spread

Limitations

Key findings

Finding 1: Intended beneficiaries learn about the benefit from different sources, with word-of-mouth being the most common source of information

The benefit‘s ultimate outcome is to expand primary care services in under-served rural and remote communities. According to the benefit’s logic model, doctors and nurses must be aware of the benefit in order to be incentivized to relocate to designated communities.

Consultations and communication with key stakeholder groupsFootnote 7 occurred as part of the regulatory development process in the fall of 2011Footnote 8.

Additional engagement with these stakeholder groups, as well as with the National Advisory Group on Student Financial Assistance (NAGSFA) occurred in Fall 2011 prior to the 30-day public comment period for the proposed regulations published in the Canada Gazette, Part I and the publication of amended regulations in the Canada Gazette, Part II.

Since these early engagement activities, there has been limited activity on the part of ESDC to create awareness of the benefit among intended beneficiaries or other key stakeholders. Program representatives indicated that awareness creation and stakeholder engagement have not historically been a focus of program activity but noted that stakeholder consultations are currently taking place at the time of writing the report in response to the Budget 2022 benefit enhancement announcement.

Level of participation in the consultation and engagement activities

While the early consultation and engagement activities likely contributed to creating awareness of the benefit among key stakeholder groups, more than a decade has elapsed since they occurred. Therefore, while a few (n=4) provincial/territorial key informants reported that their jurisdiction had been involved in stakeholder consultations during the design phase of the benefit, most (n=14) were unsure if their jurisdiction had been involved. All key informants representing student financial aid administrators and professional associations (n=7) were unaware of these initial consultations.

Only a few (n=3) provincial/territorial key informants out of 18 interviewed reported being involved.

Among intended beneficiaries (actual beneficiaries and non-beneficiaries), word-of-mouth (family and friends) is the most common source of information about the benefit followed by post-secondary institutions, the national student loans service centre, and the Government of Canada website respectively.

According to the Doctors and Nurses Loan Forgiveness Program: Survey of Recipients - December 8, 2020:

Based on the non-beneficiary surveys: Survey of Professionals 2022 and Survey of Students 2022:

Results from both surveys of professionals (beneficiaries and non-beneficiaries) indicate that more doctors than nurses refer to their post-secondary institutions to learn about the benefit, while more nurses than doctors find out about the benefits from the National Student Loans Service Centre and from the Government of Canada website.

Considering a related provincial program can help identifying approaches for improving benefit communication which may also be applicable to the federal context. A survey conducted in 2017 with 131 recipients of the Saskatchewan Student Loan Forgiveness for Nurses and Nurse Practitioners indicates that most of the recipients learned about that program through the provincial government website (50%), followed by employers or co-workers (31%) and educational institutions or instructors (27%).

Respondent suggestions to improve program awareness and communications include:

Finding 2: A low proportion of intended beneficiaries (non-beneficiaries) are aware of the details of the benefit: additionally, there is varied awareness of the benefit among different professional and student sub-groups

Awareness of the benefit among non-beneficiaries

Among intended beneficiaries (non-beneficiariesFootnote 9), 64% of professionals and 50% of students have at least some awareness of the benefit:

Figure 4. Awareness of the benefit among non-beneficiaries
Chart of insert chart title: description follows
  • Sources: Survey of Professionals 2022; Survey of Students 2022.
Figure 4: Text description
Survey response Professionals (n=5,761) Students (n=1,913)
Yes (I know the eligibility details and benefit amounts) 23% 10%
Somewhat (I know that the benefit exists) 41% 40%
No (I have never heard about the benefit) 36% 50%

Awareness of the benefit: Comparison among non-beneficiary groups

Figure 5. Comparison of awareness of the benefit among non-beneficiary groups
Chart of insert chart title: description follows
  • Sources: Survey of Professionals 2022; Survey of Students 2022.
Figure 5: Text description
Survey response Doctors (n=794) Nurses (n=4,967) Medical students (n=401) Nursing students (n=1,512)
Yes (I know the eligibility details and benefit amounts) 34% 21% 13% 10%
Somewhat (I know that the benefit exists) 35% 43% 56% 36%
No (I have never heard about the benefit) 31% 36% 31% 55%

The non-beneficiary surveys showed that among both professionals and students, awareness of the benefit varies by region and area:

Finding 3: Beneficiaries found the application process and requirements to be straightforward, manageable, and attainable: among the small number of rejections, most were commonly due to application errors (35%) and resulting processing delays

The application process

How the application process was perceived by beneficiaries (2020 survey)

The process of applying for the benefit is perceived by beneficiaries to be straightforward and manageable:

Non-beneficiaries’ experiences

The 2022 survey of professionals who are not beneficiaries offers further insight into the application process from the user’s perspective. Only a few (17%) respondents reported that they currently work in a community they believe to be an under-served rural or remote community. Of these:

Of the small number of respondents who applied for the benefit but were rejected, the most common reasons given were:

Additionally, 18% of these respondents did not know why their application was rejected.

The current application process is generally seen as straightforward and manageable with ‘easy to meet’ eligibility requirements. There were some application errors which led to benefit rejections and nearly half of applicants were unaware of their application status. It may therefore be worth revisiting the application process and enhancing communication with applicants to ensure the user experience is as streamlined as possible.

Finding 4: A large majority of key stakeholders in the Provinces and Territories have at least some awareness of the benefit: however, inadequate communication about the benefit was identified as a barrier to awareness

Awareness of the benefit among key stakeholders

A large majority (n=21) of the stakeholders in the provinces and territories were familiar with the benefit and learned about it through various channels.

Figure 6. Channels where the provincial and territorial stakeholders hear about the benefit
Figure 6. Channels where the provincial and territorial stakeholders hear about the benefit: description follows
  • Sources: Survey of Professionals 2022; Survey of Students 2022.
Figure 6: Text description

Provincial and territorial stakeholders hear about the benefit through various channels.

  • Involvement in the FPTCHW (n=2)
  • Internal sources (n=6)
    • Recipients
    • Environmental staff
    • Other staff
  • Media or public information sources (n=6)
  • Presentation by ESDC (n=1)

Barriers to awareness of the benefit

The majority of these key informants perceive that the primary barrier to awareness of the benefit is information not being communicated to, or reaching, relevant stakeholders.

Figure 7. Barriers to awareness
Figure 6. Channels where the provincial and territorial stakeholders hear about the benefit: description follows
  • Sources: Survey of Professionals 2022; Survey of Students 2022.
Figure 7: Text description

There are various barriers to awareness of the benefit among stakeholders.

  • Communication gap between the program and intended beneficiaries
  • Competing priorities within the healthcare sector
  • Communication gap among professional associations
  • High staff turnover in healthcare and student financial aid fields

As already noted, the Program representatives indicated that creating awareness was not built into the benefit activity plans.

The benefit has relied on word of mouth for the dissemination of information by key stakeholders in the provinces and territories including the financial advisors at the schools of medicine and nursing.

There was a low response rate among student financial aid advisors. Feedback received suggests that it may be due to a lack of familiarity with the benefit, suggesting this group of key stakeholders may be less aware than others.

Strategies to improve awareness of the benefit

This finding is consistent with those from a survey of recipients of a similar program to the federal benefit, Saskatchewan's Student Loan Forgiveness for Nurses and Nurse Practitioners. The need to increase awareness of both the provincial and federal programs was identified and the following strategies were suggested by recipients:

Key informants in this evaluation made similar suggestions. Additionally, they proposed:

Finding 5: The number of doctors and nurses receiving loan forgiveness increased between 2013 and 2021, particularly over the early years of the benefit

Given that there are no official benefit indicators, the evaluation considered how the number of beneficiaries has increased since inception to investigate how the benefit may be incentivising health professionals to work in designated communities.

As of 2020, there have been 15,390 unique beneficiaries who applied for the benefit (2,785 doctors and 12,605 nurses).

Key informants suggest that benefit uptake could be increased further by:

  1. enhancing promotion of the benefit (n=9)
  2. expanding benefit eligibility to more health care professions (n=9)
Figure 8. Total number of beneficiaries by fiscal year
Figure 8. Total number of beneficiaries by fiscal year: description follows
Figure 8: Text description
Year Number of doctors and nurses
2013 to 2014 1,580
2014 to 2015 2,849
2015 to 2016 3,853
2016 to 2017 4,682
2017 to 2018 5,195
2018 to 2019 5,527
2019 to 2020 5,490
2020 to 2021 4,383

Figure 8 demonstrates that the number of beneficiaries has increased since fiscal year 2013 to 2014:

Finding 6: beneficiaries and other stakeholders report that the amount of the benefit may not be sufficient on its own to incentivise doctors and nurses to relocate to designated communities

Evidence from the academic literature suggests that while loan forgiveness programs can have a positive impact on recruitment of healthcare professionals to rural and remote areas, they are insufficient on their own to influence decisions to remain in those areas. In studies that attempted to estimate what a sufficient benefit would be to incentivize relocation, the proposed amount is usually vastly beyond what is offered by the programFootnote 10.

This evidence is corroborated by the surveys of professional recipients and non-recipients of the benefit. Specifically, the evaluation found:

Beneficiaries

About a fifth (21%) of beneficiaries reported that the loan forgiveness benefit was “very impactful” to their decision to work in designated communities:

Without the benefit, 38% of doctors and 42% of nurses (41% combined) reported that they would be unlikely to move to a designated community.

Non-beneficiariesFootnote 11

Less than half (45%) of non-beneficiary professionals agreed that the value of the loan forgiveness benefit is sufficient to incentivise them to relocate to designated communities:

By contrast, 67% of doctors and 70% of nurses (69% combined) would be unlikely to move to designated communities without the benefit.

For students, 57% of medical students and 52% of nursing students (52% combined) would be unlikely to relocate for work if the benefit did not exist.

Key stakeholders provided further insight as to why this may be the case. Most key informants (n=19) reported that the value of the loan forgiveness benefit is insufficient to incentivise doctors and nurses to relocate to designated communities. The most frequent reason given (n=13) was that there are numerous other, larger incentives available. These include, for example:

Finding 7: Factors other than the benefit, such as debt levels, cost of living, or personal motivations, contribute to influencing the decision to work in designated communities

One reason why the benefit may not be sufficient on its own comes from the fact that there are several factors in addition to the benefit that affect individuals' choices. Evidence gathered in the evaluation supports this notion and reveals what factors are influencing decisions to work in designated communities.

Academic literature identifies several non-financial barriers to relocation which may impact doctors' and nurses' decisions to relocate to designated communities. These include:

Key informants identified a number of barriers to relocation, which may contribute to limited interest among doctors and nurses. These include:

Survey data also highlights several factors influencing the decision to work in designated communities among beneficiaries and students.

Beneficiaries

Taking advantage of the loan forgiveness benefit was the third most common reason for working in remote or rural communities (34%).

The most common reasons given for working in a designated community related to being familiar with and preferring to live in a rural or remote community (44%), followed by gaining experience in the health field (40%).

Students

Finding 8: Since benefit inception, beneficiaries provided services to an increasing number of communities (611 in 2019) across the country

Beneficiaries provided primary healthcare services in designated underserved, rural, and remote communities across the 13 Canadian provinces and territories. However, distribution of Quebec, the Northwest Territories and Nunavut should be interpreted differently than other provinces and territories as they have their own student financial assistance programs. Both maps depicted in Figures 9 and 10 demonstrate the considerable ‘reach’ of the benefit over the years. Note that the benefit is not designed with provincial-based targets.

Figure 9 shows growth in the number of doctors and registered nurses working in the designated communities (each community represented by a point) over the 8-year period between 2012 and 2019. Growth is calculated by comparing the total number of beneficiaries working in a designated community in the first half of the benefit (2012 to 2015) with the number working therein during the latter half (2016 to 2019). The green dots represent increases in the number of beneficiaries in communities across Canada.

Figure 9 Change in CSD coverage from the first half of the benefit (2012 to 2015) to the latter half (2016 to 2019)
Figure 9 Change in CSD coverage from the first half of the benefit (2012 to 2015) to the latter half (2016 to 2019): description follows
  • Source: ESDC Geomatics team using the benefit’s data on number of beneficiaries from 2012 to 2019.
Figure 9: Text description

Figure 9 is a map of Canada representing the change in census subdivision coverage from the first half of the benefit (2012 to 2015) to the latter half (2016 to 2019). Census subdivisions in which at least one beneficiary was active from 2012 to 2019 are represented by three colours of dots:

  • CSDs in which there was an increasing number of beneficiaries from the first to the second half of the benefit are represented by green dots
  • CSDs in which there was a stable number of beneficiaries are represented by yellow dots
  • CSDs in which there was a decreasing number of beneficiaries are represented by red dots

The dots are spread across the entire map. Most dots are clustered in the areas around large populated cities, closer to the border with the United States.

Figure 10 shows the extent to which the benefit is sustaining its reach into designated communities by highlighting how consistently a given community (represented by a point) was serviced from 2012 to 2019. Note that communities may be served consistently but with different individual beneficiaries for some years.

Figure 10. Number of years each CSD was serviced by at least 1 benefit recipient
Figure 10. Number of years each CSD was serviced by at least 1 benefit recipient: description follows
  • Source: ESDC Geomatics team using the benefit’s data on number of beneficiaries from 2012 to 2019.
Figure 10: Text description

Figure 10 is a map of Canada representing the number of years census subdivisions were serviced by at least 1 benefit recipient over the course of 8 years from 2012 to 2019. Each census subdivision which was serviced by at least 1 beneficiary over the period is represented by a shaded dot, with different shadings indicating how many of those years at least one beneficiary was active in the community.

Most of the dots fall into the extremes, showing mostly 1 to 2 or 7 to 8 years of activity with at least 1 beneficiary.

The map shows the extent to which the benefit is sustaining its reach into designated communities by highlighting how consistently communities were serviced by beneficiaries. Communities may be served consistently but with different individual beneficiaries.

Geographic spread

In 2019, 8% of doctors and 4% of nurses working in rural and remote communities were receiving the benefit.

Communities across the country were served by beneficiaries. For some Provinces and territories jurisdictions, these beneficiaries made up over 10% of all doctors and nurses in recent years.

From 2012 to 2020, 942 unique communities benefitted from having a nurse who was a beneficiary, and 501 communities benefitted from a doctor who was a beneficiary.

Beneficiaries worked in communities that are considerably far from census metropolitan areas:

There is considerable variation by province, and Tables E-1 and E-2 in Annex E present further details.

The total number of communities with at least 1 doctor or nurse beneficiary is generally trending upward. The increase was more noticeable as the benefit was developing from 2012 to 2015. The fluctuations observed in the latter years of the benefit may be due to several factors, but could not be determined within the scope of the evaluation.

Figure 11 below depicts these trends disaggregated by profession. Figure 12Footnote 12 gives the overall trend. Note that the total number for a given year is less than the sum of doctors and nurses as both may be working in a given community.

Figure 11. NumberFootnote 13 of census subdivisions (CSDs) being served by doctors and nurses on the benefit
Figure 11. Number of census subdivisions (CSDs) being served by doctors and nurses on the benefit: description follows
  • Source: Canada Student Financial Assistance program data.
Figure 11: Text description
Year CSDs with Doctors CSDs with Nurses
2012 75 363
2013 162 503
2014 212 567
2015 255 595
2016 284 597
2017 319 631
2018 308 634
2019 284 563
Figure 12. Total number of unique census subdivisions being served by doctors and nurses on the benefit
Figure 12. Total number of unique census subdivisions being served by doctors and nurses on the benefit
  • Source: Canada Student Financial Assistance program data.
Figure 12: Text description
Year Number of unique CSDs
2012 382
2013 532
2014 601
2015 637
2016 641
2017 677
2018 685
2019 611

Finding 9: The majority (69%) of the beneficiaries surveyed continued to work in designated communities after they were no longer eligible for loan forgiveness

The evaluation was not able to track beneficiaries beyond their time on the benefit, nor was it designed to track behaviour of all beneficiaries longitudinally over a period of time. However, survey evidence provides insight into beneficiary behaviour beyond the time spent on the benefit.

In the 2020 survey of beneficiaries, a majority (69%) of those who were no longer on the benefit continued to remain in a designated community after their benefit eligibility expired (63% of Doctors and 70% of Nurses). Of note:

Finding 10: Several factors, such as loan amounts, family status, distance from central metropolitan area, and province of work, contribute to how long doctors and nurses claim the benefit

Beneficiaries can receive the benefit for a maximum of 5 years. By performing an odds ratio estimates from a logistic regression on program administrative data, it was possible to determine the socio-demographic and loan characteristics influencing beneficiaries' decisions to remain on the benefit over multiple years.

The full logistic regression results for doctors and nurses can be found in Figures E-1 and E-2 in Annex E.

Figure 13. Number of years claiming the benefit, by profession
Figure 13. Number of years claiming the benefit, by profession
  • Source: Canada Student Financial Assistance program data.
Figure 13: Text description
Number of Years Doctors Nurses
1 year 54% 38%
2 years 24% 28%
3 years 14% 17%
4 years 6% 10%
5 years 2% 6%

Figure 13 depicts the number of years that doctors and nurses spent on the benefit while working in designated communities.

There is some evidence of differential impact among the beneficiaries (finding 6). In addition, there are other factors besides the benefit which seem to influence doctors' and nurses' decisions to relocate (finding 7) and stay (finding 10) in designated communities. The propensity of some of these factors also seem to vary across different groups. It may therefore be worth investigating further to discern whether there are opportunities to adjust the benefit in better attracting doctors and nurses from different backgrounds.

Finding 11: Key informantsFootnote 14, including those from provinces and territories, were unable to comment on the contribution of the benefit to the expansion of primary healthcare services: however, a majority of professionals and students held the perception that it will

Ultimately, the loan forgiveness benefit is expected to contribute to the expansion of primary healthcare in designated communities, where “expansion” refers to the number of family doctors and nurses working in these communities.

Overall, 55% of professionals and 74% of students consider it likely that the benefit will contribute to expansion of the primary healthcare services.

Key stakeholders were mostly unable to comment on the extent to which the benefit is contributing to the expansion of primary healthcare services:

It should also be noted here that key informants were asked to comment on the impact of growing adoption of telemedicine on the movement of family physicians and nurses to designated communities for the provision of primary health care services. Most key informants (n=15) reported that the intention of using telemedicine in their jurisdiction was not to replace in-person care. Some key informants (n=7) viewed telemedicine as an opportunity to increase the number of doctors and nurses relocating to designated communities, addressing some of the stress associated with being the only health care practitioner in the community.

Conclusion: Mapping of recommendations and findings

Recommendation 1: The program should explore outreach opportunities to increase awareness of the benefit among key stakeholders, especially among intended beneficiaries

Evidence collected during the evaluation suggests that the program undertook extensive consultations prior to launching the benefit. Detailed information about the benefit eligibility, and the application process is easily accessible online on the government website. However, since its launch awareness-creation has not been a key component of the benefit. As such, word-of-mouth has been the most common means of communication about the benefit among students and professionals [Finding 1]. This has resulted in a majority of intended beneficiaries not being aware of the benefit and disproportionate levels of awareness among certain groups [Finding 2]. Key provincial and territorial stakeholders are aware of the benefit, but they identified inadequate communication to be a barrier to awareness. [Finding 4]. They further suggest that increased promotion of the benefit may increase take-up beyond its increased uptake since inception [Finding 5]. Engaging in targeted awareness promotion of the benefit, especially among intended beneficiaries who are least aware of the benefit, will likely lead to an increase in the number of applications and ultimately in the number of doctors and nurses relocating to designated communities.

Recommendation 2: The program should explore the development of tools to help consistently measure and monitor the program’s benefit

Results metrics were not identified for the program, which introduced challenges to the extent to which the evaluation may comment on the success of the benefit in meeting its objectives [Annex C]. To assess the extent to which the benefit is meeting its goals, evaluation collaborated with the program to develop a logic model and was able to identify several findings which indicate the extent to which the benefit is succeeding. These include:

Identifying result metrics and strengthening data integration strategy would allow the program to take indicators such as those found in the evaluation and make proper judgements about the successes of the benefit. It would better allow for improved data integration to inform policy analysis, research, and evaluation.

Annex A: Benefit logic model

Figure A-1. Benefit logic model
Figure A-1. Benefit logic model: description follows
Text description of Figure A-1

Ultimate outcome

Improved access to critical, front-line health care among individuals living in underserved, rural and remote communities in Canada.

Intermediate outcome

Increased relocation of doctors and nurses to underserved, rural and remote communities in Canada to provide critical, front-line healthcare services.

Immediate outcomes

  • Increased motivation of Doctors and Nurses to relocate to and work in underserved, rural or remote communities in Canada
  • Increased applications for DNLF benefit by doctors and nurses

Outputs

  • Awareness created about DNLF benefit for Doctors, Nurses, and key stakeholder groups
  • Processes, Systems, Policies and Guidelines for Doctors and Nurses Students' loan forgiveness application established

Activities

  • Create awareness about DNLF benefit for Doctors, Nurses, and Key stakeholder groups
  • Engage with key stakeholder groups about the DNLF program
  • Develop Policy materials and Guidelines for Doctors and Nurses Loan forgiveness program
  • Establish system and processes for loan forgiveness application

Inputs

  • Program budget, human resources, time

Annex B: Evaluation questions

The evaluation questions for this evaluation are as follows:

Annex C: Structural limitations

Unavailability of results metrics and a strong data integration strategy for the benefit introduced challenges to the assessment of the extent to which the benefit is achieving its goals and the ability of the evaluation to answer the evaluation questions relating to impact.

The unavailability of results metrics introduces a pair of challenges.

The purpose of a data integration strategy

A strong data integration strategy would allow for better informing policy analysis, research and evaluation.

A data integration strategy ensures unified and accurate data for informed decision-making. Without it, fragmented data might lead to inefficiencies and missed opportunities.

Due to its nature as a benefit, no performance information profile or other performance measurement framework was developed.

Why there is no performance measurement framework already

The Policy on Results requires a Performance Information Profile to be developed for all programs listed in the departmental Program Inventory. Because the benefit is not listed in the inventory, it is not required to develop one.

Annex D: Methodology

The evaluation of the Loan Forgiveness for Family Doctors and Nurses benefit used a mixed-method approach including 4 lines of evidence. This approach ensured adequate data triangulation to support robust evidence-based findings, conclusions, and recommendations.

Document and literature review

The document and literature review included:

A review of approaches and best practices from other jurisdictions (Canadian and international) was also conducted.

Key limitations

There was a relatively small number of internal documents available for review. As a result, the extent to which the document review could contribute to addressing the evaluation questions was limited.

Key informant interviews

A total of 29 interviews were conducted with a diversity of key stakeholders:

The evaluation used a sampling plan that targeted a specific number of individuals within the above categories of key stakeholders.

The scale used to report qualitative findings is as follows:

Key limitations

Despite many individuals (n=92) being identified, invited to participate, and contacted multiple times, fewer than the 36 targeted interviews were completed. Feedback suggests that a lack of familiarity with the benefit was a perceived barrier to participation for some. Of note:

Surveys of professionals and of students

Two distinct surveys of benefit non-recipients were carried out over the course of the evaluation:

For both surveys, a census strategy with saturation sampling was employed given available databases could not identify the target population. Thus, over 110,000 email survey invitations were sent to individuals categorized in the CSFA program as studying or having studied in the 'medicine' or 'health science' fields. Attempted responses by individuals not in the target population were screened out in the surveys.

In total, 1,913 student and 5,761 professional surveys were completed. The data was then cleaned to ensure accuracy and analysed in frequency tables and select cross-tabulations. For the text responses, a systematic random sample was chosen for coding.

Key limitations

While every effort was made to compare the results from the 2020 survey of beneficiaries and 2022 survey of non-beneficiary professionals, the survey questions were not identical. Comparisons were made where similarities were present, but caution is required when making direct comparisons between beneficiaries and non-beneficiaries based only on the survey reports.

The pool of non-beneficiaries was drawn from CSFA program data, which necessarily excludes doctors and nurses who never engaged with the program during their studies. As such, the opinions of this sub-group of non-beneficiaries are not accounted for in the professional survey.

Administrative and data analysis

The administrative data analysis combined data from various sources to examine outcome trends of family doctors and nurses who participated in the CSFA loan forgiveness benefit from fiscal year 2013 to 2014 and 2018 to 2019, the most recent data available at the time of analysis. The data analysis combined data from 2 primary sources:

Due to time constraints, the evaluation could not secure an approval to link data from Canada Revenue Agency T1 and T4 files with the CSFA data. This kind of linkage could have had the potential to enrich the analysis with additional data on labour market history and sociodemographic characteristics, subject to an adequate sample size following the linkage.

A regression analysis to examine factors influencing the time spent on loan forgiveness

Given that the available data consists only of beneficiaries, the extent to which the benefit incentivises doctors or nurses to work in under-served, rural or remote communities was limited. To perform such an analysis, a comparison group comprised of eligible non-beneficiaries would have been needed.

Instead, factors influencing the length of time spent on the benefit was examined using a logistic regression model under the premise that more time spent on the benefit implies longer time spent in a rural or remote community. The results allowed for the identification of factors likely exerting influence on the decision making of doctors and nurses when working in rural or remote communities.

By design, health care professionals are required to relocate first and work in the designated remote and rural areas for a year before receiving the benefit. In some cases, this translates in the application for the benefit occurring years after graduation. This means that the loan information from student records may be out of date by several years. To address this issue, the sample of doctors and nurses was limited to only those who opted for the benefit within their first year after graduation. This increases the likelihood that the loan application data used is current and valid.

The use of geomatics

The consolidated data file was also used by ESDC's geomatics team to produce several geo-spatial maps to visually represent the program's reach and spread across the country.

Key limitations: Examining financial incentives to move and work is complex

The analysis of doctors and nurses’ decision to move and work in a remote and rural areas is inherently complex, involving several variables which could influence their decision.

Program beneficiaries and stakeholders (for example, Provincial and Territorial representatives, Federal-Provincial-Territorial Committee on Health Workforce, Representatives from professional associations, and student financial aid administration) identified some of these variables, including:

Future policy analysis, research and evaluation activities would benefit from exploring how these variables could be further used through data linkage opportunities and/or other data strategies. A richer dataset would enable more refined quantitative analysis, including a potential incremental impact analysis using comparison groups. Nevertheless, while the feasibility of building a valid comparison group (eligible non-beneficiaries) could be further explored, its identification would remain challenging since the benefit is available across the country to all eligible nurses and doctors who have a loan with the Canada Student Financial Assistance program.

Annex E: Additional data

The following tables present additional detail to supplement the data presented in the main body of finding 9.

Table E-1. Distance from work address to closest CMA in kilometers (km): Doctors
Distance to closest CMA (number) AB BC MB NB NL NS NT/NU ON PE QC SK YK National
0 to 49 kilometers 275 839 83 131 55 3 0 1367 0 1 1 0 2755
50 to 99 kilometers 732 463 74 162 13 485 0 946 179 1 162 0 3,217
100 to 249 kilometers 463 647 184 438 142 550 0 676 67 3 626 0 3,796
250 to 499 kilometers 421 159 125 0 244 358 0 375 0 2 102 0 1786
500 to 999 kilometers 47 351 59 0 53 0 69 81 0 1 7 0 668
Over 1000 kilometers 0 4 7 0 4 0 127 0 0 2 0 108 252
Total number of nurses 1938 2463 532 731 511 1396 196 3445 246 10 898 108 12474
Average distance (kilometers) 165 190 241 128 328 156 1452 122 108 469 170 1444 196
Median distance (kilometers) 89 93 207 112 264 108 1470 75 88 290 137 1442 104
Table E-2. Distance from work address to closest CMA in kilometers (km): Nurses
Distance to closest CMA (number) AB BC MB NB NL NS NT/NU ON PE QC SK YK National
0 to 49 kilometers 275 839 83 131 55 3 0 1367 0 1 1 0 2755
50 to 99 kilometers 732 463 74 162 13 485 0 946 179 1 162 0 3,217
100 to 249 kilometers 463 647 184 438 142 550 0 676 67 3 626 0 3,796
250 to 499 kilometers 421 159 125 0 244 358 0 375 0 2 102 0 1786
500 to 999 kilometers 47 351 59 0 53 0 69 81 0 1 7 0 668
Over 1000 kilometers 0 4 7 0 4 0 127 0 0 2 0 108 252
Total number of nurses 1938 2463 532 731 511 1396 196 3445 246 10 898 108 12474
Average distance (kilometers) 165 190 241 128 328 156 1452 122 108 469 170 1444 196
Median distance (kilometers) 89 93 207 112 264 108 1470 75 88 290 137 1442 104

The following figures present additional detail to supplement the data presented in the main body of finding 10.

The figures display the odds ratio estimates of spending one additional year on the benefit for various socio-demographic and loan characteristics of beneficiaries. If the odds ratio is less than one, then the odds of spending an additional year on the benefit are reduced. For odds ratios greater than one, the odds of spending an additional year on the benefit are increased.

Figure E-1. Odds ratio estimates from logistic regression: Doctors
Figure E-1. Odds ratio estimates from logistic regression: Doctors - Text description follow
  • Source: Canada Student Financial Assistance program data.
Figure E-1: Text description
Loan amount
Loan amount Odds ratio estimate
Under $10,000 0.10
$10,000 to under $20,000 (base) 1
$20,000 to under $30,000 1.84
$30,000 to under $40,000 1.84
$40,000 and greater 2.06
Grant amount
Grant amount Odds ratio estimate
Under $5,000 (base) 1
$5,000 to under $10,000 0.881
$10,000 to under $15,000 0.364
$15,000 to under $20,000 0.186
$20,000 and greater 0.482
Sociodemographic and other
Characteristic Base Odds ratio estimate
Married or common law Single 2.011
Had dependant No dependant 1.003
Repayment Assistance Plan No Repayment Assistance Plan 1.639
Distance to nearest Census Metropolitan Area
Distance Odds ratio estimate
0 to 49 kilometers (base) 1
50 to 99 kilometers 1.715
100 to 249 kilometers 1.848
250 to 499 kilometers 2.113
500 to 999 kilometers 3.359
1000 kilometers and farther 5.307
Province
Province Odds ratio estimate
Ontario (base) 1
Alberta 1.401
Manitoba 3.258
Newfoundland and Labrador 5.476
Figure E-2. Odds ratio estimates from logistic regression: Nurses
Figure E-2. Odds ratio estimates from logistic regression: Nurses - Text description follow
  • Source: Canada Student Financial Assistance program data.
Figure E-2: Text description
Loan amount
Loan amount Odds ratio estimate
Under $10,000 (base) 1
$10,000 to under $20,000 3.14
$20,000 to under $30,000 3.89
$30,000 to under $40,000 4.60
$40,000 and greater 4.31
Grant amount
Grant amount Odds ratio estimate
Under $5,000 (base) 1
$5,000 to under $10,000 0.63
$10,000 to under $15,000 0.39
$15,000 to under $20,000 0.35
$20,000 and greater 0.35
Sociodemographic and other
Characteristic Base Odds ratio estimate
Age (25 to 34) Under 25 1.23
Married or common law Single 1.23
Married and spousal income of $40,000 to under $80,000 No spousal income 0.57
Married and spousal income of $80,000 to under $120,000 No spousal income 0.31
Had dependant No dependant 1.01
Parents’ income of $80,000 to under $120,000 No parental income 0.68
Parents’ income of $120,000 and greater No parental income 0.57
Distance to nearest Census Metropolitan Area
Distance Odds ratio estimate
0 to 49 kilometers (base) 1
50 to 99 kilometers 1.20
100 to 249 kilometers 1.23
250 to 499 kilometers 1.30

Page details

Date modified: