Horizontal Evaluation of the Labour Market Development Agreements

Sources of data and information

The analysis takes into account all the quantifiable costs and benefits directly related to EBSM delivery and participation that can be measured given the information available. The analysis is comprehensive in that it accounts for the vast majority of possible direct costs and benefits.

However, the analysis does not account for all costs and benefits. For example, there are factors that can lead to an understatement of the benefits (for example, positive spillovers to other family members) and other factors that can lead to an overstatement of the benefits (for example, effects on skill prices or displacement).

This study relied on integrated data from the EI Part I and II Databank and Income Tax records from the Canada Revenue Agency. Information about earnings, use of EI, and use of social assistance was taken from the study of incremental impacts.^{Footnote 22} The program costs were calculated using information available in the EI Monitoring and Assessment Reports.

Relative to the previous cycle of evaluation, the methodology has been extended to incorporate one of the indirect health benefits associated with increased labour market attachment. In particular, the methodology includes an estimate of the change in public health care cost due to the decline in health care utilization resulting from program participation.

Data on average public healthcare costs by income quintiles are taken from the report Lifetime Distributional Effects of Publicly Financed Health Care in Canada (2013) by the Canadian Institute for Health Information.

Program costs are measured using information on LMDA expenditures and new interventions reported in the EI Monitoring and Assessment Reports. Other costs and benefits are assessed using integrated administrative data from the EI Part I and II databank and the Canada Revenue Agency.

Incremental impacts measured over the second year of participation and up to 5 post-program years are discounted by 3% to bring them to a common base with the program cost and benefits incurred in the program start year. This 3% rate accounts for the interest the government could have collected if the funds used to pay for the program had been invested. Incremental impacts are estimated using 2010 constant dollars and this accounts for inflation.

The costs and benefits accounted for in the calculations

Program costs: costs incurred by the government for delivering the program (that is, administration and direct program costs calculated from data reported in the EI Monitoring and Assessment Reports).

Marginal social cost of public funds: loss incurred by society when raising additional revenues such as taxes to fund government spending. The value is estimated at 20% of the program cost, sales taxes, income taxes, impacts on EI and impacts on SA paid or collected by the government.

Foregone earnings: estimated net impacts on participants’ earnings during the participation period. During labour market program participation, some individuals have lower earnings than what they would have received if they had not participated.

Employment earnings: incremental impacts on participants’ earnings during and after participation. In-program earnings represent the foregone earnings for participants.

Fringe benefits: the employer-paid health and life insurance as well as pension contributions. They are estimated at 15% of the incremental impacts on earnings.

Federal and provincial income taxes: incremental impacts on federal, provincial and territorial taxes paid by participants.

Sales taxes: the sales taxes paid by participants estimated as incremental impacts on earnings multiplied by the propensity to consume (97%), the proportion of household spending on taxable goods and services (52%) and the total average federal and provincial sales tax rate (11%).

Social assistance and Employment Insurance benefits collected: incremental impacts on SA and EI benefits use by participants following participation.

Canada Pension Plan and Quebec Pension Plan contribution and EI premiums: these contributions and premiums were identified from the Canada Revenue Agency data and then, the incremental impacts on Canada Pension Plan and Quebec Pension Plan contributions and EI premiums were estimated.

Public health care costs: estimated impact of participation in EBSMs on public health care costs shown as an average change per participant over the post-program period examined.

Incidence of employment

Chart 1 presents the incremental impacts on the incidence of employment^{Footnote 23} for active and former claimants by type of program. The estimates can be interpreted as a change in the probability of being employed following participation.

Active claimants in SD, TWS and JCP increase their incidence of employment relative to similar non-participants. Former claimants in SD, TWS and JCP increase their incidence of employment relative to similar participants who receive only EAS.

Active claimants in EAS-only experience a small positive, but not statistically significant, impact on their incidence of employment. EAS are relatively modest activities such as counselling, job search assistance and case management, which focus on quicker returns to work for participants. EAS supports that are not provided with longer interventions, are not expected to increase participants’ skills or influence their employment levels to a large extent.

*The impact is non-statistically significant, however, it is still valid in terms of informing the direction of the impact (negative or positive).

Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Employment earnings

Chart 2 presents the average annual increase in employment earnings for active and former EI claimants over the 4 years post-participation. Active EI claimants in SD, TWS, JCP, and EAS increase their employment earnings compared to similar non-participants.

Former EI claimants in SD and TWS increase their employment earnings relative to similar participants who receive only EAS services. Former claimants in JCP have lower employment earnings compared to similar participants in only EAS services.

Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Use of EI benefits

As shown in Chart 3, active claimants reduce their use of EI benefits in the post-program period compared to similar non-participants. In the post-program period, former claimants in SD, TWS and JCP increase their EI benefits use relative to similar participants who receive EAS services only. 

In the case of SD and TWS, the increase in the use of EI by former claimants is consistent with previous evaluations and is not necessarily a negative impact. Following participation, former claimants are likely to requalify for EI benefits due to their stronger labour market attachment demonstrated by increases in their incidence of employment and earnings.

The increase in the use of EI for former claimants in JCP is a negative impact given the decrease in employment earnings. Therefore, for former EI claimants, this intervention will not be effective from a cost-benefit perspective. However, the supplemental study for JCP confirmed that the program has added value for participants, employers, and communities that cannot be taken into account.

Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Use of social assistance (SA) benefits

As shown in Chart 4, most active and former EI claimant participants decrease their use of SA benefits in the post-program period.

Active EI claimants in SD and TWS decrease their use of SA benefits in the post-program period compared to similar non-participants. Active EI claimants in EAS services experience a small increase in the use of SA benefits compared to similar non-participants.^{Footnote 24}

Former EI claimants in SD, TWS and JCP decrease their use of SA benefits compared to similar participants in only EAS services.

*The impact is non-statistically significant, however, it is still valid in terms of informing the direction of the impact (negative or positive).

Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Dependence on income support

As shown in Chart 5, overall active and former claimants reduce their dependence on government income supports.

*The impact is non-statistically significant, however, it is still valid in terms of informing the direction of the impact (negative or positive).

Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Female participants

Nearly 300,350 EI active and former claimant participants in LMDA programs and services, between 2010 and 2012 are female, representing nearly 49% of participants.

The profile of female participants is presented in Table 5 by age, sociodemographic group, and marital status. Information about their educational attainment, occupation and industry is based on the latest job they held prior to applying for EI benefits. Information about sociodemographic groups is self-reported.

*Values may not equal 100% due to rounding or missing information.

Top 3 occupational groups for female participants

Active claimants:

• clerical personnel = 20%
• intermediate sales and service personnel = 17%
• other sales and service professionals = 11%

Former claimants:

• intermediate sales and service personnel = 20%
• clerical personnel = 18%
• other sales and service professionals = 15%

Top 3 industries for female participants

Active claimants:

• retail trade = 13%
• manufacturing = 11%
• healthcare and SA = 10%

Former claimants:

• retail trade = 15%
• accommodation and food services = 12%
• healthcare and SA = 10%

Main findings: Female participants improve their labour market attachment through increases in their incidence of employment and earnings. Excluding JCP, participants also decrease their dependence on government income supports. While female former claimants increase their use of EI benefits irrespective of the program received, only JCP participants saw an increase in their dependence on government income supports.

Table 6 presents the detailed incremental impacts.

For example, the results reveal that relative to similar non-participants female active claimants in SD have higher annual earnings (+$1,964 per year) and increase their incidence of employment (+4.7 percentage points). They also depend less on government income supports (-2.4 percentage points), by decreasing their use of EI (-$318 per year) and SA (-$75 per year) benefits.

Female former claimants in TWS increase their annual earnings (+$1,478 per year) and incidence of employment (+5.6 percentage points) compared to similar participants in EAS only. They also lower their reliance on government income supports (-2.3 percentage points), mostly by decreasing their use of SA benefits (-$344).

Statistical significance level *** 1%; ** 5%; * 10%, other values are non-statistically significant.

Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Male participants

Nearly 308,840 EI active and former claimant participants in LMDA programs and services between 2010 and 2012 are male, representing about 51% of participants.

The profile of male participants is presented in Table 7 by age, sociodemographic group, and marital status. Information about their educational attainment, occupation and industry is based on the latest job they held prior to applying for EI benefits. Information about sociodemographic groups is self-reported.

*Values may not equal 100% due to rounding or missing information.

Top 3 occupational groups for male participants

Active claimants:

• semi-skilled manual workers = 21%
• other manual workers = 19%
• skilled crafts and trades = 15%

Former claimants:

• semi-skilled manual workers = 21%
• other manual workers = 21%
• skilled crafts and trades = 15%

Top 3 industries for male participants

Active claimants:

• manufacturing = 18%
• construction = 16%
• administrative and support, waste management and remediation services = 10%

Former claimants:

• manufacturing = 19%
• construction = 16%
• administrative and support, waste management and remediation services = 11%

Main findings: Male participants in SD and TWS improve their labour market attachment through increases in their incidence of employment and employment earnings. They also decrease their dependence on government income support, that is, their combined use of EI and SA benefits. Mixed and non-statistically significant results are found for participants in JCP and EAS.

Table 8 presents the detailed incremental impacts.

For example, the results reveal that relative to similar participants in EAS only, male former claimants in SD have higher annual earnings (+$865 per year) and incidence of employment (+2.5 percentage points). They also depend less on government income supports (-0.9 percentage points), mostly by decreasing their use of SA benefits (-$128).

Male active claimants in TWS have higher annual earnings (+$1,657 per year) and incidence of employment (+3.9 percentage points) compared to similar non-participants. They also lower their income support reliance rate (-2.4 percentage points), by decreasing their use of EI benefits (-$379 per year).

Statistical significance level *** 1%; ** 5%; * 10%, other values are non-statistically significant.

Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Youth participants

Nearly 172,730 EI active and former claimant participants, between 2010 and 2012, were 30 years of age or younger when they began their program, representing about 28% of participants.

The profile of youth participants is presented in Table 9 by gender, sociodemographic group, and marital status. Information about their educational attainment, occupation and industry are based on the latest job they held prior to applying for EI benefits. Information about sociodemographic groups is self-reported.

*Values may not equal 100% due to rounding or missing information.

Top 3 occupational groups for youth participants

Active claimants:

• other manual workers = 18%
• clerical personnel = 12%
• semi-skilled manual workers = 12%

Former claimants:

• other manual workers = 18%
• other sales and service professionals = 15%
• intermediate sales and service personnel = 14%

Top 3 industries for youth participants

Active claimants:

• construction = 14%
• retail trade = 13%
• manufacturing = 12%

Former claimants:

• retail trade = 15%
• manufacturing = 12%
• accommodation and food services = 12%

Main findings: Youth in SD and TWS improve their labour market attachment through increases in their incidence of employment and employment earnings. They also decrease their dependence on government income support (that is, the combined use of EI and SA benefits). Mixed and non-statistically significant results are found for participants in JCP and EAS.

Table 10 presents the detailed incremental impacts.

For example, the results reveal that relative to similar non-participants, youth active claimants in SD have higher annual earnings (+$1,887 per year) and incidence of employment (+1.9 percentage points). They also depend less on government income supports (-2.4 percentage points), by decreasing their use of EI (-$349 per year) and SA (-$150 per year) benefits.

Youth former claimants in TWS have higher annual earnings (+$2,122 per year) and incidence of employment (+5.1 percentage points) compared to similar participants in EAS only. They also lower their reliance on government income supports (- 3.3 percentage points), by decreasing their use of SA (- $409).

Statistical significance level *** 1%; ** 5%; * 10%, other values are non-statistically significant.

Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Older worker participants

Nearly 79,500 EI active and former claimant participants, between 2010 and 2012, were 55 years of age or older when they began their program, representing about 13% of participants.

The profile of older worker participants is presented in Table 11 by gender, sociodemographic group, and marital status. Information about their educational attainment, occupation and industry are based on the latest job they held prior to applying for EI benefits. Information about sociodemographic groups is self-reported.

*Values may not equal 100% due to rounding or missing information.

Top 3 occupational groups for older worker participants

Active claimants:

• semi-skilled manual workers = 16%
• clerical personnel = 14%
• intermediate sales and service personnel = 11%

Former claimants:

• semi-skilled manual workers = 16%
• other sales and service professionals = 12%
• clerical personnel = 12%

Top 3 Industries for older worker participants

Active claimants:

• manufacturing = 16%
• retail trade = 12%
• administrative and support, waste management and remediation services = 9%

Former claimants:

• manufacturing = 16%
• retail trade = 11%
• administrative and support, waste management and remediation services = 11%

Main findings: Incremental impacts reveal that older workers in SD and TWS improve their labour market attachment. Older workers who are active claimants and participate in JCP also improve their labour market attachment. However, only former EI claimants in TWS also decrease their dependence on government support following participation. 

Table 12 presents the detailed incremental impacts.

For example, the results reveal that compared to similar participants in EAS only , older worker former claimants in TWS have higher annual earnings (+$2,254) and incidence of employment (+8.9 percentage points). They also depend less on government income supports (-3.3 percentage points), mostly by decreasing their use of SA benefits (-$278).

Older worker active claimants in JCP have higher annual earnings (+$1,241) and incidence of employment (+8.3 percentage points) compared to similar non-participants. However, the increase in EI benefits (+$466) indicates that participants are unable to maintain the employment secured following the intervention.

Statistical significance level *** 1%; ** 5%; * 10%, other values are non-statistically significant.

Note: Impacts are estimated over 4 post-program years (or 5 years in the case of Employment Assistance Services).

Indigenous participants

Nearly 36,400 EI active and former claimant participants, between 2010 and 2012, self-identify as being Indigenous Canadians, representing about 6% of participants.  

The profile of Indigenous participants is presented in Table 13 by gender, age and marital status. Information about educational attainment, occupation and industry are based on the latest job held prior to applying for EI benefits. The profile of Indigenous participants is generally comparable to the one observed for LMDA participants at large across Canada.

*Values may not equal 100% due to rounding or missing information.

Top 3 occupational groups for Indigenous participants

Active claimants:

• other manual workers = 18%
• semi-skilled manual workers = 15%
• intermediate sales and service personnel =11%

Former claimants:

• other manual workers = 20%
• semi-skilled manual workers = 13%
• other sales and service professionals = 13%

Top 3 industries for Indigenous participants

Active claimants:

• construction = 15%
• public administration = 14%
• accommodation and food services = 8%

Former claimants:

• construction = 15%
• public administration = 14%
• accommodation and food services = 10%

Main findings: Indigenous participants in SD, TWS and EAS improve their labour market attachment through increases in their incidence of employment and employment earnings. They also decrease their dependence on government income support (that is, the combined use of EI and SA benefits). The findings are consistent with findings from the Indigenous Skills and Employment Training Strategy evaluation.

Table 14 presents the detailed incremental impacts.

For example, the results reveal that relative to similar non-participants, Indigenous active claimants in SD have higher annual earnings (+$5,759 per year) and incidence of employment (+7.4 percentage points). They also depend less on government income supports (-3.4 percentage points), by decreasing their use of SA benefits (-$182 per year).

Indigenous former claimants in TWS have higher annual earnings (+ $2,312 per year) and incidence of employment (+ 6.7 percentage points) compared to similar participants in EAS only. They also lower their reliance on government income supports (-4 percentage points), mostly by decreasing their use of SA benefits (-$471 per year).

Statistical significance level *** 1%; ** 5%; * 10%, other values are non-statistically significant.

Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Persons with disabilities

Nearly 38,000 EI active and former claimant participants, between 2010 and 2012, self-identify as persons with disabilities, representing about 6% of participants.

The profile of persons with disability participants is presented in Table 15 by gender, age, and marital status. Information about their educational attainment, occupation and industry is based on the latest job they held prior to applying for EI benefits.

Recent interviews with 40 service providers working with persons with disabilities reveal that a significant proportion of participants are looking or only capable of working on part-time basis. In fact, the outcome trends reveal that at least 40% of persons with disabilities earn less than $10,000 per year in the 4 post-program years.

*Values may not equal 100% due to rounding or missing information.

Top 3 occupational groups for persons with disabilities

Active claimants:

• other sales and service professionals = 14%
• semi-skilled manual workers = 14%
• clerical personnel = 14%

Former claimants:

• other sales and service professionals = 16%
• other manual workers = 14%
• semi-skilled manual workers = 13%

Top 3 industries for persons with disabilities

Active claimants:

• retail trade = 13%
• manufacturing = 12%
• administrative and support, waste management and remediation services = 11%

Former claimants:

• retail trade = 13%
• manufacturing = 12%
• administrative and support, waste management and remediation services = 12%

Main findings: Persons with disabilities, who are active claimants in SD and TWS, improve their incidence of employment and decrease their dependence on income support. Participants have small positive but non-statistically significant impacts on employment earnings. Former EI claimants in SD and TWS improve their labour market attachment and decrease their overall level of dependence on income supports.

Table 16 presents the detailed incremental impacts.

For example, the results reveal that relative to similar participants in EAS only, persons with disabilities who are former claimants in SD have higher annual earnings (+$2,087) and incidence of employment (+6.3 percentage points). They also lower their reliance on government income supports (-1.7 percentage points) mostly by decreasing their receipt of SA benefits (-$436).

Persons with disabilities who are active claimants in TWS have higher incidence of employment (+8.1 percentage points) compared to similar non-participants. They also lower their reliance on government income supports (-8.1 percentage points) mostly by decreasing their use of SA benefits (-$641).

Statistical significance level *** 1%; ** 5%; * 10%, other values are non-statistically significant.

Note: Impacts are estimated over 4 post-program years.

Recent immigrants^{Footnote 31}

Nearly 24,500 EI active and former claimant participants in LMDA programs and services, between 2010 and 2012, are recent immigrants, representing about 4% of participants.

The profile of recent immigrant participants is presented in Table 17 by gender, age and marital status. Information about their educational attainment, occupation and industry is based on the latest job they held prior to applying for EI benefits.

*Values may not equal 100% due to rounding or missing information.

Top 3 occupational groups for recent immigrants

Active claimants:

• semi-skilled manual workers = 14%
• other manual workers = 14%
• clerical personnel = 12%

Former claimants:

• other sales and service professionals = 17%
• other manual workers = 15%
• semi-skilled manual workers = 14%

Top 3 industries for recent immigrants

Active claimants:

• manufacturing = 18%
• administrative and support, waste management and remediation services = 17%
• retail trade = 8%

Former claimants:

• administrative and support, waste management and remediation services = 19%
• manufacturing = 17%
• retail trade = 10%

Main findings: Recent immigrants who are active EI claimants in SD, TWS and EAS improve their labour market attachment. Participants in SD and EAS also reduce their use of EI benefits. The incremental impacts for recent immigrant former EI claimants are non-statistically significant.

Table 18 presents the detailed incremental impacts. For example, the results reveal that relative to similar non-participants, recent immigrants who are active claimants in SD have higher annual earnings (+$2,942) and incidence of employment (+6.7 percentage points). They also depend less on government income supports (-1 percentage point), mostly by decreasing their use of EI benefits (-$324 per year).

Recent immigrants who are active claimants in EAS have higher annual earnings (+$2,120) and incidence of employment (+4.7 percentage points) relative to similar non-participants.  Active claimants also lower their use of EI benefits (-$251 per year).

Statistical significance level *** 1%; ** 5%; * 10%, other values are non-statistically significant.

Note: Impacts are estimated over 4 post-program years (or 5 years in the case of Employment Assistance Services).

Visible minorities

Nearly 29,000 EI active and former claimant participants in LMDA programs and services, between 2010 and 2012, self-identify as visible minorities, representing about 5% of participants.

The profile of visible minority participants is presented in Table 19 by gender, age, and marital status. Information about their educational attainment, occupation and industry are based on the latest job they held prior to applying for EI benefits.

*Values may not equal 100% due to rounding or missing information.

Top 3 occupational groups for visible minorities

Active claimants:

• semi-skilled manual workers = 16%
• clerical personnel = 14%
• other sales and service professionals = 12%

Former claimants:

• semi-skilled manual workers = 15%
• other sales and service professionals = 14%
• clerical personnel = 14%

Top 3 industries for visible minorities

Active claimants:

• manufacturing = 18%
• administrative and support, waste management and remediation services = 12%
• retail trade = 9%

Former claimants:

• manufacturing = 19%
• administrative and support, waste management and remediation services = 14%
• retail trade = 9%

Main findings: Visible minorities who are active EI claimants in SD and TWS improve their labour market attachment. Only those who participate in SD, are also able to decrease their reliance on government income supports. Visible minority participants who are former EI claimants in TWS improve their labour market attachment. Former participants in SD do not benefit from the program.

Table 20 presents the detailed incremental impacts.

For example, the results found that relative to similar non-participants, visible minorities who are active claimants in SD, have higher annual earnings (+$1,605) and incidence of employment (+8.9 percentage points). They also depend less on government income supports (- 3.3 percentage points).

Former claimants in TWS have higher annual earnings (+ $2,775) and incidence of employment (+ 5.0 percentage points) compared to similar participants in EAS only.

Statistical significance level *** 1%; ** 5%; * 10%, other values are non-statistically significant.

Note: Impacts are estimated over 4 post-program years (or 5 years in the case of Employment Assistance Services).

Heterogeneity of treatment effect study

A supplemental study, the heterogeneity of treatment effects,^{Footnote 34} examines alternative methods to traditional incremental impact analysis. This new method uses newly developed causal machine learning methods to examine whether the effectiveness of EBSMs varies across participants’ sociodemographic characteristics. ^{Footnote 35},^{Footnote 36}

This information may help to inform program and policy making decisions in the future. The findings for the Heterogeneity of Treatment Effects study for SD active participants can be found in Appendix A.

Skills Development^{Footnote 38}

During the 2010 to 2012 period, SD represents almost 52% of EBSM expenditures under the LMDAs across Canada. The average duration of an SD Action Plan Equivalent is 49 weeks for active claimants and 48 weeks for former claimants. As shown in Table 21, over the 10 year post-program period the benefit for active claimants is +$5,508 higher than the costs, yielding a social return of 54% on investment. This means that if the government spends $1 on SD for active EI claimants, it generates +$1.54 of benefit for society. It takes 8.3 years for the benefits to recover the costs of programing. Overall, there are savings to health care costs of $178 per participant.

Targeted Wage Subsidies

During the 2010 to 2012 period, TWS represents 3% of total EBSM expenditures across Canada. The average duration of a TWS Action Plan Equivalent is 38 weeks for active claimants and 31 weeks for former claimants. As shown in Table 21, over the 5 year post-program period the benefit for former claimants is +$9,052 higher than the costs, yielding a social return on investment of 123%. A savings to health care costs of $169 is found per participant 5 years after the program. The costs are recovered during the first year following participation (0.7 year).

Of all EBSMs, TWS has the most positive results for former claimants.

Job Creation Partnerships^{Footnote 39}

In addition to helping participants obtain work experience in the funded projects, JCP also benefits the communities where the projects are implemented. However, community impacts are not accounted for in this cost-benefit analysis, since they are not captured in the administrative data and are difficult to quantify.

In this context, the benefits from the social perspective are likely underestimated. Overall, JCP represented 2% of the total EBSM investment in the 2010 to 2012 period. The average duration of a JCP Action Plan Equivalent is 33 weeks for active claimants, and 30 weeks for former claimants.

As shown in Table 21, over the 5 year post-program period the benefits of delivering JCP to active claimants are -$13,902 lower than the costs 5 years after participation, yielding a return on investment of -118%. It takes 16.1 years for the benefits to recuperate the cost. A savings of $52 in health care costs is found per participant following participation.

Employment Assistance Services^{Footnote 40}

EAS includes a variety of services such as computer access for job search services, group sessions to prepare for an interview, career counselling, and action plan development. The administrative data, however, do not allow to identify what proportion of EAS interventions belong to each category or the intensity of services offered to participants.

While EAS are often provided with other EBSMs, this analysis examined only participants who received one or more EAS without participating in other EBSMs. EAS represents about 30% of total EBSM expenditures between 2010 and 2012. The average length of an EAS-only Action Plan Equivalent is 12 weeks compared to between 33 to 49 weeks for active EI claimant participants in other EBSMs.

As shown in Table 21, over the 5 year post-program period the benefits for active claimants in EAS is -$2,395 lower than the costs, yielding a social return on investment of -290%. It takes 7.7 years after participation for the benefits to recover the costs. No savings to health care costs is found.

Overall, the goal of EAS is not to help participants acquire more skills, therefore, increasing participants’ earnings after participation is not necessarily expected. Conducting a cost-benefit analysis for EAS is a challenge as it is not possible to attribute a dollar figure to the return to employment. However, including earnings in the cost-benefit calculation is still very relevant since it captures partially the positive impact of the quicker return to work. 

Program design and delivery

The following is a summary of guidelines from provinces and territories that were delivering the self-employment programming in fall 2018. Key informants did confirm and complement the information found in program guidelines.

The Self-Employment program aims to assist participants in creating employment for themselves by providing them with a range of services including: 

• assistance with business plan development
• counselling, coaching and mentoring
• entrepreneurial training and workshops

In addition to being unemployed and EI-eligible, participants must not already own and/or operate their businesses prior to program participation.

Provinces and territories have the flexibility to design and deliver the program to meet their labour market needs. In fall 2018, the program was delivered mainly through third-party organizations, including:

• business development corporations
• Community Futures (used in rural areas)
• community organizations serving specialized groups (such as, women and francophone)
• private management or consulting firms

Program officials report that the amount allocated to the Self-Employment program depends on regional allocations, demand for the program and local labour market conditions. Three provinces and territories have decided not to deliver the program because other entrepreneurial programs exist, high costs relative to the demonstrated results, or low demand and labour market conditions.

The application process aims to ensure that participants are suited for self-employment, have a viable business idea and the financial resources to launch a business.

Participants’ employment outcomes

The following is a summary of labour market outcomes and satisfaction rates from a survey of self-employment participants across 9 provinces and territories completed in winter 2020.^{Footnote 42} A total of 2,199 individuals responded to the survey with a 40.5% response rate.

Self-employment participants increased their employment level by 15 percentage points from 59% in the year before participating to 74% at the time of survey. That is, 2 to 4 years after program participation. The increase is mainly due to an increase in the percentage of self-employed participants.

Type of businesses created, survival rates and success factors

Nearly 50% of survey respondents launched a self-employment business and it was still in operation in winter 2020 (2 to 4 years following program participation).

Among the 1,365 respondents who started a business, 68% of them were still operating their business at 2 to 4 years post-program. Another 4% sold their business, but it was still operational. Twenty-four percent (24%) of respondents were unable to maintain the operation of the business they started as part of the program.

The business survival rate is consistent with a 2018 Statistics Canada study^{Footnote 43} that found that less than half of unincorporated self-employed individuals continued operations for more than 2 years.

Half of self-employment businesses were launched in other services^{Footnote 44}; professional, scientific and technical services; as well as in construction and retail trade.

Regarding factors influencing the success or failure of self-employment businesses:

• participants who started a business and were still in operation at the time of survey attributed their business success to:
  • their dedication, hard work and positive attitude
  • the high demand for their services or products
  • the quality of service provided
  • their own abilities, experience, knowledge and skills
  • their network and business contacts
• participants who started a business but were forced to close it attributed the closure to:
  • poor sales and low revenues
  • small market
  • workload
  • finding another job
• participants who did not launch a business attributed this to:
  • the lack of funding, and the level of uncertainty and risk involved
  • workload, work life balance, and underestimating the required commitment

Earning outcomes and reliance on income support

Survey respondents were not comfortable answering questions that related to their earnings. This situation made it difficult to compare the pre- and post-earnings of self-employment participants.

Overall, there appears to be an increase in the number of participants reporting less than $10,000 in earnings annually. However, survey respondents, who are able to maintain the operation of their business, are more likely to report earning more or the same as before participating in the program.

As a complement to the earning questions, survey respondents did assess their financial well-being. When considering their entire financial situation:

• 73% of respondents said that they are financially about the same or better off after the program
• 70% of respondents said that their household net worth is about the same or higher after the program

In line with survey findings, 14 provincial/territorial program managers state that immediate increases in earnings are not necessarily an expected outcome of the program. 

Regarding the reliance on government income support, participants reduce reliance on the use of EI and SA following program participation.

Satisfaction with services received and current employment

A high percentage of respondents who started a self-employment business report that they are equally or more satisfied with their job situation after program participation. Those who are able to maintain the operation of their business are 29 percentage points more likely to report being more satisfied, compared to those whose business closed (76% compared to 47%).

The survey did examine the contribution of the program to the success of self-employment businesses. At least 81% of survey respondents who launched a self-employment business rate the following services and training as very or somewhat important to the business launch, operation and success:

• assistance with business plan development
• one-on-one mentoring / advice or counselling supports
• discussion on risks and challenges of self-employment
• assessment of entrepreneurial readiness
• living allowance during participation and financial assistance with business start-up costs
• information about and assistance to access capital
• training on budgeting, financial management, marketing, business operation and sales

Challenges and lessons learned related to program design and delivery

The following is based on key informant interviews with program managers, and front-line third-party service providers, and case workers.

Key informants identify the following challenges related to program design and delivery, including:

• the lack of clear communication between service providers and the provincial or territorial department and/or Service Canada regarding the confirmation of eligibility for Employment Insurance
• restrictive contract provisions for service providers
• complex assessment process of candidates
• difficulty to serve remote and rural areas

Best practices related to program design and delivery included:

• using local and specialized organizations to deliver the program
• using standardized tools for business plan development
• relying on local expertise to assess business viability
• providing ongoing mentorship, advice and counselling
• using specialists to deliver tailored training
• providing participants with opportunity to network and facilitating their knowledge of and access to funding

Key considerations for Self-Employment program and policy development

The following considerations emerged as part of the study.

The Self-Employment program can benefit from an updated objective specifying that it is dedicated to eligible participants who have a viable business idea, the financial or in-kind resources to launch a business, and the required level of dedication.

The data collection process should include only participants who have been deemed suitable for self-employment and accepted into the program. This will require excluding candidates who attended information sessions alone or those deemed not suited for self-employment. The latter participants can be reported under Employment Assistance Services.

Indicators of program success can include: increase in employment and/or self-employment levels; medium-term increase in earnings; business survival rate similar to the local economy and/or the sector; and acquisition of transferable skills.

Provinces and territories may wish to consult with their service delivery network on the extent to which identified challenges are applicable to their unique context, and how best to address them along with integrating lessons learned that can benefit program delivery.

Rationale

The Self-Employment program aims to assist participants in creating employment for themselves. The participant’s application process is structured and aimed to ensure that they are suited for self-employment, have a viable business idea, and the financial resources to launch a business. However, the survey revealed that:

• one third of participants did not launch a business
• 176 survey respondents confirmed that they did not participate in the program
• nearly one quarter of participants were unable to maintain the operation of the business they had started as part of the program

Participants who started a business under the Self-Employment program and were still in operation at the time of the survey attributed their business success to: their dedication, hard work and positive attitude; the high demand for their services or products; the quality of service provided; their own abilities, experience, knowledge and skills; their network; and, business contacts. Those who started a business but were forced to close it attributed the closure to: poor sales and low revenues; small market; workload; and, finding another job. Finally, participants who did not launch a business during program participation attributed this to: the lack of funding; the level of uncertainty and risk involved; workload, work-life balance; and, underestimating the required commitment.

The survey confirmed that participants acquire transferable skills through training and workshops, they experience increase in employment and medium-term earnings, and they create additional jobs. As well, business survival rates mirror those observed for small business in the economy. These indicators are useful in measuring and reporting program success as well as managing contribution agreements with service providers.

Program description

The following is a summary of guidelines from 8 provinces and territories that were delivering JCP in the years 2018 and 2019. Key informants did confirm and complement the information found in program guidelines.

Program objective

The objective of JCP is to provide participants with the opportunity to gain work experience or training that improves their employment prospects.

Provinces and territories use 2 models to deliver the program:

• the first, and most common, is the provision of funds to an organization that implements a community-benefiting project while providing work experience to participants
• the second is the provision of classroom-based training in combination with work experience for participants through third-party training providers

Participants receive benefits from EI Part I or II.^{Footnote 46} Benefits to participants follow the prevailing wage rate, up to the maximum EI weekly benefit rate. Provinces and territories may provide additional funding to project holders or training providers to cover the project or training costs.

Program delivery

The design and delivery of JCP allow provinces and territories to address a variety of barriers to employment experienced by their citizens (such as, lack of work experience). Provinces and territories can use the program to address various labour market needs by targeting sub-groups of individuals, professions or economic sectors in demand and communities.

Program managers report that the amount allocated to the program depends on government priorities, demand for the program, previous funding levels and labour market needs. Five provinces and territories did not deliver the program in 2018 and 2019 because of budget priorities, lack of interest by organizations and job seekers, and the temporary nature of jobs offered under the program.

In addition to gaining work experience, key informants expect participants to develop work-related skills and to enhance their career development, job search abilities, and to improve their personal well-being. Project holders can benefit from the program through increased capacity, implementing their projects, and increasing their presence within local communities.

For employers that provide work experience to trained participants, benefits are mostly associated with gaining a source of trained employees. At the community level, projects support the local economy and provide new assets (such as, restored buildings or hiking trails) or services (such as, support for newcomers).

Challenges and lessons learned

Key informants identify challenges related to:

• the recruitment of participants and organizations
• ensuring that funded projects are assisting participants in addressing their barriers to employment
• program administration and monitoring

Key informants identify the following elements as contributing factors to participants’ success:

• implementation is effective when contract coordinators provide support to project holders during the writing of their program proposal and throughout project implementation
• on-site visits, open communication, and establishing clear project expectations support effective monitoring
• for training providers, participant recruitment can improve by involving employers in the recruitment process and in curriculum development
• classroom-based training is most effective when it is occupation or sector-specific and aligned with current or forecasted labour demand
• participants who benefit the most from the program face one or multiple barriers to employment such as work experience, networking opportunities, skills training, or assistance with reintegrating into the labour force
• experienced organizations who used the program previously can implement effective projects, as they have the capacity to manage funding and to provide support to participants
• projects found to be best suited for funding are those who align with community needs, government priorities and labour market issues. Examples of well-suited project sectors and activities include event coordination, arts and culture, and construction

Key considerations for JCP program and policy development

The following considerations emerged as part of the JCP study.

Consideration #1: JCP could benefit from an updated logic model and narrative explaining the program theory, from the development of detailed Terms and Conditions, or simply from an update to the “Process for Determination of Similarity”^{Footnote 47} document, in consultation with provinces and territories.

The objective of JCP is to enable unemployed persons who qualify as “insured participants” under section 58 of the EI Act to obtain employment by providing them with employment opportunities through which they can gain work experience to improve their long-term employment prospects.

The qualitative and quantitative evidence, documented in this study, demonstrate that JCP is generally meeting its objectives by assisting participants to improve their labour market attachment following their program participation. As well, JCP has benefits to organizations, employers and local communities.

However, the current program description is not comprehensive enough to cover all the programs similar to JCP that are designed and implemented by provinces and territories. For example, it is not clear to what extent the combination of training and work experience or placements provided in 3 provinces and territories are aligned with the description of JCP in the document entitled “Process for Determination of Similarity”. As is, the incremental impacts for these programs should be reported separately from JCP and SD. One province has recently reclassified the program as SD instead of JCP in its annual plan. This may require creating a new program category (for example, Training to Work or Training and Work Experience) or expanding the definition of JCP.

Moreover, the “Process for Determination of Similarity” document does not address the issue of repeat participation for participants and organizations, though there may be advantages to this approach. Evidence suggests that some participants can benefit from more than one JCP participation, or simply from a participation of a longer duration. Furthermore, evidence from key informant interviews suggests that established organizations who used JCP in the past are well suited to implement JCP projects.

Consideration #2: Compared to other Employment Benefits offered under the LMDAs, JCP is a less attractive program and is the least used. The program could benefit from increased promotion to organizations and more importantly from reducing its administrative burden.

On average, JCP accounted for 1.5% of total LMDA expenditure between 2012 to 2013 and 2016 to 2017. Only 2 provinces spent close to 7% of their LMDA expenditure on JCP in 2016 to 2017. Furthermore, the program is not used in 5 provinces and territories because of budget priorities, lack of interest by organizations and job seekers, and the temporary nature of jobs.

Awareness and interest in the program was a challenge, despite its positive outcomes. Key informants reported the lack of promotion to potential organizations and identified the lack of interest by organizations and job seekers as challenges. Recruitment of participants may be affected by the low level of financial support available to them. The availability of skilled participants can support the recruitment of low-capacity organizations.

Key informant interviews revealed the complex nature of JCP administration for both organizations and government staff. The administration of the program includes activities such as submitting a project proposal, producing monthly reports, advertisement and recruitment, confirmation of EI eligibility, and monitoring participants and project holders. In the current design of JCP, key informants identified delivery challenges related to the burden of program administration and monitoring for both contract coordinators and small organizations.

Funded organizations

Funded organizations include non-profits such as:

• industry associations, sector councils, employer associations and businesses/employers
• Indigenous organizations, educational institutions and training providers (private and non-profit)
• municipal and local governments including Indigenous governments

Targeted labour market issues

Funded projects target current and/or forecasted skills and/or labour shortages. These projects also target specific unemployed populations (for example, women, youth, Indigenous peoples, newcomers, persons with disabilities and the self-employed).

Generally, projects target labour market issues associated with:

• lack of capacity for human resource planning resulting in employee attraction and retention challenges
• projected or actual growth of industry/business
• aging workforce
• businesses downsizing/closure
• limited employment opportunities in Indigenous, small and remote communities
• barriers to employment experienced by a target population

The majority of projects reviewed align with their respective provincial or territorial program objectives and eligible activities.

Partnerships

All participating provinces and territories confirm that program officials carried out activities to support the formation and maintenance of partnerships. Provincial and territorial departments and key informants explained that partners’ expertise, network and financial contribution are all essential to project implementation and success.

The document review of 117 projects confirm that:

• partnerships were established to support the delivery of the majority of projects
• partners made a financial or in-kind contribution:
  •  the most common forms of in-kind contribution were expertise, staff time to project administration and delivery, office/event space and equipment
• project activities delivered with the support of partners include:
  • labour market and human resource research
  • training development and/or delivery
  • career/job awareness, workforce strategy/plan development, and engagement
  • awareness of labour market information and human resource tools development

Challenges and lessons learned

Provinces and territories, and key informants identified challenges related to project holder recruitment and setting up projects (application, identifying partners). Additional challenges include program administration and monitoring.

Actions of program officials and project characteristics that are conducive to the success of the program include:

• ongoing communication and relationship building between provincial and territorial government officials and project holders
• clearly defined labour market issues/needs that the projects aimed to address. These generally occur through clear project objectives, activities, expected outputs and outcomes, and performance measurement requirements
• ensuring that project holders have the capacity to deliver the project
• strong partnerships between project holders, government officials, and stakeholders (community, sector) resulting in valuable contributions to the project delivery

Key considerations for Labour Market Partnership program and policy development

The following considerations emerged as part of the Labour Market Partnerships study.

Considering that the current performance indicators do not reflect the diversity of activities funded under Labour Market Partnerships, it is important for ESDC and provinces and territories to discuss current Labour Market Partnerships funded activities in order to make recommendations on how to best report on results.

Considering the intermittent or limited use of the program in some provinces and territories, it is essential to share lessons learned about successful Labour Market Partnerships projects. Particularly, for projects targeted to employers (such as workplace or employer-sponsored training), and those assisting communities and economic sectors dealing with labour market adjustment issues (contraction or expansion).

Funded organizations

Funded organizations include:

• not-for-profit organizations (such as research organizations, school boards, and Indigenous organizations)
• businesses/employers
• educational institutions and training providers
• municipal and local governments including Indigenous governments

Funded Research and Innovation activities

Research and Innovation projects encompassed a variety of activities including:

• development and/or testing of new approaches to improve employment outcomes for clients with some projects also focusing on persons with disabilities, youth, Indigenous, and other demographic groups
• strengthening service providers
• improving learning and post-secondary education with a focus on expanding online course delivery
• funding for cost-sharing of internships, temporary work placements, or training
• delivering career fairs or career/employment information presentations
• research

Innovation definition and criteria

Provinces and territories use different criteria to determine what is innovative. For example, in 4 provinces and territories, innovative approaches are deemed to be either:

• tools or processes that have not been used in the jurisdiction or Canada in general; and/or
• for a specific client group and that help to improve existing services or programs

Dissemination and adoption of innovative approaches

Five participating jurisdictions confirmed that project results can be shared with other provincial/territorial departments. They can also be shared with various stakeholders, which can include employment practitioners, employers, communities, researchers, and other key industry stakeholders.

Dissemination activities include: 

• synthesising best practices
• developing knowledge and presenting products such as webinars, conference presentations, infographics, online products, and articles
• creating/enriching portals of the project holders
• issuing media releases
• developing and piloting a new apprenticeship curriculum

Performance measurement

Evaluation of project results was an integral part of Research and Innovation performance measurement in 5 provinces/territories.^{Footnote 51}

Challenges and lessons learned

Project documents and program officials identified challenges related to testing and identification of innovative approaches including:

• project holder staff turnover due to poor terms of employment and job security
• recruitment and/or retention of participants to implement pilot projects
• recruitment of employers as partners in the delivery of supports to participants

In relation to factors contributing to successful testing and identification of innovative approaches, program officials highlighted the importance of:

• project holders employing experienced staff
• possessing organizational structure and financial reporting capacity
• having strong commitments with partners
• providing detailed cost estimates as part of their project proposals
• having clear project implementation plan with measurable outcomes
• continuous application intake to address on-going and emerging labour market issues

Labour market outcomes

The labour market outcomes are based on individuals who began their participation during the 2010 to 2012 period. Statistics focus on 5 years before program participation and 5 years after the program start year.

Active claimants

As shown in Chart 6, program participants increase their average earnings from $19,325 in the fifth year pre-program to $56,131 in the fifth year after the program start year.

Text Description – Chart 6

Year pre-post participation	Employment earnings for active claimants
5 years pre-program	$19,325
4 years pre-program	$21,309
3 years pre-program	$22,841
2 years pre-program	$25,890
1 year pre-program	$32,552
Program start year	$33,607
1 year post-program	$41,638
2 years post-program	$47,322
3 years post-program	$51,971
4 years post-program	$54,795
5 years post-program	$56,131

The proportion of employed participants declines by 1 percentage point annually after the program start year, but remains around 96%. The proportion of participants on EI Part I decreases from 100% in the program start year to 35% in the fifth year after the program start year. Participants decrease their dependence on income support from 23% in the program start year to 5% in the fifth year after participation.

Former claimants

As shown in Chart 7, program participants increased their average earnings from $21,772 in the fifth year pre-program to $58,158 in the fifth year after the program start year.

Text Description – Chart 7

Year pre-post participation	Employment earnings for former claimants
5 years pre-program	$21,772
4 years pre-program	$25,123
3 years pre-program	$27,146
2 years pre-program	$30,176
1 year pre-program	$34,663
Program start year	$41,811
1 year post-program	$48,732
2 years post-program	$53,120
3 years post-program	$56,470
4 years post-program	$57,669
5 years post-program	$58,158

The proportion of employed participants declined by 1 percentage point annually after the program start year, but remained around 95%. The proportion of participants on EI Part I decreased from 70% in the program start year to 28% in the fifth year after the program start year. Participants decreased their dependence on income support from 12% in the program start year to 5% in the fifth year after participation.