Evaluation of the Canada – Newfoundland and Labrador Labour Market Development Agreement

From: Employment and Social Development Canada

3. Comparison of key findings by program type
4. Conclusions
5. Recommendations
References
Acronyms
Appendix A - Methodology
Appendix B - Detailed results Skills Development
Appendix C - Detailed results Skills Development-Apprentices Supports
Appendix D – Detailed results JobsNL
Appendix E - Detailed results Self-Employment Assistance
Appendix F - Detailed results Job Creation Partnerships
Appendix G - Detailed results Employment Assistance Services
Appendix H - Overview of studies included in this report

Alternate formats

Evaluation of the Canada – Newfoundland and Labrador Labour Market Development Agreement [PDF - 754 KB]

Large print, braille, MP3 (audio), e-text and DAISY formats are available on demand by ordering online or calling 1 800 O-Canada (1-800-622-6232). If you use a teletypewriter (TTY), call 1-800-926-9105.

Executive summary

Introduction

Employment and Social Development Canada (ESDC) worked jointly with Newfoundland and Labrador and 11 other Provinces and Territories (P/Ts) to undertake the 2012 to 2017 second cycle of the Labour Market Development Agreement (LMDA) evaluation. The first cycle of the LMDA evaluation was carried out between 1998 and 2012 and involved conducting bilateral formative and summative evaluations in all P/Ts. Under the second cycle, the evaluation work consisted of conducting two to three studies per year on the Employment Benefits and Support Measures (EBSMs) similar programming delivered under these agreements. The studies generated evaluation evidence on the effectiveness, efficiency and design/delivery of EBSMs for Canada, for Newfoundland and Labrador and for the 11 other P/Ts that opted for a joint evaluation process with Canada.

Under LMDAs, Canada transfers $2.14B in Employment Insurance (EI) Part II funds to P/Ts for the design and delivery of programs and services to help unemployed individuals, mainly eligible under EI, to find and maintain employment.

Programs and services delivered by Newfoundland and Labrador have to correspond to the EBSM categories defined under the Employment Insurance Act. The following is a short description of the five programs and services examined in the evaluation:

Skills Development (including Apprentice Supports) helps participants obtain employment skills by giving them financial assistance in order to attend classroom training.
Targeted Wage Subsidies (JobsNL) help participants obtain on-the-job work experience by providing employers with a wage subsidy.
Self-Employment (Self-Employment Assistance) provides financial assistance and business planning advice to participants to help them start their own business.
Job Creation Partnerships provides participants with opportunities to gain work experience that will lead to ongoing employment. Employment opportunities are provided by projects that contribute to developing the community and the local economy.
Employment Assistance Services such as counselling, job search skills, job placement services, provision of labour market information and case management.

Three additional programs and services are available under the LMDAs and they are: Labour Market Partnerships, Research and Innovation and Targeted Earnings Supplements. They were not evaluated as part of this evaluation. The Research and Innovation and Targeted Earnings Supplements programs are not used in Newfoundland and Labrador while Labour Market Partnerships will be evaluated at a later date.

Table i provides an overview of the share of funding allocated to the five EBSMs examined under the second cycle for the LMDA evaluation and the average cost per participant.

**Table i. Share of LMDA funding and average cost per participant**
Program and service	Share of funding 2014 to 2015	Average cost per participant 2002 to 2005
Skills Development (including Apprentice Supports)	77.2%	$11,797
Job Creation Partnerships	6.2%	$7,013
JobsNL (Targeted Wage Subsidies)	6.4%	$9,577
Self-Employment Assistance	4.4%	$17,100
Employment Assistance Services	4.1%	$741
Labour Market Partnerships	1.2%	-
Total	100%	–

Sources: EI Monitoring and Assessment Reports 2002 to 2003 to 2014 to 2015.

Note: the average cost per participate was calculated based on the unit of analysis called the Action Plan Equivalent. This unit accounts for all the interventions taken by a participant.

This report presents a summary of the findings from eight studies produced on Newfoundland and Labrador LMDA interventions. Results are presented for active and former EI claimants as well as for long-tenured workers^{Footnote 1}, youth (under 30 years old) and older workers (55 years old and over) when the number of participants was sufficient to conduct quantitative analyses. Active EI claimants were receiving EI benefits at the time of their EBSM participation. Former EI claimants received EI up to three years before starting their EBSM participation.

Key findings

Effectiveness and efficiency of EBSMs

Incremental impacts and cost-benefit analyses addressed EBSM effectiveness and efficiency. The incremental impacts demonstrate that LMDA programs and services are improving the labour market attachment of active EI claimants, including youth and older workers who participated in Skills Development and Employment Assistance Services. For former claimants, participation in Skills Development led to an increase in labour market attachment. Social benefits of participation exceeded the cost of investments for most interventions over time. Finally, providing Employment Assistance Services interventions earlier during an EI claim (first four weeks) produced larger impacts on earnings and employment and facilitated earlier return to work. This demonstrates the importance of targeting early participation of EI active claimants.

Figure i presents the incremental impacts on the incidence of employment for active and former claimants by type of program. The estimates can be interpreted as a change in the probability of being employed following participation. For example, participation in Skills Development increases the probability of being employed by 1.8 percentage points for active EI claimants relative to unemployed non-participants.

For active claimants, the incremental impacts demonstrate that participants in Employment Assistance Services improved their incidence of employment in the medium-term only (1.1 and 1.2 percentage points in years four and five following participation). However, as demonstrated in the report, participation in Employment Assistance Services led to increase in employment earnings. As well, providing Employment Assistance Services earlier during the EI claim (first four weeks) produced larger impacts on earnings and employment and facilitated earlier returns to work.

**Figure i. Change in probability of being employed in participants relative to non-participants**

	Percentage Points
Active Claimants	Former Claimants
Skills Development	1.8	4.2
JobsNL	2.7	3.0
Job Creation Partnerships	2.0	2.9*
Employment Assistance Services	0.4*

Figure ii presents the cumulative increase in employment earnings for active and former claimants over the five years post-participation. Active EI claimants who participated in Skills Development, JobsNL and Employment Assistance Services increased their employment earnings compared to similar non-participants. For former EI claimants who participated in Skills Development, JobsNL and Job Creation Partnerships, the incremental impacts were not statistically significant. As such, former claimants did not increase their employment earnings compared to former EI claimants who received only Employment Assistance Services exclusively.

**Figure ii. Increased cumulative earnings of participants relative to non-participants**

	Gains in Earnings
Active Claimants	Former Claimants
Skills Development	$16,644	$930*
JobsNL	$13,639	$5,823*
Job Creation Partnerships	$4,205*	-$7,855*
Employment Assistance Services	$12,531

*The estimates are not statistically significant at the 95% confidence level.

Table ii presents the number of years required for the social benefits to exceed program costs. Social benefits to participation exceeded investment costs in a period ranging between the second year of program participation to just over 17 years after participation. This excludes former claimants who participated in Job Creation Partnerships for whom the investment costs are not likely to be recouped. Job Creation Partnerships represented nearly 5% of new EBSM interventions in 2014 to 2015 in Newfoundland and Labrador.

**Table ii. Number of years for the benefits to exceed program costs**
	Skills Development	JobsNL	Job Creation Partnerships	Employment Assistance Services
Active claimants	6	2.5	15.7	2.1
Former claimants	17.4	2nd Year of participation^{Footnote 2}	Benefits may never recover the costs	n/a

Main challenges about EBSM design and delivery

Key informants interviews with service providers and program managers as well as the reviewed documents and the questionnaires completed by Newfoundland and Labrador representatives identified challenges about EBSM design and delivery:

Skills Development

As reported in the evaluation, the Skills Development application process aims to ensure that prospective participants are choosing a trade that will meet labour market demand. This may contribute to program effectiveness.

According to interviewed service providers and managers, the main challenges related to Skills Development design and delivery included:

Transportation is difficult for participants in rural areas.
Lack of staff for the current caseload particularly in rural areas.
Some prospective participants might have benefited from Skills Development but lacked sufficient labour market attachment to qualify for EI and participate in the program.
The Skills Development application process takes time to complete and there may be a waiting period before the start of training which creates financial barriers for participants when their EI benefits end.
The perceived insufficient level of available financial support to address costs associated with tuition, relocation and childcare.
The perceived insufficient level of available financial support to deal with the high demand for training.

Skills Development – Apprentice Supports

Existing literature showed that there is a fairly high non-completion rate among apprentices (40 to 50%)^{Footnote 3}. Furthermore, subject matter literature revealed that despite the growth in apprenticeship registrations in Canada, there has not been a corresponding increase in completing all required training and passing the certification exam. It was not possible with available data to generate a reliable estimation of the completion rate of Skills Development-Apprentice Supports participants. According to key informants, apprentices do not complete their training due to factors such as:

Financial constraints.
The requirement to travel in order to participate in block release training.
The apprentice realises that he/she is not suited for the trade selected.
Employers are not willing or able to let their apprentices go on training.
Change in the economy such as an economic downturn and sudden lack of work.

JobsNL

JobsNL is an employer-driven program that provides financial support to employers for positions that promote sustainable long-term or seasonal employment. Challenges with the design and delivery of JobsNL included:

Lack of staff expertise in dealing with all types of clients.
Lack of awareness of JobsNL among employers particularly in rural areas.
Lack of employers willing to participate in the program.
Lack of access to Records of Employment to help improve employer assessments and verify whether employers have recently laid off staff for similar positions.

Employment Assistance Services

According to key informants from 10 P/Ts, challenges with the design and delivery of Employment Assistance Services included:

Lack of awareness about Employment Assistance Services among potential participants
Current budget allocation is not enough to support the delivery of Employment Assistance Services and has led some service providers to eliminate services
Service providers cannot provide all the services needed for participants facing multiple barriers to employment. They have to refer these individuals to other organizations

Recommendations

A total of seven recommendations emerge from the evaluation findings. They are as follows:

The study on the timing of Employment Assistance Services participation showed that receiving assistance early after starting an employment insurance claim can lead to better labour market impacts.

Recommendation 1: Consideration should be given by Newfoundland and Labrador to request timely access to data on new EI recipients for targeting purposes and to raise awareness about programs and services if necessary.

Key informants reported that the lack of work experience, transportation in rural areas, financial challenges and EI eligibility may be barriers to accessing training. As well, Skills Development accounts for 77% of LMDA spending in 2014 to 2015 with 49.3% of funding allocated to Adult Basic Education, Essential Skills, Literacy and pre-apprenticeship training.

Recommendation 2: Consideration should be given to removing barriers to accessing training by alleviating financial challenges and expanding eligibility requirements.

Recommendation 3: It is recommended that Adult Basic Education, Essential Skills, Literacy and pre-apprenticeship training interventions be reported separately from other Skills Development interventions given their unique objectives.

Key informants across Canada confirmed the necessity of having labour market information to support the delivery of Employment Assistance Services. They, however, pointed out that at the National level, the labour market information had limited value, and at the regional or local level, was outdated or unavailable.

Recommendation 4: Consideration should be given to enhance the production of relevant labour market information to support service delivery.

The evaluation was not able to produce a conclusive assessment of the Self-Employment Assistance effectiveness and efficiency since the data used to assess the impacts on earnings may not be the best source of information available to reflect the financial wellbeing of the participants. As well, little is known about the design and delivery of this program. Overall, it is not clear whether the participant’s success in improving their labour market attachment through self-employment is more closely associated with their business idea and their entrepreneurship skills than with the assistance provided under the Self-Employment Assistance program.

Recommendation 5: Consideration should be given to examine in more detail the design and delivery of Self-Employment Assistance and whether the performance indicators for this program are appropriate.

Job Creation Partnerships was found to be less effective for former claimants. As well, the evaluation has not yet examined the design and delivery of this program. Therefore, a lot remains unknown about how this program operates and the factors that could be changed to improve its effectiveness.

Recommendation 6: Consideration should be given to examine the design and delivery of Job Creation Partnerships to better understand how this program operates and how to improve its effectiveness.

Overall, the LMDA evaluation was able to produce a sound assessment of EBSM effectiveness and efficiency because the team had access to rich data on EI claimants and was capable of linking them to EBSM participation data and Canada Revenue Agency taxation files. However, some data gaps limited the evaluation’s ability to assess how EBSMs operate.

Recommendation 7: Improvements in data collection is recommended to address key program and policy questions of interest to the federal and provincial/territorial governments. Specifically:

Collect data on whether participants are members of designated groups including Indigenous peoples, persons with disabilities and recent immigrants.
Collect data on the type of training funded under Skills Development and the type of assistance provided under Employment Assistance Services. Newfoundland and Labrador, ESDC and other P/Ts should work together to define common categories for both EBSMs.

Management response

The Department of Advanced Education, Skills and Labour accepts the findings and recommendations presented in the Evaluation of the Canada-Newfoundland and Labrador Labour Market Development Agreement Synthesis Report. The Department is pleased to provide the following Management Response to the evaluation’s recommendations.

Recommendation 1: Consideration should be given by Newfoundland and Labrador to request timely access to data on new Employment Insurance recipients for targeting purposes and to raise awareness about programs and services if necessary.

Response: Potential implementation of Targeting, Referral and Feedback and two-way data sharing is being discussed during bilateral negotiations of the amended Labour Market Development Agreement. The province looks forward to these discussions.

Recommendation 2: Consideration should be given to removing barriers to accessing training by alleviating financial challenges and expanding eligibility requirements.

Response: The province periodically reviews and assesses policies to ensure that programming is appropriately responsive to client’s needs including financial supports and eligibility requirements in light of changing labour market considerations in our province.

Response: The province has the capability to report separately on these programs but it requires manual effort. In addition, the current LMDA does not permit the exchange of this data. However, as part of negotiations on the amended LMDA, the province is exploring system upgrades to meet accountabilities under the new Performance Measurement Strategy. Depending on upgrades required and associated costs, this separate reporting could be possible.

Recommendation 4: Consideration should be given to enhance the production of relevant labour market information to support service delivery.

Response: The province does produce labour market information at the regional and local level that supports program policy and service delivery. For example, in 2015, the province updated its provincial occupational forecast report covering 2015 to 2025. Further, the newly formed national Labour Market Information Council will likely influence additional labour market information that can be further utilized to support program policy and service delivery.

Response: The multilateral LMDA Evaluation Steering Committee has been tasked with overseeing the selection of topics for consideration. This study may be considered as part of the upcoming Cycle III evaluation.

Recommendation 7: Improvements in data collection is recommended to address key program and policy questions of interest to the federal and provincial/territorial governments. Specifically:

Collect data on whether participants are members of designated groups including Indigenous peoples, persons with disabilities and recent immigrants.
Collect data on the type of training funded under Skills Development and the type of assistance provided under Employment Assistance Services. EDSC, Newfoundland and Labrador, and other Provinces and Territories should work together to define common categories for both Employment Benefits and Support Measures.

Response: Improvements in data collection is being discussed as part of the new Performance Measurement Strategy that will inform the amended LMDA. Additional recommendations for improvements in data collection will also be informed through the established multilateral steering committee and approved by designated officials in each jurisdiction.

1. Introduction

Employment and Social Development Canada (ESDC) worked jointly with Newfoundland and Labrador and 11 other Provinces and Territories (P/Ts) to undertake the 2012 to 2017 second cycle for the Labour Market Development Agreement (LMDA) evaluations. The first cycle of LMDA evaluation was carried out between 1998 and 2012 and involved the conduct of bilateral formative and summative evaluations in all P/Ts. Under the second cycle, the evaluation work consisted of conducting two to three studies per year on the Employment Benefits and Support Measures (EBSMs) similar programming delivered under these agreements. The studies generated evaluation evidence on the effectiveness, efficiency and design/delivery of EBSMs for Canada, Newfoundland and Labrador and for the 11 other P/Ts that opted for a joint evaluation process with the Government of Canada.

This report presents a summary of the findings from the studies conducted for Newfoundland and Labrador and is organised as follows:

Introduction with an overview of the studies summarized in this report including their scope, methodology, and contextual information on the LMDAs
Findings section with a discussion around the rationale for investing in labour market programming and summary of evaluation evidence
Comparison of key findings by program type
Conclusions and lessons learned
Recommendations that emerge from the evaluation findings and areas for future investigation

1.1 Labour Market Development Agreement background

LMDAs are bilateral agreements between Canada and each P/T, and were established under Part II of the 1996 Employment Insurance (EI) Act. As part of these agreements, Canada transfers $2.14B annually to P/Ts (including $190M in administration funds) to design and deliver programs and services to assist individuals prepare for, obtain and maintain employment. Specifically, Newfoundland and Labrador receives approximately $122.2M in EBSM funding each year.

The Canada-Newfoundland and Labrador LMDA was signed on September 4, 2008. The agreement transferred responsibility to Newfoundland and Labrador for the design and delivery of the programs and services classified under two categories: 1) Employment Benefits and 2) Support Measures.

Employment Benefits

Employment Benefits are offered to unemployed individuals who 1) are actively on EI (that is, active claimants); 2) ended their benefit period within three years before participating (that is, former claimants); or 3) established a claim for maternity or parental benefits within the past five years and are returning to the labour force for the first time (that is, former claimants)^{Footnote 5}. Employment Benefits include the following categories:

Skills Development (including Apprentice Supports) helps participants obtain employment skills by giving them financial assistance to enable them to select, arrange and pay for classroom training.
Targeted Wage Subsidies (JobsNL) help participants obtain on-the-job work experience by providing employers with financial assistance to help with the wages of participants.
Self-Employment (Self-Employment Assistance) provides financial assistance and business planning advice to EI-eligible participants to help them start their own business. This financial assistance is intended to cover personal living expenses and other expenses during the initial stages of the business.
Job Creation Partnerships provides participants with opportunities to gain work experience that will lead to ongoing employment. Employment opportunities are provided by projects that contribute to developing the community and the local economy.
Targeted Earnings Supplements encourage unemployed persons to accept employment by offering them financial incentives. This program was not covered by the evaluation.

Support Measures

Support Measures are available to all unemployed individuals including those not eligible to receive EI and include:

Employment Assistance Services such as individual counselling, action planning, help with job search skills, job-finding clubs, job placement services, the provision of labour market information, case management and follow-up.
Labour Market Partnerships provide funding to help employers, employee and employer associations, and communities improve their capacity to deal with human resource requirements and implement labour force adjustments. These partnerships involve developing plans and strategies, and implementing labour force adjustment measures. This support measure was not covered by the evaluation.
Research and Innovation supports activities that identify better ways of helping people prepare for or keep employment and be productive participants in the labour force. Funds are provided to eligible recipients to enable them to carry out demonstration projects and research for this purpose. This support measure was not covered by the evaluation.

Table 1 provides an overview of the share of funding allocated to the five programs and services examined under the second cycle for LMDA evaluation and the average cost per participant. It is noted that the average cost per participant was calculated based on the 2002 to 2005 data from the EI Monitoring and Assessment Reports. The 2002 to 2005 period corresponds to the cohort of participants selected for incremental impacts and cost-benefit analysis in the LMDA evaluation.

**Table 1. Share of LMDA funding and average cost per participant**
Program and service	Share of funding 2014 to 2015	Average cost per participant 2002 to 2005
Skills Development (including Apprentice Supports)	72.2%	$11,797
Job Creation Partnerships	6.7%	$7,013
JobsNL (Targeted Wage Subsidies)	6.4%	$9,577
Self-Employment Assistance	4.4%	$17,100
Employment Assistance Services	4.1%	$741
Labour Market Partnerships	1.2%	–
Total	100%	–

Source: EI Monitoring and Assessment Reports 2002 to 2003 to 2014 to 2015.

Note: the average cost per participate was calculated based on the unit of analysis called the Action Plan Equivalent. This unit accounts for all the interventions taken by a participant.

1.2 Methodology

This section presents key aspects of the quantitative analyses carried out as part of the LMDA studies, while a more detailed description of the methodology is provided in Appendix A.

All quantitative analyses were based on administrative data from the EI Part I (EI claim data) and Part II (EBSM participation data) linked to the T1 and T4 taxation files from the Canada Revenue Agency. Incremental impact analyses and the cost-benefit analyses were based on up to 100% of participants in the reference period selected.

Incremental impacts analysis

Five studies assessed program effectiveness by estimating incremental impacts from EBSM participation on participants’ labour market experience (for example, earnings from employment/self-employment, incidence of employment, use of EI or Social Assistance and dependence on income support) after participation. The role of the incremental impact analysis is to isolate the effects of participation from other factors such as the economic cycle. In order to achieve this, the incremental impact analysis compared the labour market experience of participants, before and after their participation, with that of non-participants (see the example of incremental impact calculation in Figure 1).

The matching of participants and comparison group members used up to 75 socio-demographic and labour market variables observed over five years before participation. Two different comparison groups were used to measure impacts for active and former EI claimants. For active claimants, the incremental impacts were measured relative to a comparison group of active claimants who were eligible to, but did not, participate in EBSMs during the reference period.

Former claimants can be underemployed and unable to requalify for EI, out of the labour force for various reasons or on social assistance. Based on previous evaluation methodologies, on expert advice, and given the difficulty in generating a suitable comparison for former claimants using administrative data alone, the comparison group for former claimants was created using individuals who participated in Employment Assistance Services only during the reference period. This is a conservative approach given the fact that participation in Employment Assistance Services can lead to limited effects on labour market outcomes. In other words, the experience of former claimants who received employment benefits (that is, Skills Development, JobsNL, Self-Employment Assistance and Job Creation Partnerships) was compared to the experience of former claimants who received low intensity employment services (that is, Employment Assistance Services only). Due to this difference in measurement, incremental impacts estimated for active claimants should not be directly compared to former claimants^{Footnote 6}.

**Figure 1. Example of incremental impact calculation**

Factors accounted for in the cost-benefit analysis

Program efficiency was assessed through a cost-benefit analysis which compared the cost of participating in the program for the participants and the cost of delivering the program for the government to the benefits generated by the program. Overall, this analysis provided insights on the extent to which the program is efficient for the society (that is, for both the participants and the government). The costs and benefits accounted for in the calculations were as follows (see detailed definitions in Appendix A):

Program cost includes program and administration costs paid by the government.
Marginal social costs of public funds represent the loss incurred by society when raising additional revenues such as taxes to fund government programs.
Employment earnings consist of incremental impacts on participants’ earnings during and after participation. The calculation accounts for the participant’s forgone earnings during participation (that is, opportunity cost). Employment earnings were also increased by 15% to account for fringe benefits such as the employer-paid health, life insurance and pension contributions.

Strengths and limitations of the studies

One of the key strengths of the studies is that all quantitative analyses were based on administrative data rather than survey responses. Compared to survey data, administrative data are not subject to recall errors or response bias.

The propensity score models used to match participants and non-participants for the incremental impact analyses are judged to be robust in part because they were based on five years of pre-participation data and on a vast array of variables including socio-demographic characteristics, location, skills level related to the last occupation and indicators of labour market attachment.

Sensitivity analysis and the use of alternative estimation methods have increased confidence in the incremental impact estimates. However, one limitation with the propensity score matching techniques is that no one can be fully sure the impacts were not influenced by factors not captured in the data.

The cost-benefit analysis accounted for all quantifiable costs and benefits that are directly attributable to the EBSMs and could be estimated with the available administrative data. The analysis did not account for non-quantifiable benefits such as improvements in participant’s wellbeing or for the multiplier effect of increased spending on the economy.

It should be noted that it is not possible to produce an analysis of the incremental impacts for participants in Skills Development- Apprentice Supports. Assessing these impacts poses a methodological challenge because program participants are already employed and are expected to return to their employment after completing their training. Therefore, expected labour market impacts cannot be examined using a similar approach as for other EBSMs, which are expected to help participants return to employment. As well, the data available does not permit the identification of a proper comparison group since program participants alternate between work and training and no other potential counterfactuals have similar employment and training patterns. In this context, it is possible to provide an analysis of labour market outcomes associated with program participants but not possible to produce estimates of program effectiveness. While outcomes can provide some insights about the labour market experience of participants before and after participation, it is not possible to attribute the change observed in the outcomes to program participation. For example, a change in average annual earnings from before to after participation could be due to program participation or to other factors such as the maturation effect of youth, the economic cycle, lay off, etc.

When interpreting qualitative findings, readers should keep in mind that these are based on the perception of a small number of key informants who are directly involved in the design or delivery of the program. Their perception may be representative of their own region or community but not necessarily of the entire province. Since the number of key informants interviewed in each study is small (that is, ranging between six and seven), the number of informants who reported a specific finding is not indicated in the report. However, the report does note when there was a clear disagreement between key informants.

1.3 Overview of the studies summarized in this report

Findings presented in this report were drawn from eight separate studies. These studies examined issues related to program effectiveness, efficiency, design/delivery and used a mix of qualitative and quantitative methods. Each study examined evaluation issues in relation to active and former EI claimants:

Four quantitative studies estimated the incremental impacts for the participation in EBSMs for three cohorts of participants: 2002 to 2005, 2006 to 2008 and 2007 to 2009.
Three qualitative studies examined issues and questions related to program design and delivery in summer 2015.

Table H1 in Appendix H presents an overview of these studies, including the type of evidence generated, the methods used, the reference period and the length of the post-program period over which program effects were observed.

2. Evaluation findings

2.1 Rationale and labour market context

LMDA investments align with provincial government priorities

Active labour market programs are fairly similar across the Organization for Economic Co-operation and Development countries and consist of skills training in a classroom setting, work experience with employers (often subsidized) or in the public/non-profit sector, return-to-employment assistance and self-employment assistance. In Newfoundland and Labrador, the Department of Advanced Education, Skills and Labour administers a range of active labour market programming targeted at various groups of individuals. Programs and services offered to active and former EI claimants are mainly funded under the LMDA.

In the 2014 to 2017 strategic plan^{Footnote 7}, the Department of Advanced Education, Skills and Labour identified two strategic priorities to meet the Newfoundland and Labrador labour market needs. They are:

Alignment of labour market supply and demand.
Support for people to participate in employment opportunities.

These priorities were also mentioned in Newfoundland and Labrador’s 2016 Speech from the Throne where the government highlighted the “need for innovative and responsive labour market arrangements to support employment programs and skills training” and to “increase the number of apprentices”^{Footnote 8}.

Overall, the incremental impacts reported in the LMDA evaluation and discussed in this report demonstrated that LMDA funded programs and services delivered in Newfoundland and Labrador are generally helping participants to improve their labour market experience after participation and contribute to meeting the Department of Advanced Education, Skills and Labour’s key strategic priorities.

2.2 Skills Development

2.2.1 Program description

Based on a document review and seven key informant interviews completed in summer 2015

The objective of Skills Development in Newfoundland and Labrador is to assist active and former EI claimants to obtain the skills (from basic to advanced skills training) they need for employment. The program provides financial assistance to eligible individuals attending training. The level of support is based on a needs assessment which is part of the case management process. Financial supports include:

Basic living allowance
Trade tools
Childcare/dependent care
Transportation
Books
Living away from home allowance
Disability supports
Tuition
Equipment
Tutoring
Other instructional costs
Health and dental costs
Professional psycho-education assessment

Table 2, below, shows that most of the Skills Development interventions provided in Newfoundland and Labrador during the 2014 to 2015 fiscal year were aimed at vocational or occupational skills training (51%). Other types of training, such as pre-apprenticeship training, was the second most common type of training (32%) followed by adult basic education or essential skills training (17%).

**Table 2. Types of training supported 2014 to 2015**
Types of training	Proportion of Interventions	Number of Interventions
Adult Basic Education/Essential Skills	17.0%	945
Vocational/Occupational Training, including college or university	50.7%	2,826
Others (that is, pre-apprenticeship training)	32.3%	1,804
Total	100%	5,575

Source: Information submitted by the Newfoundland and Labrador Department of Advanced Education, Skills and Labour

In 2013 and 2014, Skills Development incorporated changes including offering basic living allowances to participants, reducing the mandatory contributions for participants from 20% to 10% with a maximum of $1,000; and, migrating the case management system to the Labour Market Programs Support System.

2.2.2 Program delivery

Based on a document review and seven key informant interviews completed in summer 2015

Skills Development participants are usually case managed by government Client Services Officers. Client Services Officers conduct needs assessments, employment assessments and complete an employment plan. Individuals are expected to conduct labour market research on potential occupations and identify a training program that is of interest to them.

All Skills Development applications are entered into the Labour Market Programs Support System. Approval decisions are based on labour market research, feasibility/employment prospects; personal suitability, and funding availability. On average, the length of the application and approval process ranges between six and eight weeks.

2.2.3 Targeting to labour market demand

Based on a document review and seven key informant interviews completed in summer 2015

The Skills Development application process, through case management and labour market research, ensures that prospective participants are choosing training that will meet the labour market demand. In particular, participants must demonstrate the demand for their chosen field of training. Applications that do not demonstrate sufficient demand may not be approved.

Local labour market information is used by the applicant to help inform their training choice. In addition, participants may be required to speak to employers or employees from their chosen occupation.

Table 3 presents the top five occupations that Skills Development participants were trained for during the year preceding the study.

**Table 3. Top five Skills Development supported occupations**
Occupation	Number of participants
Office Administration	418
Heavy Equipment Operator	416
Electrician	370
Occupational Health and Safety	230
Steamfitter/Pipefitter	177

Source: Information submitted by the Newfoundland and Labrador Department of Advanced Education, Skills and Labour

2.2.4 Profile of Skills Development participants

As shown in Table B1 in Appendix B, the majority of active claimants who started Skills Development participation between 2002 and 2005 and 2006 to 2008 were male (63% and 61% respectively). They were mainly between 25 and 44 years old (56%) in 2002 to 2005, and relatively older in 2006 to 2008 with 53% aged 35 years and over. Participants in both the 2002 to 2005 and 2006 to 2008 cohorts most frequently held occupations requiring college or apprenticeship training prior to participation (44% for the 2002 to 2005 cohort and 39% for the 2006 to 2008 cohort).

Former claimants who started Skills Development participation in 2002-2005 were males (52%) compared to 52% females for the 2006 to 2008 cohort. They were mainly between 25 and 44 years old (61%) in 2002 to 2005, and relatively younger in 2006 to 2008 with 58% aged under 35 years old. Former claimants in 2002 to 2005 most frequently held positions before participation requiring only on-the-job training (33%) compared to those in 2006 to 2008 who most frequently occupied jobs requiring secondary or occupational training (35%).

2.2.5 Incremental impacts

Active claimants

As shown in Table B2 in Appendix B, active claimants who started Skills Development participation between 2002 and 2005 had incremental gains in earnings and incidence of employment in four of the five years after participation. As shown in Figure 2, after the first year following participation, gains in earnings continuously increased over time and ranged between $2,450 and $5,119. Similarly, increases in the incidence of employment ranged between 1.8 and 2.8 percentage points. Participants also decreased their use of EI (by a cumulative total of $1,090) and social assistance (by a cumulative total of $161) during the post-program period, and reduced their level of dependence on income support between 2.0 and 2.8 percentage points following participation.

**Figure 2. Increased earnings of active and former Skills Development participants relative to non-participants^{Footnote 9}**

	Gains in Earnings
Active Claimants	Former Claimants
Year 1 Post-Program	$303*	-$2,072
Year 2 Post-Program	$2,450	-$691*
Year 3 Post-Program	$3,969	$435*
Year 4 Post-Program	$4,803	$962*
Year 5 Post-Program	$5,119	$2,296

Active claimants who started Skills Development participation in the 2006 to 2008 period had incremental gains in earnings and incidence of employment over the entire three years post-participation. Participants also decreased their use of EI benefits and their dependence on government income support. Impacts on the use of social assistance were not statistically significant.

Overall, active claimants increased their labour market attachment through increases in earnings, incidence of employment and a decrease in the level of dependence on government income support (use of EI and social assistance).

The results for sub-groups of active claimants were as follows:

Youth (under 30 years old) who started Skills Development participation in the 2002 to 2005 period had incremental gains in earnings and incidence of employment in most years after participation. As well, they decreased their use of EI and their dependence on income support over most post-program years. Impacts on the use of social assistance were not statistically significant.
Older workers (55 years old and over) who started Skills Development participation in the 2002 to 2005 period had incremental increases in earnings and incidence of employment during the post-program period. Their use of EI increased over the entire post-program period. The increase in EI use reflects the inability of some older workers to retain the employment secured following Skills Development participation. Impacts on the use of social assistance and the dependence on government income support were generally not statistically significant.
Long-tenured workers who started Skills Development participation between 2007 and 2009 increased their earnings and incidence of employment in the second and third years after participation. Incremental impact estimates on the use of EI, social assistance and the level of dependence on income support were not statistically significant.

Former claimants

Former claimants who started Skills Development participation between 2002 and 2005 had incremental increases in the incidence of employment ranging between 4.0 and 5.7 percentage points in four of the five post-participation years (see Table B3 in Appendix B). The incremental impacts on earnings were mixed since participants had a short-term decrease in earnings in year one post-program ($2,072) and an increase of $2,296 in year five post-program. As well, former claimants increased the use of EI by a cumulative of $1,356 following participation in Skills Development, indicating the inability for some participants to maintain the employment secured during the post-program period. Former claimants decreased the use of social assistance by a cumulative of $1,204 and reduced their dependence on income support by 3.5 percentage points in the first year following participation.

Former claimants who started participation between 2006 and 2008 decreased earnings and their EI use in the first year post-program but the subsequent years were not statistically significant. Their incidence of employment increased in the second and third years after participation and their use of social assistance decreased by a cumulative of $576 following participation.

Overall, former claimants increased their incidence of employment and decreased reliance on social assistance following participation. As well, the 2002-2005 participants increased their use of EI following participation. The increase in EI use indicates the inability of some former claimants to maintain the employment secured following participation. It can also be argued that the increase in EI use is an indication of some increase in labour market attachment for this client group since they did experience increases in the incidence of employment as well as a decrease in the use of social assistance. However, the increase in labour market attachment was not sustained in the post-program period considering the absence of positive incremental impacts on earnings.

Youth who started Skills Development participation between 2002 and 2005 experienced increases in incidence of employment (between 3.6 and 6.8 percentage points) in four of the five post-program years. They decreased their employment earnings, their use of social assistance and dependence on government income support in the first year after participation. Youth also experienced a cumulative increase in EI use of 6.6 weeks.

2.2.6 Cost-benefit results

As shown in Table B4 in Appendix B, for active claimants who started Skills Development participation in the 2002 to 2005 period, program benefits recovered the costs within six years after participation from the social perspective. For former claimants, program benefits were $20,184 lower than the costs six years after the end of participation. As such, the benefits will match the costs in 17.4 years after participation.

2.2.7 Challenges and Lessons Learned about Skills Development Design and Delivery

Based on a document review and seven key informant interviews completed in summer 2015

Key informants identified the following challenges in relation to the design and delivery of Skills Development:

Transportation is difficult for participants in rural areas
Lack of staff for the current caseload particularly in rural areas
Some prospective participants might have benefited from Skills Development but lacked sufficient labour market attachment to qualify for EI and participate in the program
The Skills Development application process takes time to complete and there may be a waiting period before the start of training which creates financial barriers for participants when their EI benefits end
The perceived insufficient level of available financial support to address costs associated with tuition, relocation and childcare
The perceived insufficient level of available financial support to deal with the high demand for training

Key informants also identified a number of lessons learned and best practices:

Provide financial support after the completion of training while participants are searching for employment
The use of the Skills Development Benefit Assessment Calculator helped to standardize entitlements
Increase support for the living allowance and medical/dental coverage for the participants’ families
Decrease the level of contribution that participants must make for their training.
Provide additional resources or learning disability supports
Using the Labour Market Programs Support System has improved case management, the application process, and client follow-up

2.3 Skills Development-Apprentices Supports

2.3.1 Program description

Based on a document review and six key informant interviews completed in summer 2015

The program provides funding to EI eligible individuals to participate in class call (block release) training of less than 12 weeks in duration that is not eligible for student aid.

Table 4 presents the number of apprentice participants for the top five trades.

**Table 4. Top five supported occupations**
Occupation	Number of participants
Electrician	1,006
Steamfitter/Pipefitter	312
Carpenter	306
Welder	266
Heavy Duty Equipment Technician	205

Source: Information submitted by the Newfoundland and Labrador Department of Advanced Education, Skills and Labour

In the past few years, there have been some administrative changes to the Skills Development –Apprentice Supports program. In 2013 to 2014, the program:

Integrated with Skills Development programming to standardize financial benefit and supports.
Introduced a basic living allowance which streamlined the administrative process for participants’ applying for dependent care support.
Developed a process to ensure strategic funding investments are made to support priority clients (for example, verification of family income based on the previous years’ income tax records).
Reduced the mandatory participant contribution to 10%, up to a maximum of $1,000.

2.3.2 Profile of Skills Development–Apprentice Supports participants

The majority of active claimants who started Skills Development –Apprentice Supports participation in 2003 to 2005 and 2013 to 2014 were male (97% and 92% respectively) (see Table C1 in Appendix C). The 2003 to 2005 cohort were older with 52% being 35 years of age and older compared to the 2013 to 2014 cohort who were 34 years old and younger (62%). As well, participants most frequently held occupations requiring college or apprenticeship training before participation (80% and 91% respectively).

Sixty-three percent of former claimants who participated in Skills Development –Apprentice Supports in 2003 to 2005 were male and 45 years of age and older (42%). For the 2013 to 2014 cohort, the percentage of male former claimants increased to 77%, however, this cohort was younger than the 2003 to 2005 cohort with 46% being 25 to 34 years old. Most commonly, claimants had occupations requiring college or apprenticeship training prior to participation (41% and 56% respectively).

2.3.3 Labour market outcomes

The labour market outcomes were based on individuals who began their participation during the 2003 to 2005 period. Statistics presented in Tables C2 and C3 in Appendix C focused on five years prior and seven years after the program start year.

Active claimants

As shown in Figure 3, the employment earnings of active claimants increased from $18,127 to $63,362 between the fifth year before participation and the seventh year after the participation start year. However, their annual average incidence of employment slightly declines in the seven years after the participation start year (98% to 93%) compared to before participation (96% to 100%). This decline can be partially explained by retirement decisions given the fact that 8% of participants were 55 years old and over at the start of program participation. Participants reported less EI benefits during the post-program period and less dependence on government income support.

**Figure 3. Average earnings for active claimant participants in Skills Development –Apprentice Supports**

	Employment Earnings
5 yrs pre-program	$18,127
4 yrs pre-program	$22,190
3 yrs pre-program	$22,823
2 yrs pre-program	$24,362
1 yrs pre-program	$26,295
Program start year	$26,375
1 yrs post-program	$37,536
2 yrs post-program	$44,163
3 yrs post-program	$54,148
4 yrs post-program	$59,051
5 yrs post-program	$55,975
6 yrs post-program	$59,904
7 yrs post-program	$63,362

Former claimants

As shown in Figure 4, the employment earnings of former claimants increased from $10,544 in the fifth year pre-program to $27,131 in the seventh year after the participation start year. The average annual proportion of employed from 94% in the program start year to 80% in year seven after the program start year. This decline can be partially explained by retirement decisions given the fact that 16% of participants were 55 years old and over at the start of program participation. Participants decreased their use of EI in the post-program period.

**Figure 4. Average earnings for former claimant participants in Skills Development-Apprentice Supports**

	Employment Earnings
5 yrs pre-program	$10,544
4 yrs pre-program	$14,299
3 yrs pre-program	$14,591
2 yrs pre-program	$13,556
1 yrs pre-program	$10,129
Program start year	$12,572
1 yrs post-program	$18,750
2 yrs post-program	$22,217
3 yrs post-program	$24,478
4 yrs post-program	$26,478
5 yrs post-program	$25,591
6 yrs post-program	$25,278
7 yrs post-program	$27,131

2.3.4 Challenges and lessons learned about Skills Development –Apprentice Supports design and delivery

Based on a document review and six key informant interviews completed in summer 2015

Existing literature has shown that there is a fairly high non-completion rate among apprentices in Canada (40 to 50%)^{Footnote 10}. Furthermore, subject matter literature revealed that despite the growth in apprenticeship registrations in Canada, there has not been a corresponding increase in completing all required training and passing the certification exam^{Footnote 11}. While available data do not provide reliable information on completion and non-completion rates for Skills Development –Apprentice Supports participants, key informants interviewed in the evaluation identified reasons that could lead apprentices to drop-out from the apprenticeship process. These include:

Financial constraints
Travelling to participate in block release training
The apprentice realises that he/she is not suited for the trade selected
Employers are not willing or able to let their apprentices go on training
Change in the economy such as an economic downturn and sudden lack of work

Challenges in relation to the design and delivery of Skills Development –Apprentice Supports or apprenticeship training in general included:

Apprentices have difficulties travelling to attend the block release training.
Training choices do not always match labour market demand.
There is no list of registered apprentices ready for block release training.
Program staff experienced an increased workload prior to the start of the training programs.
Financial hardships for apprentices due to delays in processing EI claims.
Lack of coordination between the Department of Advanced Education, Skills and Labour staff and training institutions.
The apprenticeship-journeyman ratio (2:1) is too high.
Participants with multiple barriers may require more time for training or alternative delivery options:

Need supports that are not always consistently available in the training institutions;
Have difficulties finding tutors particularly in rural areas.

Best practices and lessons learned related to the program design and delivery included:

Co-locating Skills Development –Apprentice Supports staff with Industrial Training staff helped resolve issues and increased the flexibility of the program.
The detailed step-by-step information provided to apprentices at the beginning of the process assists them in understanding what supports are available, what steps are required, and what experience they should be gaining between each class call.
Apprentices with multiple barriers may require an alternative to the eight-week model of block release training.

2.4 JobsNL

2.4.1 Program description

Based on a document review and six key informant interviews completed in summer 2015

The objective of JobsNL is to encourage employers to hire unemployed EI-eligible individuals they would not normally hire resulting in these individuals gaining work experience. To reach this objective, JobsNL provides a subsidy to employers to cover a portion of the participant’s salary.

The JobsNL program provides financial support to positions that promote sustainable long-term full-year employment or seasonal employment. Priority for funding is provided to occupations and sectors identified by Department of Advanced Education, Skills and Labour with consideration given to evenly distributing the funding, especially in areas of high unemployment as well as to income support clients, persons with a disability or recent post-secondary graduates.

The length of the subsidy can vary from 10 to 52 weeks. Positions must provide between 30-40 hours per week of employment to the participant. The maximum amount of the subsidy is $14,450 regardless of the wage rate and duration of the subsidy. Employment related expenses are not covered by JobsNL support.

Table 5 presents the top five sectors that JobsNL participants were trained for during the year preceding the study.

**Table 5. Top five JobsNL supported sectors**
Sectors	Proportion of participants
Accommodation and Food Services	22.5%
Retail Trade	22.4%
Construction	13.9%
Arts, Entertainment and Recreation	7.5%
Health Care and Social Assistance	5.8%

Source: Newfoundland and Labrador Department of Advanced Education, Skills and Labour

2.4.2 Program delivery

Based on a document review and six key informant interviews completed in summer 2015

JobsNL is an employer-driven program. Employers initiate the application process by contacting Client Service Officers expressing their interest in participating in the program. Client Service Officers determine the employers’ eligibility for the program and, if eligible, the employer is responsible for finding a potential candidate.

Once the employer identifies a potential candidate, Client Service Officers assess the potential candidate to ensure a good fit. The Client Service Officer obtains consent from the individual, confirms their eligibility for the program and gathers their personal information for contract purposes. An Agreement Manager assesses the application, determines entitlements, recommends approval and obtains signatures and monitors payments to the employer. The employment contract can be approved by either the Client Service Officer or the Client Services Manager, depending on the amount of the subsidy.

JobsNL applications are submitted online using the self-service Labour Market Program Support System. Employers identify the number of hours and the wage associated with the position.

Overall, the application and approval process usually takes approximately two to four weeks.

2.4.3 Profile of JobsNL participants

As shown in Table D1 in Appendix D, active claimants who started JobsNL participation in 2002 to 2005 were predominately male (54%) compared to those who participated in 2006 to 2008 who were mostly female (58%). Sixty-three percent of those who started participating between 2002 and 2005 were between 25 and 44 years of age compared to the 2006 to 2008 participants who were mainly aged 35 years and older (62%). One third of the 2002 to 2005 participants and 40% of the 2006 to 2008 participants most frequently held jobs requiring secondary or occupational training prior to program participation.

Slightly more than half of former claimants in the 2002 to 2005 cohort were male (52%) compared to the 2006 to 2008 cohort who were mainly female (59%). Participants in both cohorts were generally between 25 and 44 years of age (65% and 59%, respectively). Thirty five percent of the 2002 to 2005 participants and 37% of the 2006 to 2008 participants’ most frequently held occupations requiring secondary or occupational training prior to JobsNL participation.

2.4.4 Incremental impacts

Detailed incremental impact results for active and former claimants are presented in Tables D2 and D3 in Appendix D.

Active claimants

Active claimants who started JobsNL participation between 2002 and 2005 had incremental increases in earnings after participation ranging from $2,177 to $3,462 (see Figure 5). Their incidence of employment also increased between 2.6 and 4.0 percentage points in three of the five years following participation. Participants reduced the use of EI by a cumulative of $3,685 or 9.7 weeks and decreased their dependence on income support between 3.4 and 5.0 percentage points following participation. Results for social assistance use were not statistically significant.

Active claimants who started JobsNL participation between 2006 and 2008 increased their earnings and incidence of employment over the three post-program years. Participants reduced the use of EI and social assistance as well as the level of dependence on income support following participation.

Overall, active claimants improved their labour market attachment through increases in earnings and incidence of employment as well as decrease in the use of EI and the overall dependence on government income support.

**Figure 5. Increased earnings of active and former JobsNL participants relative to non-participants**

	Gains in Earnings
Active Claimants	Former Claimants
Year 1 Post-Program	$2,189	$2,398
Year 2 Post-Program	$2,177	$1,006*
Year 3 Post-Program	$3,462	$1,484*
Year 4 Post-Program	$3,191	$724*
Year 5 Post-Program	$2,620	$212*

*The estimates are not statistically significant at the 95% confidence level.

Former claimants

Former claimants who started JobsNL participation in 2002 to 2005 had incremental increases in their incidence of employment between 3.2 and 4.7 percentage points in the first three years following participation. As well, they had only a short-term increase in employment earnings following participation ($2,398 in year one). Former claimants decreased the use of social assistance in all post-program years for a cumulative of $1,443. The dependence on income support also decreased between 2.6 and 4.2 percentage points in three of the five years following participation. The overall estimates on the use of EI were not statistically significant.

Former claimants who started JobsNL in 2006 to 2008 increased their earnings by $1,737 and $1,656 in years two and three respectively following participation. As well, they increased their incidence of employment between 4.3 and 5.8 percentage points over the three post-program years. Participants also decreased their use of social assistance by a cumulative of $1,189 and the dependence on income support. The impacts on the use of EI were mixed with an increase use in year one (1.9 weeks) following participation and a decrease of $422 in year two.

Overall, former claimants increased their labour market attachment through increases in the incidence of employment and a decrease in the use of social assistance as well as in the level of dependence on government income support. As well, participants experienced short-term increases in employment earnings for both cohorts. For the 2002 to 2005 cohort, the increase in labour market attachment was not sustained in the post-program period considering the absence of positive incremental impacts on earnings beyond year one.

Youth who started their participation in the 2002 to 2005 period increased their earnings ($3,354) and their incidence of employment (6.4 percentage points) in the first year following participation only. They decreased their use of social assistance and their dependence on income support in most of the post-program years.

2.4.5 Cost-benefit results

As shown in Table D4, Appendix D for active claimants who started JobsNL participation between 2002 and 2005, the benefits of JobsNL recovered the costs within 2.5 years after participation from the social perspective. The total benefits six years after program end exceeded the costs by $10,581. Similarly, the benefits of JobsNL for former claimants exceeded the costs during the second year of program participation. The benefits six years after the end of participation exceeded the costs by $13,506.

2.4.6 Challenges and lessons learned about JobsNL design and delivery

Based on a document review and six key informant interviews completed in summer 2015

Key informants highlighted a number of challenges with the design and delivery of JobsNL including:

Lack of staff expertise to deal with all types of clients (EI and social assistance clients, Persons with Disabilities, clients with multiple barriers to employment).
Lack of awareness of JobsNL among employers particularly in rural areas.
Lack of employers willing to participate in the program.
Unable to access Records of Employment to help improve employer assessments and verify whether employers have recently laid off staff for similar positions.

Key informants were asked about the best practices and the lessons they learned in terms of the design and delivery of the program. These included:

Using the electronic Labour Market Programs Support System has reduced the amount of time required to submit progress updates by employers and process payments to employers.
There is a low administrative burden for JobsNL compared to other labour market programs.
Ensuring the same Client Service Officer assists employers with all requests related to JobsNL and other provincial programming helps build strong relationships with employers.
Ensuring that the wage subsidy will enhance participants’ skills and improve their labour market attachment is an important consideration in the application and approval process.

2.5 Self-Employment Assistance

2.5.1 Program description

Based on information available on the website of the Department of Advanced Education, Skills and Labour

The Self-Employment Assistance program provides financial and entrepreneurial assistance to eligible individuals to help them create jobs for themselves by starting a business. Funding is provided for living expenses after the EI benefit period has expired. Program duration is a maximum of 52 weeks or 78 weeks for persons with disabilities. Eligible participants must provide a 25% personal investment towards their business in either cash or in-kind contributions.

2.5.2 Profile of Self-Employment Assistance participants

As shown in Table E1 in Appendix E, active claimants who started Self-Employment Assistance participation between 2002 to 2005 were primarily male (54%) compared to those who started in 2006 to 2008 who were mainly female (53%). Most participants in both cohorts were between 25 and 44 years old (68% and 72% respectively). Both the 2002 to 2005 and 2006 to 2008 participants most frequently had occupations requiring college or apprenticeship training (36% and 37% respectively) followed by secondary or occupational training (30% each) prior to participation.

Former claimants who started Self-Employment Assistance participation in 2002 to 2005 were male (56%) and in 2006 to 2008, participants were primarily female (56%). Most participants in both cohorts were between 25 and 44 years old (66% and 70% respectively). Former claimants in both cohorts most frequently held occupations requiring college or apprenticeship training (32% and 34% respectively) prior to participation.

2.5.3 Challenges in measuring Self-Employment Assistance incremental impacts

Like other EBSMs, incremental impacts were estimated for Self-Employment Assistance participants in the 2002 to 2005 and 2006 to 2008 periods. Results showed large decreases in employment/self-employment earnings and decreases in the incidence of employment. As well, compared to similar non-participants, Self-Employment Assistance participants decreased their use of EI and reduced their dependence on government income support.

Detailed estimates are presented in Table E2 in Appendix E. However, they are not discussed in the report since they may not provide an accurate depiction of the financial well-being of Self-Employment Assistance participants in the post-program period. Impacts were examined using individual earnings reported in the T1 and T4 taxation files from Canada Revenue Agency, and measured relative to active claimants who did not participate in Self-Employment Assistance and may have been in any employment/ unemployment situation following participation (e.g., unemployed, paid employee or self-employed).

According to a study from Statistics Canada, self-employed individuals in Canada have a lower average annual income than paid employees ($46,200 versus $52,400 in 2009), but the average net worth of their households is 2.7 times greater than that of the paid employee households, which indicates that some self-employed individuals may leave funds within their business for reinvestment purposes^{Footnote 12}. Overall, this suggests that looking at individual earnings alone, without taking the net worth into consideration, may not provide a fair assessment of how well Self-Employment Assistance participants are doing financially after participation.

As well, little is known about the design and delivery of this program. In particular, there is a lack of understanding around the role played by this program in helping future entrepreneurs implement viable business plans and develop their entrepreneurship skills. Overall, it is not clear whether participant’s success in improving their labour market attachment through self-employment is more closely associated with their business idea and their entrepreneurship skills than with the assistance provided under Self-Employment Assistance program.

2.6 Job Creation Partnerships

2.6.1 Program description

Based on information available on the website of the Department of Advanced Education, Skills and Labour

The objective of Job Creation Partnerships is to provide unemployed individuals with opportunities to gain meaningful work experience. Job Creation Partnerships project activities are meant to benefit both the participant and the community. Program participants are expected to maintain or enhance their employability skills through this work experience opportunity, particularly if they have been unemployed for a long period of time. As a result of their participation in the program, participants will have recent work experience to add to their résumés. This experience, together with the networking which participants do while on a project, increases their chances of successfully finding ongoing employment.

In Newfoundland and Labrador, the program is delivered by the Department of Advanced Education, Skills and Labour. Applications are submitted online using the Labour Market Programs Support System. Eligible project sponsors include community organizations, municipal governments, band/tribal councils/crown corporations, businesses and public health and educational institutions.

2.6.2 Program delivery

Based on information available on the website of the Department of Advanced Education, Skills and Labour

Job Creation Partnerships projects must provide eligible participants with:

Work experience
Offer full-time hours to participants
Follow provincial labour legislation
Provide benefits to the community
Have a defined start and end date
Be implemented within a 52 week period
Be supported in partnership with other agencies, organizations and contributors
Be not-for-profit in nature^{Footnote 13}

Job Creation Partnerships provides funding for up to 30% of eligible project implementation costs. The amount of the financial support is negotiated with Department of Advanced Education, Skills and Labour and finalized in a funding agreement. Eligible project costs include: wages, licenses, permits, professional service fees, bank charges, utilities, material, supplies, administrative costs, participant orientation costs, workplace safety training, and capital costs. Capital costs must be cost-shared with either funding or in-kind contributions provided by the applicant.

2.6.3 Profile of Job Creation Partnerships participants

As shown in Table F1 in Appendix F, the majority of active claimants who started their Job Creation Partnerships participation between 2002 and 2005 were male (56%) compared to 62% of females for the 2006 to 2008 cohort. As well, 61% of participants in 2002 to 2005 and 72% of participants in 2006 to 2008 were 35 years and older and they most frequently held positions requiring on-the-job training before participation (42% and 45% respectively).

Former claimants who started Job Creation Partnerships participation in 2002 to 2005 were mostly male (66%) compared to 53% females for the 2006 to 2008 cohort. Sixty-one percent of those who started participating in 2002 to 2005 and 70% of the 2006 to 2008 cohort were 35 years of age and older. Participants in both cohorts most frequently held occupations requiring on-the-job training (38% and 39%) prior to participation.

2.6.4 Incremental impacts

Detailed incremental impact results for active and former claimants are presented in Tables Table F2 and F3 in Appendix F.

Active claimants

As shown in Figure 6, active claimants who started Job Creation Partnerships participation between 2002 and 2005 had incremental increases in earnings in years four and five following participation ($1,397 and $1,381 respectively). As well, they had increases in the incidence of employment in years three and five following participation by 2.6 and 2.5 percentage points respectively. Participants decreased their use of EI by a cumulative of $1,268 over the entire post program period and reduced their level of dependence on income support in the fourth and fifth years following participation by 2.6 and 2.5 percentage points respectively. The overall incremental impacts on the use of social assistance were not statistically significant.

Active claimants who started Job Creation Partnerships participation between 2006 and 2008 increased their incidence of employment in years one and three following participation by 3.2 and 2.3 percentage points respectively. Participants also decreased their use of EI by a cumulative of $1,275. Results for earnings, social assistance use and dependence on income support were not statistically significant.

Overall, active claimants reduced their use of EI following participation in Job Creation Partnerships and increased to a limited extent their probability of employment. For the 2002 to 2005 cohort, increases in earnings were observed in the medium-term only (specifically three years after the end of program participation). Job Creation Partnerships improved the labour market attachment of participants to a limited extent only.

**Figure 6. Increased earnings of active and former job creation partnerships participants relative to non-participants**

	Gains in Earnings
Active Claimants	Former Claimants
Year 1 Post-Program	-$51*	-$2,103
Year 2 Post-Program	$512*	-$2,214
Year 3 Post-Program	$966*	-$1,606*
Year 4 Post-Program	$1,397	-$1,468*
Year 5 Post-Program	$1,381	-$464*

*The estimates are not statistically significant at the 95% confidence level.

Youth (below 30 years old) who started Job Creation Partnerships participation in the 2002 to 2005 period decreased their EI use after participation. Incremental impacts on earnings, social assistance use and dependence on income support were not statistically significant. The incidence of employment increased in the fourth year (4.5 percentage points) after participation.

Former claimants

Former claimants who started Job Creation Partnerships participation between 2002 and 2005 had decreases in employment earnings in the first ($2,103) and second ($2,214) years following participation. As well, the use of EI increased by a cumulative total of 9.4 weeks following program participation. The decrease in earnings and the increase in EI use reflect the inability of participants to maintain employment after participation. The use of social assistance decreased by a cumulative total of $1,563 over the entire post-program period. Results for incidence of employment and dependence on income support were not statistically significant.

Former claimants who started Job Creation Partnerships participation between 2006 and 2008 had decreases in earnings in the post-program years ranging between $3,436 and $4,119. The use of EI increased by a cumulative of 4.6 weeks over the entire post-program period. Social assistance use decreased in the first year following participation ($145). The incremental impacts on the incidence of employment and the dependence on income support were not statistically significant.

Overall, former claimants who participated in Job Creation Partnerships increased the use of EI and decreased the use of social assistance. Considering the incremental decrease in earnings and the absence of statistically significant results on the incidence of employment, Job Creation Partnerships did not improve the labour market attachment of former claimants.

2.6.5 Cost-benefit results

As shown in Table F4 in Appendix F, for active claimants who started Job Creation Partnerships participation between 2002 and 2005, the benefits of Job Creation Partnerships recovered within 15.7 years after participation from the social perspective. The total benefits six years after program end were lower than the program cost by $7,931. For former claimants the benefits of Job Creation Partnerships may never recover the costs of the program. The benefits six years after the end of participation were lower than the program costs by $26,144.

When interpreting cost-benefit results for Job Creation Partnerships, it should also be acknowledged that Job Creation Partnerships funding helps to develop the community and the local economy. These benefits were not accounted for in the calculations.

2.7 Employment Assistance Services

2.7.1 Program description and delivery

Based on information provided by the Department of Advanced Education, Skills and Labour

The Department of Advanced Education, Skills and Labour assists unemployed individuals in Newfoundland and Labrador find employment. They provide a variety of services and supports including employment counselling and job search training as well as provide labour market information. Staff assesses the individual’s employment needs, suggest services and supports and develop employment plans.

To begin the application process, potential participants are required to complete and submit an Employment and Training Assessment that will help staff determine eligibility for services or financial supports. Staff will review the assessment and contact the applicant directly.

Employment Assistance Services is mainly delivered by staff from the Department of Advanced Education, Skills and Labour who provide case management to participants. The Province also funds some third-party organizations to provide employment assistance services to designated groups such as persons with disabilities, Indigenous individuals and women.

2.7.2 Profile of Employment Assistance Services participants

Socio-demographic characteristics

This socio-demographic profile was produced for individuals who only received Employment Assistance Services without participating in any other employment benefits programs.

Active claimants

As shown in Table G1 in Appendix G, most active claimants who started Employment Assistance Services participation in 2002 to 2005 and in 2006 to 2008 were male (64% and 58% respectively). Fifty-eight percent of those who started participating between 2002 and 2005 were between 25 and 44 years of age compared to the 2006 to 2008 participants who were mainly aged 35 years and older (61%). Active claimants most frequently held occupations requiring college or apprenticeship training before participation (34% and 36% respectively).

Former claimants

Most former claimants who started Employment Assistance Services in the 2002 to 2005 period were male (56%) compared to 51% of female participants in the 2006 to 2008 period. Fifty-seven percent of the 2002 to 2005 participants and 54% of the 2006 to 2008 participants were between 25 and 44 years of age. Both the 2002 to 2005 and 2006 to 2008 cohorts most frequently held occupations requiring secondary or occupational training prior to participation (34% each).

Labour Market barriers faced by Employment Assistance Services participants

Based on the National level results from 81 key informant interviews completed in 10 P/Ts in summer 2013

Managers and caseworkers from 10 P/Ts involved in the delivery of Employment Assistance Services were interviewed during the summer of 2013. These interviews, in addition to document reviews, provided insight into the challenges related to barriers faced by Employment Assistance Services participants. National-level qualitative findings are included in this report with the perspective of sharing lessons learned and best practices with Newfoundland and Labrador. The barriers include:

Lack of work experience or mismatched skills (nine P/Ts)
Low essential and foundational skills (eight P/Ts)
Access and affordability of transportation and childcare (eight P/Ts)
Criminal records and addictions (eight P/Ts)
Being a person with disability or having mental health issues (seven P/Ts)
Temporary, seasonal or part-time employment (seven P/Ts)
Lack of marketable skills (outdated skills, inability to network) (seven P/Ts)
Employers’ perception toward individuals in some groups (for example, visible minorities, persons with disabilities, new immigrants, Aboriginal peoples) (seven P/Ts)
Lack of employment opportunities, particularly in rural and remote areas (six P/Ts)
Language barriers (six P/Ts)
Low self-esteem, lack of motivation and negative attitude (six P/Ts)
Lack of job search/interview skills (five P/Ts).
Homelessness and lack of affordable housing (four P/Ts)

2.7.3 Incremental impacts

Results presented in Table G2 in Appendix G, indicate that active claimants who participated in Employment Assistance Services between 2002 and 2005 increased their employment earnings by a cumulative of $12,531 after participation. Participants also increased their incidence of employment in years four and five following participation (see Figure 7). As well, participants reduced their use of EI by a cumulative of $1,715 or 4.9 weeks and decreased the level of dependence on income support decreased in most post program years. The incremental impacts on the use of social assistance were not statistically significant.

**Figure 7. Incidence of employment for active claimant participants in Employment Assistance Services**

	Percentage Points
Year 1 Post-Program	-0.5*
Year 2 Post-Program	-0.2*
Year 3 Post-Program	0.6*
Year 4 Post-Program	1.1
Year 5 Post-Program	1.2

*The estimates are not statistically significant at the 95% confidence level.

Results for active participants who participated in Employment Assistance Services in 2006 to 2008 also showed incremental increases in employment earnings and gains in incidence of employment in the three years after participation. Their use of EI decreased by a cumulative of $679 or 1.3 weeks after participation and their dependence on income support did not change in the post-program period.

The results varied for the three sub-groups examined:

Youth (under 30 years old) who started an Employment Assistance Services intervention between 2002 and 2005 had incremental gains in earnings (a cumulative total of $12,420). They also decreased their EI use by a cumulative of $2,019 or 5.2 weeks following participation.
Older workers (55 years old and over) who started an Employment Assistance Services intervention between 2002 and 2005 had incremental gains in earnings (a cumulative total of $19,439) and increased the incidence of employment during most of the post-program period. Their use of EI increased by a cumulative total of $2,790 or 8.5 weeks in the post-program years while results for dependence on income support were generally not statistically significant.
Long-tenured workers who started an Employment Assistance Services intervention between 2007 and 2009 improved their earnings and incidence of employment after participation. They decreased their use of EI by a cumulative of $1,081 or 3.1 weeks. Participants increased their use of social assistance by a cumulative of $326 following participation. Dependence on income support decreased in the first year following participation by 5.4 percentage points.

Earlier participation in Employment Assistance Services improves participant’s Labour Market outcomes

The study on the effects related to the timing of participation showed that incremental impacts on earnings and employment were larger for individuals who received Employment Assistance Services early during their EI claim compared to individuals who remained on EI longer before receiving these services (see Figure 8 below and Table G3 in Appendix G). Specifically, individuals who started their participation within four weeks following the start of their EI benefit period had a total increase of $29,238 in their earnings over the five years post-program period.

**Figure 8. Cumulative incremental impacts on earnings related to the timing of participation in Employment Assistance Services**

	Cumulative Gains in Earnings
1 to 4 weeks	$29,238
5 to 8 weeks	$29,368
9 to 12 weeks	$28,074
2nd quarter	$19,217
3rd quarter	$14,535
4th quarter	$22,945

As well, the study looked at the difference between the number of EI weeks unused by participants and their comparison group to determine the effect of timing of Employment Assistance Services participation on return to employment. Those who started receiving assistance within the first four weeks of their claim returned to employment faster than the comparison group. Specifically, they returned to employment 1.6 weeks earlier than the comparison group (see Table G4 in Appendix G). Participants who started Employment Assistance Services after the fourth week of their EI claim returned to employment 1.7 to 4.8 weeks later than the comparison group.

2.7.4 Cost-benefit results

As shown in Table G5 in Appendix G, the benefits of Employment Assistance Services from the society perspective exceeded the cost by $15,837. It took 2.1 years after the end of participation for the benefits to recover the costs.

Incremental impact analyses showed that Employment Assistance Services are achieving their objectives since they increased participant’s earnings and employment and decreased their EI use after participation.

2.7.5 Challenges and lessons learned about Employment Assistance Services design and delivery

Based on the National level results from 81 key informant interviews completed in 10 P/Ts in summer 2013

Managers and caseworkers from 10 P/Ts involved in the delivery of Employment Assistance Services were interviewed during the summer of 2013. These interviews, in addition to document reviews, provided insight into the challenges and lessons learned related to program design and delivery. National-level qualitative findings are included in this report with the perspective of sharing lessons learned and best practices with Newfoundland and Labrador.

Key informants identified the following challenges related to program design and delivery:

Participants in some regions face issues with limited access to services, mobility and transportation (seven P/Ts).
There is a lack of awareness about the program among potential participants (six P/Ts).
The current budget allocation is not enough to support the delivery of Employment Assistance Services. This led some service providers to eliminate services and reduce the number of participants served (five P/Ts).
Service providers cannot necessarily provide all the services required by participants facing multiple barriers to employment. They have to refer these individuals to other organizations and sometimes, one individual can be referred to more than one organization. This may lead some participants to give up on their return-to-work process (five P/Ts).
The current performance measurement strategy does not capture the various outcomes achieved over time when assisting people with multiple barriers to employment (three P/Ts).
Service providers have difficulties in hiring skilled and knowledgeable staff (two P/Ts). As well, some service providers have a high turn-over of staff and staff training is very costly (two P/Ts).
Service providers lack capacity to follow up with each participant in order to provide job maintenance support (two P/Ts).

Key informants also provided examples of best practices and lessons leared with respect to program design and delivery. These included:

It is important to case manage participants and to provide a client-centered holistic approach through counselling, motivation, building self-esteem and assisting them in choosing a career path (nine P/Ts).
It is important for service providers to be engaged in their community and well connected to other service providers (for information sharing and referral purposes) particularly with those dealing with persons with disabilities and mental health issues. As well, partnerships and ongoing communication with employers can facilitate the labour market integration of participants through job placements and subsidy) (eight P/Ts).
Since participants with multiple barriers to employment often require more intensive interventions, it is important to conduct a strong needs assessment in order to make the best training decision (seven P/Ts).
Having a one stop shop for services and to streamline services (co-location, no wrong door approach, offering a comprehensive suite of services from self-serve to workshops, employability assessment, career orientation, need determination and ongoing case management) and to remove barriers to access and participation (seven P/Ts).
It is important to keep a long-term perspective when assisting participants facing multiple barriers to employment (six P/Ts).
Providing long-term follow-up with participants for employment retention support is seen as a best practice (six P/Ts).
Giving service providers increased flexibility when assisting participants with multiple barriers to employment particularly in terms of the length of services and the type of financial support that can be made available to participants (six P/Ts).
It is important for service providers to have dedicated workers, specialized teams to deal with participants facing multiple barriers to employment, having job coaches/developers that are dedicated and committed to support these participants (six P/Ts).
Service providers need to make appropriate referrals, when available, to specialized community organizations for Employment Assistance Services participants dealing with disabilities, mental health issues, addictions and criminal records (five P/Ts).
Providing participants with help to contact employers (for example, assistance with resumes) and networking opportunities (five P/Ts).
Providing participants with an opportunity to try and test prospective jobs (five P/Ts).
It is important for provincial/territorial governments to have strong partnerships with third-party service providers and employers in order to be able to mobilize the service delivery network in cases of emerging labour market challenges (major lay-offs, downturn, etc.) and to organize job fairs and joint group workshops (five P/Ts).
There is a need to enhance the promotion of programs and services (five P/Ts).
There is a need to provide services in an innovative way depending on local needs and reality (for example, online resources and training) (three P/Ts).
There is a need to change the way success is measured under Employment Assistance Services. For example, the progress of a participant with multiple barriers to employment should be measured through small steps from securing adequate housing to dealing with addictions, improving life skills and integrating the labour market (three P/Ts).
Services providers need additional resources in order to maintain the level and quality of services and to train staff, particularly those operating in rural areas (three P/Ts).

3. Comparison of key findings by program type

This section provides an overview of the key findings from the incremental impact analysis for Skills Development, JobsNL, Job Creation Partnerships and Employment Assistance Services for both active and former EI claimant participants who started participation in the 2002 to 2005 period.

For active EI claimants, the incremental impacts demonstrate that LMDA programs and services are improving the labour market attachment, including youth and older workers who participated in Skills Development and Employment Assistance Services. For former claimants, participation in Skills Development led to an increase in labour market attachment. Social benefits of participation exceeded the cost of investments for most interventions over time. Finally, providing Employment Assistance Services interventions earlier during an EI claim (first four weeks) produced larger impacts on earnings and employment and facilitated earlier return to work. This demonstrates the importance of targeting early participation of EI active claimants.

Overall, program participants have a higher probability of being employed than comparison group members

As shown in Figure 9, active EI claimants who participated in Skills Development, JobsNL and Job Creation Partnerships had a higher probability of being employed (that is, increased their incidence of employment) compared to similar non-participants. As well, former EI claimants who participated in Skills Development and JobsNL had a higher probability of being employed compared to former EI claimants who received low intensity interventions under Employment Assistance Services.

For active claimants, the incremental impacts demonstrate that participants in Employment Assistance Services improved their incidence of employment in the medium-term only. However, as demonstrated in the report, providing Employment Assistance Services earlier during the EI claim (first four weeks) generates significantly greater returns (see Figure 8), especially an earlier return to work compared to non-participants.

**Figure 9. Change in probability of being employed in participants relative to non-participants**

	Percentage Points
Active Claimants	Former Claimants
Skills Development	1.8	4.2
JobsNL	2.7	3.0
Job Creation Partnerships	2.0	2.9*
Employment Assistance Services	0.4*

*The estimates are not statistically significant at the 95% confidence level. For Employment Assistance Services, although the overall incremental impact was not statistically significant for the five-year post-program period, there were statistically significant impacts of 1.1 and 1.2 percentage points in years four and five following participation.

Note: The estimates in Figure 9 represent an arithmetic average of the annual incidence of employment estimates.

Increased earnings for active claimant participants compared to comparison group members

As shown in Figure 10, active EI claimants who participated in Skills Development, JobsNL and Employment Assistance Services increased their employment earnings compared to similar non-participants. For former EI claimants who participated in Skills Development, JobsNL and Job Creation Partnerships, the incremental impacts were not statistically significant. As such, former claimants did not increase their employment earnings compared to former EI claimants who received Employment Assistance Services exclusively.

**Figure 10. Increased cumulative earnings of participants compared to non-participants**

	Gains in Earnings
Active Claimants	Former Claimants
Skills Development	$16,644	$930*
JobsNL	$13,639	$5,823*
Job Creation Partnerships	$4,205*	-$7,855*
Employment Assistance Services	$12,531

*The estimates are not statistically significant at the 95% confidence level.

Skills Development and Employment Assistance Services improved the labour market attachment for most youth and older worker participants who were active EI claimants

Incremental impacts demonstrate that active EI claimants who were youth or older workers, and who participated in Skills Development and Employment Assistance Services increased their employment earnings compared to comparison group members (see Figures 11 and 12). With the exception of youth who participated in Employment Assistance Services, these participants increased their incidence of employment following participation.

**Figure 11. Cumulative increase in Employment Earnings for youth participants relative to youth non-participants**

	Cumulative Gains in Earnings
Active Claimants	Former Claimants
Skills Development	$12,977	$2,337*
JobsNL		$10,720*
Job Creation Partnerships	-$44*
Employment Assistance Services	$12,420

*The estimates are not statistically significant at the 95% confidence level.

**Figure 12. Cumulative increase in Employment Earnings for older workers participants relative to older workers non-participants**

Cumulative Gains in Earnings
Active Claimants
Skills Development	$13,630
Employment Assistance Services	$19,439

The use of EI is reduced for all active claimant participants. While EI use increased for some former claimants who participated in Skills Development and Job Creation Partnerships, it can be argued that this reflects an increase in labour market attachment for Skills Development participants since the use of social assistance decreased and that incremental impacts on employment earnings and incidence of employment are positive in the medium-term.

As shown in Figure 13, active EI claimants who participated in Skills Development, JobsNL and Job Creation Partnerships and Employment Assistance Services decreased their use of EI compared to non-participants. Active claimants who participated in Skills Development also decrease their use of social assistance benefits.

**Figure 13. Change in the cumulative use of Employment Insurance and Social Assistance for active claimants relative to non-participants**

	Cumulative use of EI and Social Assistance
Employment Insurance	Social Assistance
Skills Development	-$1,090	-$161
JobsNL	-$3,685	-$414*
Job Creation Partnerships	-$1,268	-$318*
Employment Assistance Services	-$1,715	$37*

*The estimates are not statistically significant at the 95% confidence level.

As shown in Figure 14, former claimants who participated in Skills Development, JobsNL and Job Creation Partnerships between 2002 and 2005 reduced the use of social assistance benefits following participation. As well, former claimants who participated in Skills Development and Job Creation Partnerships increased their use of EI following participation. For Job Creation Partnerships the incremental impacts show a statistically significant increase of 9.4 weeks of EI benefits following participation. The increase in EI use indicates the inability of some former claimants to maintain the employment secured in the short-term. For Skills Development participants only, it can also be argued that the increase in EI use is an indication of increased labour market attachment for this client group since they experienced increases in employment earnings and incidence of employment in the medium-term only as well as a decrease in the use of social assistance. As a reminder, former claimants are participants for whom the EI benefit period ended up to three years pre-participation.

**Figure 14. Change in cumulative use of Employment Insurance and Social Assistance for former claimants relative to non-participants**

	Cumulative Use of EI and Social Assistance
Employment Insurance	Social Assistance
Skills Development	-$1,356	-$1,240
JobsNL	-$776*	-$1,443
Job Creation Partnerships	$1,450*	-$1,563

*The estimates are not statistically significant at the 95% confidence level.

Social benefits of participation exceeded costs of investments for most interventions

As shown in Table 6, accumulated social benefits arising from participation exceeds the investment costs at a point in time between the second year of program participation to just over 17 years after participation. This excludes former claimants who participated in Job Creation Partnerships for whom the investment costs are not likely to be recouped. Job Creation Partnerships represented nearly 5% of new EBSM interventions in 2014 to 2015 in Newfoundland and Labrador.

**Table 6. Number of years for the benefits to exceed program costs**
	Skills Development	JobsNL	Job Creation Partnerships	Employment Assistance Services
Active claimants	6.0	2.5	15.7	2.1
Former claimants	17.4	2nd participation year^{Footnote 14}	Benefits may never recover the costs	n/a

4. Conclusions

Evaluation evidence presented and discussed in this report demonstrated that programs and services designed and delivered by Newfoundland and Labrador under the LMDA are generally helping participants improve their labour market experience after participation. As such, evaluation evidence suggests that LMDA funded programming contributes to achieve the following strategic priorities for the Department of Advanced Education, Skills and Labour: Alignment of labour market supply and demand and Support for people to participate in employment opportunities.

Specifically, the incremental impacts demonstrate that LMDA programs and services are improving the labour market attachment of active EI claimants, including youth and older workers who participated in Skills Development and Employment Assistance Services. For former claimants, participation in Skills Development led to an increase in labour market attachment. Social benefits of participation exceeded the cost of investments for most interventions over time. Finally, providing Employment Assistance Services interventions earlier during an EI claim (first four weeks) produced larger impacts on earnings and employment and facilitated earlier return to work. This demonstrates the importance of targeting early participation of EI active claimants.

Key informant interviews with service providers and program managers as well as the documents reviewed and the questionnaires completed by Newfoundland and Labrador government representatives also generated a few lessons about program design and delivery:

Skills Development

According to interviewed service providers and managers, the main challenges related to Skills Development design and delivery included:

Transportation is difficult for participants in rural areas
Lack of staff for the current caseload particularly in rural areas
Some prospective participants might have benefited from Skills Development but lack sufficient labour market attachment to qualify for EI and participate in the program
The Skills Development application process takes time to complete and there may be a waiting period before the start of training which creates financial barriers for participants when their EI benefits end
The perceived insufficient level of available financial support to address costs associated with tuition, relocation and childcare
The perceived insufficient level of available financial support to deal with the high demand for training

Skills Development – Apprentices supports

Existing literature showed that there is a fairly high non-completion rate among apprentices (40 to 50%)^{Footnote 15}. Furthermore, subject matter literature revealed that despite the growth in apprenticeship registrations in Canada, there has not been a corresponding increase in completing all required training and passing the certification exam^{Footnote 16}. It was not possible with available data to generate a reliable estimation of the completion rate of Skills Development-Apprentice Supports participants. According to key informants, apprentices do not complete their training due to factors such as:

Financial constraints
The requirement to travel in order to participate in block release training
The apprentice realises that he/she is not suited for the trade selected
Employers are not willing or able to let their apprentices go on training
Change in the economy such as an economic downturn and sudden lack of work

JobsNL

JobsNL is an employer-driven program that provides financial support employers for positions that promote sustainable long-term or seasonal employment.

Challenges with the design and delivery of JobsNL include:

Lack of staff expertise in dealing with all types of clients.
Lack of awareness of JobsNL among employers particularly in rural areas.
Lack of employers willing to participate in the program.
Lack of access to Records of Employment to help improve employer assessments and verify whether employers have recently laid off staff for similar positions.

Employment Assistance Services

According to key informants from 10 P/Ts, challenges with the design and delivery of Employment Assistance Services included:

Lack of awareness about Employment Assistance Services among potential participants
Current budget allocation is not enough to support the delivery of Employment Assistance Services and has led some service providers to eliminate services
Service providers cannot provide all the services needed for participants facing multiple barriers to employment. They have to refer these individuals to other organizations

5. Recommendations

A total of seven recommendations emerge from the evaluation findings. They are as follows:

The study on the timing of Employment Assistance Services participation showed that receiving assistance early after starting an employment insurance claim can lead to better labour market impacts.

Recommendation 2: Consideration should be given to removing barriers to accessing training by alleviating financial challenges and expanding eligibility requirements.

Recommendation 4: Consideration should be given to enhance the production of relevant labour market information to support service delivery.

Recommendation 7: Improvements in data collection is recommended to address key program and policy questions of interest to the federal and provincial/territorial governments. Specifically:

Collect data on whether participants are members of designated groups including Indigenous peoples, persons with disabilities and recent immigrants.
Collect data on the type of training funded under Skills Development and the type of assistance provided under Employment Assistance Services. EDSC, Newfoundland and Labrador, and other P/Ts should work together to define common categories for both EBSMs.

References

Canada Employment Insurance Commission. Employment Insurance Monitoring and Assessment Reports. 2003 to 2004 to 2013 to 2014.

Coe, Patrick. “Apprenticeship Programme Requirements and Apprenticeship Completion Rates in Canada.” Journal of Vocational Education and Training. 2013. 65(4): 575 to 605.

Employment and Social Development Canada. Analysis of Employment Benefits and Support Measures (EBSM) Profile, Outcomes and Medium-Term Incremental Impacts for 2002 to 2005 Participants in Newfoundland and Labrador: Technical Report. Evaluation Directorate. March 2014

Employment and Social Development Canada. Analysis of Employment Benefits and Support Measures (EBSM) Profile, Outcomes and Incremental Impacts for 2006 to 2008 Participants in Newfoundland and Labrador: Technical Report. Evaluation Directorate. February 11, 2015

Employment and Social Development Canada. Analysis of Employment Benefits and Support Measures (EBSM) Profile, Outcomes and Incremental Impacts for the EI Claimant Category: “Long-Tenured Workers” in Newfoundland and Labrador: Technical Report. Evaluation Directorate. January 21, 2016

Employment and Social Development Canada. Cost-Benefit Analysis of Employment Benefits and Support Measures delivered in Newfoundland and Labrador: Technical Report. Evaluation Directorate. November 30, 2015

Employment and Social Development Canada. Effects of the Timing of Participation in Employment Assistance Services in Newfoundland and Labrador: Technical Report. Evaluation Directorate. December 19, 2014

Employment and Social Development Canada. Study on JobsNL in Newfoundland and Labrador. Evaluation Directorate. January 21, 2016

Employment and Social Development Canada. Study on Skills Development-Apprentice Supports in Newfoundland and Labrador: Technical Report. Evaluation Directorate. June 28, 2016

Government of Newfoundland and Labrador. Job Creation Partnerships Program Guidelines. Department of Advanced Education and Skills. April 1, 2014

Government of Newfoundland and Labrador. Speech from the Throne. Author: March 8, 2016

LaRochelle-Côté, Sébastien and Sharanjit Uppal. "The Financial Well-Being of the Self-Employed," Perspectives on Labour and Income, vol. 23, no. 4, Winter 2011

Red Seal. Apprenticeship Completion, Certification and Outcomes. Ottawa: Red Seal, 2014.

Acronyms

EBSM Employment Benefits and Support Measures

EI Employment Insurance

ESDC Employment and Social Development Canada

LMDA Labour Market Development Agreements

P/T Provinces/Territories

Appendix A - Methodology

Qualitative data

Qualitative data reported in the Skills Development, Skills Development-Apprentice Supports and JobsNL studies were collected from key informant interviews with managers and service providers and a document/literature review. As well, questionnaires were completed by provincial/territorial government representatives for the Skills Development, Skills Development-Apprentice Supports and JobsNL studies. Table A1 provides the number of key informants interviewed in 2015.

**Table A1. Number of Key Informants Interviewed**
	Studies
	Skills Development	Skills Development-Apprentice Supports	JobsNL
Number of key informant interviews (Managers and Service Providers)	7	6	6

Quantitative methods

All quantitative analyses were conducted using linked administrative data from EI Part I (EI claim), EI Part II (EBSM participation data) and T1 and T4 taxation files on up to 100% of participants in Newfoundland and Labrador.

Incremental impacts

The incremental impact analysis compared the labour market experience of participants before and after their participation with that of a comparison group. The goal was to determine the direct effect of program participation on key labour market indicators (see Figure 1 in the introduction section).

For active claimants, incremental impacts were measured relative to a comparison group of active claimants who could have participated in the EBSMs but did not. Former claimants can be underemployed and unable to requalify for EI, out of the labour force for various reasons or on Social Assistance. Based on previous evaluation methodologies, on expert advice and given the difficulty in generating a suitable comparison for former claimants using administrative data alone, the comparison group for former claimants was created using individuals who participated in low-intensity Employment Assistance Services only during the reference period. This is a conservative approach given the fact that participation in Employment Assistance Services can lead to limited effects on labour market outcomes.

Participants and non-participants were matched based on a wide array of variables including age, sex, location, skill level required by the last occupation held prior to participation, reason for separation from employment, industry in which they were previously employed as well as employment earnings and use of EI and social assistance for each of the five years before participation.

All analyses were conducted using a unit of analysis called the Action Plan Equivalent, which combines all EBSMs given to an individual within no more than six months of each other. For reporting purposes, incremental impacts were attributed to the longest intervention of the Action Plan Equivalent when Skills Development, JobsNL, Job Creation Partnerships or Self-Employment Assistance was the longest intervention. Impacts for Employment Assistance Services were calculated for Action Plan Equivalents that contained only these services with no Employment Benefits.

The incremental impact estimates were produced using non-experimental methods, namely propensity score matching, using the Kernel Matching method, along with Difference-in-Differences method to estimate program impacts. Alternative matching techniques (i.e., Nearest Neighbour and Inverse Propensity Weighting) were also used for validation purposes.

Incremental impacts were measured for the following indicators:

Employment/self-employment earnings represent the total earnings an individual had from paid employment and/or self-employment. (This information is available by calendar year and was obtained from T1 and T4 tax return records.)
Incidence of employment/self-employment represents the incidence of having earnings from employment and/or self-employment.
Amount of EI benefits received represent the average amount of EI benefits received.
Weeks in receipt of EI benefits represents the average number of weeks during which EI benefits were received.
Social assistance benefits represent the average amount of social assistance benefits received. (This information is available by calendar year and was obtained from T1 tax return records.)
Dependence on income support represents the ratio of participant’s income that came from EI and social assistance benefits (that is, EI benefits + social assistance benefits) / (EI benefits + social assistance benefits + earnings from employment/self-employment).

Incremental impacts were estimated for different cohorts of participants:

All active and all former claimants as well as youth (under 30 years old) and older workers (55 years old and over) who started their EBSM participation between April 1, 2002 and March 31, 2005.
All active and all former claimants who stated their EBSM participation between January 1, 2006 and March 31, 2008.
Active and former claimants who were long-tenured workers and who started their EBSM participation between January 1, 2007 and December 31, 2009. Long-tenured workers covered in this study are individuals who established an EI regular or fishing benefit claim and who had paid at least 30% of the annual maximum employee EI premiums in seven of the ten years preceding their EI claim and who had collected 35 or fewer weeks of EI regular or fishing benefits in the five years preceding their claim. This definition is similar to the EI claimant category “long-tenured workers” introduced under Connecting Canadians with Available Jobs.

Cost-benefit analysis

The cost-benefit analysis compared how much it cost for individuals to participate in the programs and how much it costs the government to deliver those programs with the benefits both the participants and the government drew from those programs. The analysis was carried out from the society perspective which combines the costs and the benefits for both the participants and the government.

Costs and benefits included in the calculations were as follows:

Program costs included the administration cost and the direct cost of the EBSMs. The cost for each EBSM was calculated at the Action Plan Equivalent level. The costs were determined based on the average composition of the action plan equivalent.
The Marginal Social Cost of Public Funds represented the loss incurred by society when raising additional revenues such as taxes to fund government spending. The value was estimated as 20% of the program cost, sales taxes, income taxes, impacts on EI and impacts on social assistance paid or collected by the government.
Employment earnings consisted of incremental impacts on participants’ earnings during and after participation. The calculation accounts for the participant’s forgone earnings during participation (that is, opportunity cost). These are based on incremental impacts for the 2002 to 2005 participants.
Fringe benefits included benefits such as employer-paid health and life insurance as well as pension contributions. The rate used to calculate the fringe benefits was 15% of the incremental impact on earnings.

The program effects on EI and social assistance use, and the sales and income tax revenues were not included in the calculations since these costs and benefits cancel each other out from the social perspective by definition. For example, while EI and social assistance are benefits received by participants, they represent a cost for the government. However, as indicated above, these effects are accounted for in the calculation of the Marginal Social Cost of Public Funds.

When producing the results, to bring all costs and benefits to a common base and to account for inflation and interest on foregone government investment, the estimates for the second year of participation and up to the sixth year post-program were discounted by 5% per year. As well, when the benefits were still lower than the costs six years after program end, the payback period was calculated by assuming that the average benefit or cost measured over the fifth and six year post-program would persist over time (discounted at a 5% annual rate).

Strengths and limitations from the studies

Overall, the number of key informants interviewed was relatively small in some studies. The key informants’ responses were representative of their own experience and their own region but it is unclear if they were fully representative of the entire province.

The matching process led to the creation of comparison groups closely matched to the LMDA participants in terms of their background characteristics. Results obtained with Kernel Matching were validated with the use of two other techniques (that is, Inverse Propensity Weighting and Nearest Neighbour), increasing the level of confidence in the results. However, readers should be aware that incremental impacts may be affected by factors not captured by the matching process. For example, the motivation to seek employment was not directly measured except to the extent it was captured in prior income and labour market attachment patterns.

Readers should also keep in mind that it is not possible to compare the results obtained for each claimant type since the results for active claimants represent the effects of the EBSMs relative to non-participation while the results for former claimants represents the Employment Benefits relative to a limited treatment (that is, Employment Assistance Services).

The definition for long-tenured workers differs from the definition used in the literature as it does not consider the number of years the worker remained employed with the same employer.

The cost-benefit analysis was limited in the sense that it only took into account the quantifiable benefits and costs that were directly linked to EBSM delivery and participation and that could be estimated using available administrative data and the EI Monitoring and Assessment Report. The analysis did not capture “intangible”, non-pecuniary and indirect benefits. It did not consider the multiplier effect that improving participant’s income may have on the economy and did not account for the effect of EI Part II investment on sustaining a service delivery infrastructure and creating jobs among the governmental program service providers. As well, this analysis did not consider the displacement effect where participants may take away jobs that would otherwise be filled by other unemployed individuals. Finally, this analysis did not consider the possible effect of EBSMs on alleviating the increase in skills prices.

Appendix B - Detailed results Skills Development

**Table B1. Socio-demographic and labour market characteristics of Skills Development participants**
	Active claimants		Former claimants
	2002 to 2005	2006 to 2008	2002 to 2005	2006 to 2008
Number of observations	10,638	8,611	2,254	1,791
Gender
Male	63%	61%	52%	48%
Female	37%	39%	47%	52%
Age
Under 25	27%	23%	27%	26%
25 to 34	32%	24%	37%	32%
35 to 44	24%	24%	24%	23%
45 and over	18%	29%	13%	18%
Marital status
Married or common-law	49%	51%	41%	41%
Widow/ divorced or separated	6%	6%	10%	8%
Single	44%	42%	48%	49%
Missing data / Unknown	1%	1%	2%	2%
Skills level related to National Occupation Code associated with the last EI claim opened before Skills Development participation¹
Managerial	2%	2%	2%	3%
University	2%	1%	3%	2%
College or apprenticeship training	44%	39%	30%	30%
Secondary or occupational training	27%	30%	32%	35%
On-the-job training	25%	28%	33%	31%
Key Labour Market indicators in the year preceding the start of participation
Earnings²	$13,910	$13,497³	$7,228	$7,346³
Proportion employed	98%	99%	82%	80%
Proportion on Employment Insurance	78%	79%	83%	78%
Proportion on Social Assistance	4%	3%	16%	14%

Proportions may not add up to 100% due to rounding

¹Skill level corresponds to the type and/or amount of training or education typically required to work in the last occupation participants had before opening the last EI claim they had before participating in EBSMs:
-Managerial: Management occupations
-University: Occupations usually requiring university education that is, University degree at the bachelor's, master's or doctorate level)
-College or apprenticeship training: Occupations usually requiring college or vocational education or apprenticeship training such as two to three years of post-secondary education at a community college, institute of technology or CEGEP or two to five years of apprenticeship training or three to four years of secondary school and more than two years of on-the-job training, specialized training courses or specific work experience and/or occupations with supervisory responsibilities and occupations with significant health and safety responsibilities, such as firefighters, police officers and registered nursing assistants.
- Secondary or occupational training: Occupations usually requiring secondary school and/or occupation-specific training such as one to four years of secondary school education or up to two years of on-the-job training specialized training courses or specific work experience. -On-the-job training: On-the-job training is usually provided for occupations (that is, short work demonstration or on-the-job training or no formal educational requirements).

²Average earnings for all individuals included in the study. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2006 to 2008 participants have been adjusted by the Consumer Price Index published by Statistics Canada, using 2002 as the base year.

**Table B2. Incremental impacts for Skills Development – Active claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
All active claimants
2002 to 2005 participants (n=10,593)
Employment earnings ($)	-2,908***	-2,661***	303*	2,450***	3,969***	4,803***	5,119***	16,644***	11,075***
Incidence of employment (percentage points)	-8***	-8.6***	0	1.8***	2.5***	2.8***	2.1***	n/a	n/a
EI benefits ($)	1,516***	547***	-472***	-209*	-101*	-144**	-164**	-1,090***	972***
EI weeks	5.2***	1.9***	-2.2***	1.3***	-0.9***	-0.9***	-0.9***	-6.1***	0.9
SA benefits ($)	-16	-40***	-24	-34**	-41**	-36**	-26	-161**	-216**
Dependence on income support (percentage points)	14***	9.7***	-2.8***	-2.6***	-2.3***	-2***	-2***	n/a	n/a
2006 to 2008 participants (n=8,611)
Employment earnings ($)	-4,577***	-2,420***	1,820***	3,836***	6,477***	-	-	12,139***	5,148***
Incidence of employment (percentage points)	-9.8***	-9.0***	1.0**	2.5***	2.9***	-	-	n/a	n/a
EI benefits ($)	1,548***	-890***	-800***	-319***	-200***	-	-	-1,319***	-661***
EI weeks	4.2***	-3.4***	-3.4***	-1.8***	-1.4***	-	-	-6.6***	-5.9***
SA benefits ($)	-40***	-51***	-15	9	-13	-	-	-19	-110**
Dependence on income support (percentage points)	17.0***	1.3***	-5.5***	-2.3***	-2.1***	-	-	n/a	n/a
Sub-groups of active claimants
Youth (below 30 years old) – 2002 to 2005 participants (n=4,656)
Employment earnings ($)	-3,631***	-4,133***	-577**	1,534***	3,234***	4,451***	4,335***	12,977***	5,214**
Incidence of employment (percentage points)	-9.7***	-10***	-0.1	1.6**	1.9***	1.8***	0.8	n/a	n/a
EI benefits ($)	1,587***	632***	-596***	-212**	-142	-221**	-125	-1,296***	923***
EI weeks	6.3***	3.3***	-2.1***	-1***	0.8**	-1***	0.6**	-5.5***	4.2***
SA benefits ($)	10	-32*	-30	-20	-28	-13	3	-88	-110
Dependence on income support (percentage points)	17.1***	13.2***	-3.4***	-2.5***	-2.4***	-2.2***	-1.9***	n/a	n/a
Older workers (55 years and older (n=365)
Employment earnings ($)	-465	443	1,920**	3,624***	2,328***	2,467**	3,290***	13,630***	13,608***
Incidence of employment (percentage points)	0	3*	7.2***	9.7***	10.4***	9.6***	8.2***	n/a	n/a
EI benefits ($)	1,254***	1,161***	989***	1,072***	1,455***	1,207***	1,026***	5,750***	8,165***
EI weeks	3.3***	3***	2.9***	3.6***	4.9***	4.6***	4.2***	20.2***	26.6***
SA benefits ($)	-5	-11	48	22	-25	-8	15	52	36
Dependence on income support (percentage points)	1.2	1.7	0.7	0.1	2.9	4.3**	2.6	n/a	n/a
Long-tenured workers- 2007 to 2009 participants (n=773)
Employment earnings ($)	-7,847***	8,538***	166	5,275***	7,469***	-	-	12,909***	-3,476
Incidence of employment (percentage points)	-10.4***	-10.5***	1.1	5.8***	5.6***	-	-	n/a	n/a
EI benefits ($)	2,712***	1,151***	100	256	347	-	-	703	4,566***
EI weeks	6.3***	2.0***	-0.4	0.1	-0.1	-	-	-0.5	7.8
SA benefits ($)	10	7	51	78	71	-	-	200	217
Dependence on income support (percentage points)	18.2	8.7	-0.5	-1.1	-0.1	-	-	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table B3. Incremental impacts for Skills Development – Former claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
All former claimants
2002 to 2005 participants (n=2,242)
Employment earnings ($)	-3,385***	-4,261***	2,072***	-691	435	962	2,296***	930	-6,716**
Incidence of employment (percentage points)	-15.4***	-9.5***	0.9	4***	5.7***	5***	5.2***	n/a	n/a
EI benefits ($)	373***	-298***	-1	411***	306*	446***	195	1,356***	1,431**
EI weeks	1.3***	-1.8***	-0.4	1.2***	1.2***	1.6***	0.6	4.3**	3.7
SA benefits ($)	-260***	-576***	-292***	-226***	-224***	-253***	-209***	-1,204***	-2,040***
Dependence on income support (percentage points)	10.8***	-5.3***	-3.5***	-1.3	-1.8*	-1.1	-1.7	n/a	n/a
2006 to 2008 participants (n=1,791)
Employment earnings ($)	-5,391***	-5,928***	2,690***	-448	367	-	-	-2,774**	-14,099***
Incidence of employment (percentage points)	-19.2***	-13.5***	-0.9	3.0**	4.5***	-	-	n/a	n/a
EI benefits ($)	470***	-596***	-375***	35	287*	-	-	-53	-179
EI weeks	2.1***	-2.1***	-1.4***	-0.1	0.7	-	-	-0.7	-0.7
SA benefits ($)	-29	-375***	-225***	-200***	-154**	-	-	-576***	-977***
Dependence on income support (percentage points)	13.9***	-3.4***	-5.0***	-1.9*	-1.2	-	-	n/a	n/a
Sub-groups of former claimants
Youth (below 30 years old) – 2002 to 2005 participants (n=1,047)
Employment earnings ($)	-3,953***	-4,686***	2,022***	58	1,060	1,012	2,229*	2,337	-6,302
Incidence of employment (percentage points)	-12.9***	-6.9***	2.8	3.6**	5.9***	5.1***	6.8***	n/a	n/a
EI benefits ($)	323***	-164	-187	359*	39	420*	278	909	1,068
EI weeks	2.3***	-0.2	0.2	1.8***	0.2*	2.1***	1.3**	6.6***	8.7***
SA benefits ($)	-183*	-618***	-278***	-120	-63	-127	-66	-653	-1,453**
Dependence on income support (percentage points)	13.9***	-4***	-3.4***	-0	-0.6	0.9	-0.8	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table B4. Cost-benefit results from the social perspective for Skills Development**
Total costs and benefits over participation (1 to 2 years) and 6 years post-program	Active claimants (n=10,593)	Former claimants (n=2,242)
Program cost	-$11,797	-$12,307
Marginal social costs of public funds	-$1,814	-$2,464
Employment earnings (including participant’s forgone earnings)	$11,889	-$4,706
Fringe benefit	$1,783	-$706
Net present value (By how much do the benefits exceed the costs 6 years after participation?)	$62	-$20,184
Cost-benefit ratio (How much does it cost in EI part II funds to achieve $1 in benefit 6 years after participation?)	$1.00	Negative benefits
Payback period (How many years after participation would it take for the benefits to recover the costs?)	6.0 years after participation	17.4 years after participation

Appendix C - Detailed results Skills Development-Apprentice Supports

**Table C1. Socio-demographic and labour market characteristics of Skills Development-Apprentice Supports participants**
	Active claimants		Former claimants
	2003 to 2005	2013 to 2014	2003 to 2005	2013 to 2014
Number of observations	1,076	589	186	90
Gender
Male	97%	92%	63%	77%
Female	3%	8%	37%	20%
Age
Under 25	22%	15%	12%	20%
25 to 34	26%	47%	21%	46%
35 to 44	25%	21%	24%	16%
45 to 54	19%	11%	26%	12%
55 and over	8%	5%	16%	7%
Skills level related to National Occupation Code associated with the last EI claim opened before Skills Development-Apprentices Supports participation¹
Managerial	0%	0%	1%	1%
University	1%	0%	2%	0%
College or apprenticeship training	80%	91%	41%	56%
Secondary or occupational training	9%	5%	18%	24%
On-the-job training	10%	3%	38%	19%
Key Labour Market Indicators in the year preceding the start of participation
Earnings²	$24,791	$32,854³	$10,087	$22,206³
Proportion employed	100%	100%	90%	93%
Proportion on Employment Insurance	77%	61%	81%	55%
Proportion on Social Assistance	1%	1%	4%	1%

Proportions may not add up to 100% due to rounding

²Average earnings for all individuals included in the study. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2013 to 2014 participants have been adjusted according to the Consumer Price Index (CPI), published by Statistics Canada, to the 2002 base year.

**Table C2. Labour market outcomes for active claimants who started Skills Development-Apprentice Supports in 2003 to 2005**
Average outcomes	Pre-program period					After the program start year
Average outcomes	5 year pre	4 year pre	3 year pre	2 year pre	1 year pre	Program start year	1 year	2 year	3 year	4 year	5 year	6 year	7 year
Earnings including $0	$18,127	$22,190	$22,823	$24,362	$26,295	$26,375	$37,536	$44,163	$54,148	$59,051	$55,975	$59,904	$63,362
Earnings excluding $0¹	$18,952	$23,092	$23,692	$24,804	$26,377	$26,766	$38,258	$45,355	$55,922	$61,652	$59,375	$64,163	$67,972
Proportion employed	96%	96%	96%	98%	100%	99%	98%	97%	97%	96%	94%	93%	93%
Proportion on EI	65%	70%	76%	78%	82%	100%	85%	78%	67%	62%	61%	57%	55%
EI benefits	$5,001	$5,153	$5,964	$6,175	$6,849	$8,591	$6,693	$6,118	$5,345	$4,989	$5,571	$5,420	$4,858
Number of weeks on EI	15.01	15.42	17.21	17.17	18.64	23.79	17.87	15.79	13.21	12.01	12.93	12.13	10.09
Proportion on SA	3%	2%	1%	1%	1%	1%	1%	1%	0%	1%	1%	1%	1%
SA benefits	$114	$71	$42	$54	$48	$34	$12	$13	$19	$15	$69	$51	$57
Dependence on income support	23%	22%	24%	22%	23%	30%	21%	17%	14%	13%	15%	14%	12%
Proportion self employed	15%	17%	17%	18%	19%	25%	28%	30%	34%	36%	35%	33%	38%

N= 959. Data excludes individuals with no Canada Revenue Agency data for the five years before participation.

¹Earnings outcomes excluding individuals who reported no earnings in a given year.

SA: Social Assistance

**Table C3. Labour market outcomes for former claimants who started Skills Development-Apprentice Supports in 2003 to 2005**
Average outcomes	Pre-program period					After the program start year
Average outcomes	5 year pre	4 year pre	3 year pre	2 year pre	1 year pre	Program start year	1 year	2 year	3 year	4 year	5 year	6 year	7 year
Earnings including $0	$10,544	$14,299	$14,591	$13,556	$10,129	$12,572	$18,750	$22,217	$24,478	$26,478	$25,591	$25,278	$27,131
Earnings excluding $0¹	$10,920	$14,719	$14,844	$14,548	$11,283	$13,416	$19,654	$23,725	$27,497	$30,112	$30,003	$31,226	$33,972
Proportion employed	97%	97%	98%	93%	90%	94%	95%	94%	89%	88%	85%	81%	80%
Proportion on EI	72%	74%	84%	85%	81%	71%	82%	82%	76%	73%	72%	70%	62%
EI benefits	$4,386	$4,383	$5,916	$6,982	$6,830	$4,534	$6,084	$6,738	$6,676	$6,308	$7,135	$7,014	$5,903
Number of weeks on EI	19.12	19.77	22.98	24.49	23.87	16.83	21.22	23.35	22.9	19.93	21.15	19.41	15.52
Proportion on SA	4%	2%	2%	5%	4%	6%	6%	2%	2%	3%	3%	4%	3%
SA benefits	$117	$67	$80	$221	$188	$280	$273	$227	$186	$97	$220	$243	$165
Dependence on income support	33%	30%	35%	41%	45%	35%	36%	38%	34%	30%	34%	34%	28%
Proportion self employed	10%	13%	11%	11%	10%	13%	16%	20%	17%	18%	19%	19%	17%

N= 176 Data exclude individuals with no Canada Revenue Agency data for the 5 years before participation

¹Earnings outcomes excluding individuals who reported no earnings in a given year.

SA: Social Assistance

Appendix D - Detailed results JobsNL

**Table D1. Socio-demographic and labour market characteristics of JobsNL participants**

	Active claimants		Former claimants
	2002 to 2005	2006 to 2008	2002 to 2005	2006 to 2008
Number of observations	834	728	1,224	688
Gender
Male	54%	42%	52%	41%
Female	46%	58%	48%	59%
Age
Under 25	11%	11%	11%	13%
25 to 34	33%	27%	35%	33%
35 to 44	30%	28%	30%	26%
45 and over	24%	34%	24%	27%
Marital status
Married or common-law	58%	57%	55%	51%
Widow / divorced or separated	9%	10%	11%	9%
Single	31%	31%	33%	37%
Missing data / Unknown	1%	2%	1%	3%
Skills level related to National Occupation Code associated with the last EI claim opened before JobsNL participation¹
Managerial	3%	5%	6%	5%
University	4%	3%	6%	6%
College or apprenticeship training	27%	26%	29%	31%
Secondary or occupational training	33%	40%	35%	37%
On-the-job training	32%	25%	24%	22%
Key Labour Market Indicators in the year preceding the start of participation
Earnings²	$12,901	$13,437³	$10,497	$8,031³
Proportion Employed	98%	99%	89%	82%
Proportion on Employment Insurance	76%	75%	79%	76%
Proportion on Social Assistance	5%	3%	7%	8%

Proportions may not add up to 100% due to rounding

²Average earnings for all individuals included in the study. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2006 to 2008 participants have been adjusted by the Consumer Price Index published by Statistics Canada, using 2002 as the base year.

**Table D2. Incremental impacts for JobsNL – Active claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Active claimants
2002 to 2005 Participants (n=832)
Employment earnings ($)	944***	2,743***	2,189***	2,177***	3,462	3,191***	2,620***	13,639***	17,327***
Incidence of employment (percentage points)	3.3***	4.3***	2.6***	1.2	3.3***	2.4*	4***	n/a	n/a
EI benefits ($)	-671**	-1,202***	-969***	-772***	-695***	-596***	-654***	-3,685***	-5,558***
EI weeks	-2.2***	-3.2***	-2.2***	-2***	-1.9***	-1.7***	-1.9***	-9.7***	-15.1***
SA benefits ($)	-30	-89*	-88*	-72	-88*	-91*	-75	-414*	-533*
Dependence on income support (percentage points)	-5.4*	-7.2***	-4.1***	-3.5***	-5***	-3.4***	-4.2***	n/a	n/a
2006 to 2008 participants (n=728)
Employment earnings ($)	-114***	4,134***	4,229***	3,331***	3,234***	-	-	10,795***	14,815***
Incidence of employment (percentage points)	3.4***	5.1***	6.5***	5.3***	4.9***	-	-	n/a	n/a
EI benefits ($)	-1,389***	-2,446***	-1,677***	-1,229***	-1,055***	-	-	-3,961***	-7,796***
EI weeks	-3.7***	-7.0***	-4.4***	-3.1***	-2.5***	-	-	-9.9***	-20.5***
SA benefits ($)	-72**	-100**	-126**	-113**	-156***	-	-	-395***	-566**
Dependence on income support (percentage points)	-7.3***	-13.8***	-8.8***	-5.8***	-5.5***	-	-	n/a	n/a

Significance level *** 1%; ** 5%; * 10%

SA: Social Assistance

**Table D3. Incremental impacts for JobsNL – Former claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Former claimants
2002 to 2005 participants (n=1,218)
Employment earnings ($)	5,662***	6,229***	2,398***	1,006	1,484	724	212	5,823	17,714***
Incidence of employment (percentage points)	15***	10.7***	4.7***	4***	3.2***	0.7	2.2	n/a	n/a
EI benefits ($)	-171	202	253	-34	-341*	-203	-451**	-776	-745
EI weeks	-0.7	0.7	0.7	0	-1.1*	-0.8	-1.4**	-2.6	-2.6
SA benefits ($)	-427***	-549***	-369***	-292***	-231***	-266***	-285**	-1,443***	-2,418***
Dependence on income support (percentage points)	-15.4***	-7.7***	-1.7	-2.3*	-3.9***	-2.6**	-4.2***	n/a	n/a
2006 to 2008 participants (n=688)
Employment earnings ($)	1,251**	2,138***	426	1,737**	1,656**	-	-	3,818	7,209**
Incidence of employment (percentage points)	13.7***	11.4***	5.8***	5.7***	4.3**	-	-	n/a	n/a
EI benefits ($)	260	460**	248	-422**	-140	-	-	-314	405
EI weeks	1.2	2.5***	1.9***	-0.2	0.3	-	-	2.1	5.8**
SA benefits ($)	-297***	-478***	-395***	-427***	-369***	-	-	-1,189***	-1,963***
Dependence on income support (percentage points)	-6.9***	-2.5	-0.1	-5.3***	-3.9***	-	-	n/a	n/a
Sub-groups of former claimants
Youth (below 30 years old) – 2002 to 2005 participants (n=389)
Employment earnings ($)	5,064***	6,880***	3,354***	1,697	2,370	2,862	438	10,720	22,665***
Incidence of employment (percentage points)	13.1***	9***	6.4***	2	4.4*	-0.8	2.2	n/a	n/a
EI benefits ($)	680***	340	133	89	-507*	-634**	-426	-1,344	-325
EI weeks	1.7**	1.5*	0.4	0.3	-1.6*	-2.1**	-1.7*	-4.5	-1.3
SA benefits ($)	-475***	-654***	-453***	-401***	-286**	-237	-295**	-1,672***	-2,801***
Dependence on income support (percentage points)	-13.6***	-9.9***	-4.8***	-3.3*	-5.2***	-5**	-6.1***	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table D4. Cost-benefit results from the social perspective for JobsNL**
Total costs and benefits over participation (1 to 2 years) and 6 years post-program	Active claimants (n=832)	Former claimants (n=1,218)
Program cost	-$9,577	-$7,579
Marginal social costs of public funds	-$21	-$1,402
Employment earnings (including participant’s forgone earnings)	$17,510	$17,116
Fringe benefit	$2,627	$2,567
Net present value (By how much do the benefits exceed the costs 6 years after participation?)	$10,581	$13,506
Cost-benefit ratio (How much does it cost in EI part II funds to achieve $1 in benefit 6 years after participation?)	$0.50	$0.40
Payback period (How many years after participation would it take for the benefits to recover the costs?)	2.5 years after participation	2nd year after participation

Appendix E - Detailed results Self-Employment Assistance

**Table E1. Socio-demographic and labour market characteristics of Self-Employment Assistance participants**
	Active claimants		Former claimants
	2002 to 2005	2006 to 2008	2002 to 2005	2006 to 2008
Number of observations	467	314	241	140
Gender
Male	54%	47%	56%	44%
Female	46%	53%	44%	56%
Age
Under 25	5%	4%	3%	3%
25 to 34	33%	32%	27%	37%
35 to 44	35%	40%	39%	33%
45 and over	28%	24%	29%	27%
Marital status
Married or common-law	71%	66%	66%	61%
Widow/ divorced or separated	8%	10%	9%	12%
Single	21%	24%	23%	22%
Missing data / Unknown	1%	1%	2%	4%
Skills level related to National Occupation Code associated with the last EI claim opened before Self-Employment Assistance participation¹
Managerial	9%	9%	8%	10%
University	9%	7%	15%	9%
College or apprenticeship training	36%	37%	32%	34%
Secondary or occupational training	30%	30%	27%	29%
On-the-job training	16%	18%	17%	18%
Key Labour Market Indicators in the year preceding the start of participation
Earnings²	$18,191	$15,084³	$10,227	$8.500³
Proportion Employed	99%	99%	84%	83%
Proportion on Employment Insurance	70%	77%	91%	81%
Proportion on Social Assistance	2%	2%	6%	3%

Proportions may not add up to 100% due to rounding

²Average earnings for all individuals included in the study. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2006 to 2008 participants have been adjusted by the Consumer Price Index published by Statistics Canada, using 2002 as the base year.

**Table E2. Incremental impacts for Self-Employment Assistance –Active claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Active claimants
2002 to 2005 participants (n=462)
Employment earnings ($)	-8,962***	-9,978***	-6,713***	-4,435***	-3,062***	-2,163*	-1,617	-17,990***	-36,930***
Incidence of employment (percentage points)	-20.8***	-33.1***	-20.6***	-14.7***	-12.1***	-7.9***	-8.9***	n/a	n/a
EI benefits ($)	3,832***	661**	-2,717***	-2,299***	-1,517***	-1,449***	-1,125***	-9,108***	-4,615***
EI weeks	9.1***	0.3	-9.4***	-7.7***	-4.9***	-4.4***	-3.5***	-29.9***	-20.4***
SA benefits ($)	9	-53	-38	-37	-40	-44	-38	-196	-241
Dependence on income support (percentage points)	33.9***	22***	-11***	-8.5***	-5.8***	-5.1***	-3**	n/a	n/a
2006 to 2008 participants (n=314)
Employment earnings ($)	-8,809***	-11,701***	-8,102***	-5,771***	-5,647***	-	-	-19,520***	-40,030***
Incidence of employment (percentage points)	-22.9***	-35.6***	-19.9***	-18.3***	-14.8***	-	-	n/a	n/a
EI benefits ($)	1,825***	-2,919***	-3,860***	-2,951***	-2,268***	-	-	-9,079***	-10,172***
EI weeks	3.6***	-9.8***	-12.0***	-8.9***	-7.0***	-	-	-27.9***	-34.0***
SA benefits ($)	-29	-120**	-78	-52	-64	-	-	-193	-342
Dependence on income support (percentage points)	31.0***	1.1	-14.6***	-11.4***	-7.8***	-	-	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

Appendix F - Detailed results Job Creation Partnerships

**Table F1. Socio-demographic and labour market characteristics of Job Creation Partnerships participants**
	Active claimants		Former claimants
	2002 to 2005	2006 to 2008	2002 to 2005	2006 to 2008
Number of observations	1,388	1,346	972	1,341
Gender
Male	56%	38%	66%	47%
Female	44%	62%	34%	53%
Age
Under 25	13%	10%	13%	8%
25 to 34	25%	17%	25%	21%
35 to 44	27%	24%	27%	25%
45 and over	34%	48%	34%	45%
Marital status
Married or common-law	61%	62%	56%	58%
Widow/ divorced or separated	8%	8%	7%	9%
Single	30%	29%	35%	31%
Missing data / Unknown	1%	1%	1%	3%
Skills level related to National Occupation Code associated with the last EI claim opened before Job Creation Partnerships participation¹
Managerial	2%	3%	2%	2%
University	4%	5%	5%	4%
College or apprenticeship training	19%	15%	27%	25%
Secondary or occupational training	32%	33%	28%	29%
On-the-job training	42%	45%	38%	39%
Key Labour Market Indicators in the year preceding the start of participation
Earnings²	$7,795	$6,460³	$5,750	$4,580³
Proportion Employed	96%	98%	81%	76%
Proportion on Employment Insurance	75%	80%	84%	77%
Proportion on Social Assistance	10%	6%	28%	21%

Proportions may not add up to 100% due to rounding

²Average earnings for all individuals included in the study. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2006 to 2008 participants have been adjusted by the Consumer Price Index published by Statistics Canada, using 2002 as the base year.

**Table F2. Incremental impacts for Job Creation Partnerships – Active claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Active claimants
2002 to 2005 participants (n=1,387)
Employment earnings ($)	-2,744***	-984**	-51	512	966*	1,397**	1,381**	4,205*	476
Incidence of employment (percentage points)	n/a	n/a	0.5	1.9*	2.6**	2.3*	2.5**	n/a	n/a
EI benefits ($)	1,985***	-468**	-513***	-250*	-183	-194	-128	-1,268**	249
EI weeks	5.9***	-1.1**	-0.5	0.2	0.4	0.2	0.4	0.7	5.5**
SA benefits ($)	-79**	-17	-17	-26	-59	-81	-135**	-318	-414
Dependence on income support (percentage points)	17.2***	0.7	-1.3	-0.9	-0.6*	-2.6***	-2.5***	n/a	n/a
2006 to 2008 participants (n=1,346)
Employment earnings ($)	-2,869***	-927***	-300	-678	-837*	-	-	-1,815	-5,610***
Incidence of employment (percentage points)	0.4	0.8	3.2***	1.6	2.3**	-	-	n/a	n/a
EI benefits ($)	-391***	-1,723***	-714***	-445***	-116	-	-	1,275***	3,389***
EI weeks	2.6***	-4.4***	-1.0**	-0.2	0.9**	-	-	-0.2	-2.1
SA benefits ($)	-99	-46	-29	-5	-29	-	-	-64	-209
Dependence on income support (percentage points)	9.7***	-6.1***	-1.3	0.2	1.0	-	-	n/a	n/a
Sub-groups of active claimants
Youth (below 30 years old) – 2002 to 2005 participants (n=349)
Employment earnings ($)	-3,400***	-1,386**	-468	-488	186	406	321	-44	-4,829
Incidence of employment (percentage points)	n/a	n/a	0.6	1.5	1.4	4.5**	1.1	n/a	n/a
EI benefits ($)	2,109***	-334	-762***	-415	-562**	-647**	-438	-2,824***	-1,048
EI weeks	4.3***	-2**	-2.2**	-1.6*	-1.9**	-2**	-1.2	-8.9**	-6.6
SA benefits ($)	-36	35	13	19	-51	-101	-103*	-222	-223
Dependence on income support (percentage points)	16.9***	0.5	-2.5	-2.2	-3.1	-4.2**	-4.3**	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table F3. Incremental impacts for Job Creation Partnerships –Former claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Former claimants
2002 to 2005 participants (n=962)
Employment earnings ($)	-2,944***	-2,458***	-2,103***	-2,214***	-1,606*	-1,468	-464	-7,855*	-13,258***
Incidence of employment (percentage points)	n/a	n/a	3*	3.3*	3	2.4	2.6	n/a	n/a
EI benefits ($)	-284*	-439***	43	517***	342*	178	370*	1,450*	727
EI weeks	-1.2*	-1.1	1.2*	3***	2.1***	1.5**	1.6**	9.4***	7.1*
SA benefits ($)	-492***	-504***	-278***	-280***	-309***	-326***	-369***	-1,563***	-2,558***
Dependence on income support (percentage points)	1	-6.4***	-1.5	0.6	0.1	-0.7	-0.7	n/a	n/a
2006 to 2008 participants (n=1,341)
Employment earnings ($)	-4,400***	-4,309***	-4,119***	-3,436***	-4,051***	-	-	-11,611***	-20,324***
Incidence of employment (percentage points)	-10.2***	-1.4	1.1	0.6	0.5	-	-	n/a	n/a
EI benefits ($)	-321**	-452***	-107	62	268	-	-	223	-550
EI weeks	-1.1**	-0.6	1.4**	1.2**	2.0***	-	-	4.6***	2.9
SA benefits ($)	-230***	-151**	-145**	-105	-65	-	-	-311	-689**
Dependence on income support (percentage points)	2.6**	-3.6***	-0.7	0.1	1.6	-	-	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table F4. Cost-benefit results from the social perspective for Job Creation Partnerships**
Total costs and benefits over participation (1 to 2 years) and 6 years post-program	Active claimants (n=1,387)	Former claimants (n=962)
Program cost	-$7,013	-$6,741
Marginal social costs of public funds	-$1,194	-$1,869
Employment earnings (including participant’s forgone earnings)	$240	-$15,247
Fringe benefit	$36	-$2,287
Net present value (By how much do the benefits exceed the costs within 6 years after participation?)	-$7,931	-$26,144
Cost-benefit ratio (How much does it cost in EI part II funds to achieve $1 in benefit 6 years after participation?)	Negative benefits	Negative benefits
Payback period (How many years after participation would it take for the benefits to recover the costs?)	15.7 years after participation	Benefits may not recover the costs

Appendix G - Detailed results Employment Assistance Services

**Table G1. Socio-demographic and labour market characteristics of Employment Assistance Services participants**
	Active claimants		Former claimants
	2002 to 2005	2006 to 2008	2002 to 2005	2006 to 2008
Number of observations	7,444	11,904	3,870	5,330
Gender
Male	64%	58%	56%	49%
Female	36%	42%	44%	51%
Age
Under 25	21%	15%	22%	17%
25 to 34	33%	24%	33%	30%
35 to 44	25%	26%	24%	24%
45 and over	20%	35%	20%	28%
Marital status
Married or common-law	48%	54%	40%	41%
Widow/ divorced or separated	8%	8%	11%	10%
Single	43%	37%	46%	45%
Missing data / Unknown	2%	2%	3%	4%
Skills level related to National Occupation Code associated with the last EI claim opened before Employment Assistance Services participation1
Managerial	2%	2%	2%	3%
University	4%	2%	4%	3%
College or apprenticeship training	34%	36%	26%	29%
Secondary or occupational training	31%	32%	34%	34%
On-the-job training	28%	27%	34%	31%
Key Labour Market Indicators in the year preceding the start of participation
Earnings2	$15,099	$14,0773	$8,152	$9,2893
Proportion Employed	98%	99%	81%	83%
Proportion on Employment Insurance	74%	80%	75%	70%
Proportion on Social Assistance	6%	4%	19%	13%

Proportions may not add up to 100% due to rounding

²Average earnings for all individuals included in the study. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2006 to 2008 participants have been adjusted by the Consumer Price Index published by Statistics Canada, using 2002 as the base year.

**Table G2. Incremental impacts for Employment Assistance Services**
Indicators	In-program period	Post-program period						Total in- and post-program
Indicators	In-program period	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Active claimants
2002 to 2005 Participants (n=7,444)
Employment earnings ($)	-1,374***	518***	1,495***	2,703***	3,639***	4,176***	12,531***	11,157***
Incidence of employment (percentage points)	-1.2***	-0.5	-0.2	0.6	1.1**	1.2**	n/a	n/a
EI benefits ($)	570***	-393***	-359***	-332**	-318***	-313***	-1,715***	-1,144***
EI weeks	2.4***	-1.2***	-1***	-0.9***	-0.9***	-0.9***	-4.9***	-2.4**
SA benefits ($)	23	64***	21	-24	-21	-4	37	60
Dependence on income support (percentage points)	6.4***	-0.4	-0.9**	-1.5***	-1.6***	-1.3***	n/a	n/a
2006 to 2008 Participants (n=11,904)
Employment earnings ($)	-1,563***	927***	2,158**	2,503***	-	-	5,588***	4,025***
Incidence of employment (percentage points)	0.3	2.0***	2.2***	2.1***	-	-	n/a	n/a
EI benefits ($)	670**	-375***	-176***	-128**	-	-	-679***	-9
EI weeks	2.5***	-0.9***	-0.3**	-0.1	-	-	-1.3***	1.1**
SA benefits ($)	-54***	27*	25	16	-	-	68	14
Dependence on income support (percentage points)	0.1***	0.0***	0.0***	0.0	-	-	n/a	n/a
Sub-groups of active claimants
Youth (below 30 years old) – 2002 to 2005 participants (n=2,891)
Employment earnings ($)	-961***	643*	1,399***	2,645***	3,779***	3,952***	12,420***	11,459***
Incidence of employment (percentage points)	-1.2	-1.7**	-0.9	0.2	0.2	0.5	n/a	n/a
EI benefits ($)	487***	-344***	-390***	-462***	-416***	-408***	-2,019***	-1,532***
EI weeks	2.2***	-1***	-1**	-1.2***	-1***	-1**	-5.2***	-3*
SA benefits ($)	102***	118***	61*	20	9	68*	277*	379**
Dependence on income support (percentage points)	5.8***	0.3	-0.7	-1.6**	-1.5*	-0.6	n/a	n/a
Older workers (55 years old and over) – 2002 to 2005 participants (n=307)
Employment earnings ($)	-422	1,972*	3,285***	4,886***	4,718***	4,577***	19,439***	19,017***
Incidence of employment (percentage points)	0	4.8***	3.5	5.5***	8.5***	7.5***	n/a	n/a
EI benefits ($)	843***	160	329	594*	736**	970***	2,790**	3,633***
EI weeks	1.8**	0.8	1.1	1.9*	2.2**	2.6**	8.5**	10.3**
SA benefits ($)	-70	2	38	8	41	-4	85	15
Dependence on income support (percentage points)	2.7	-1	1.7	1.8	3	4.4**	n/a	n/a
“Long-Tenured workers”- 2007 to 2009 participants (n=853)
Employment earnings ($)	-2,694***	1,619*	2,871***	3,988***	-	-	8,479***	5,785*
Incidence of employment (percentage points)	-0.1	4.8***	3.7***	3.1**	-	-	n/a	n/a
EI benefits ($)	672***	-635***	-274	-172	-	-	-1,081**	-409
EI weeks	1.7***	-1.7***	-0.7	-0.7	-	-	-3.1**	-1.4
SA benefits ($)	13	125***	107**	94**	-	-	326***	340***
Dependence on income support (percentage points)	3.1**	-5.4***	-0.6	-1.6	-	-	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table G3. Incremental impacts related to the timing of participation in Employment Assistance Services**
Cohorts (start of EAS-only after start of an EI claim)	n=	In-program	Post-program period					Total impact post-program	Total impact in- and post- program
Cohorts (start of EAS-only after start of an EI claim)	n=	In-program	1 year	2 years	3 years	4 years	5 years	Total impact post-program	Total impact in- and post- program
Employment earnings ($)
1 to 4 weeks	929	1,015***	1,698***	4,158***	5,953***	8,558***	8,872***	29,238***	30,253***
5 to 8 weeks	735	45	2,661***	4,867***	6,646***	7,071***	8,123***	29,368***	29,413***
9 to 12 weeks	708	-1,410***	1,539***	4,201***	6,518***	8,144***	7,673***	28,074***	26,664***
2nd quarter	1,899	-1,388***	1,369***	2,459***	4,102***	5,220***	6,067***	19,217***	17,829***
3rd quarter	1,244	-3,150***	934***	1,990***	3,066***	3,999***	4,546***	14,535***	11,384***
4th quarter	687	-4,252***	1,451***	2,929***	5,175***	6,323***	7,067***	22,945***	18,694***
Incidence of employment (percentage points)
1 to 4 weeks	929	2.9***	1.7	1.7	2.9**	2.9**	4.8***	n/a	n/a
5 to 8 weeks	735	2.5***	1.3	2.1*	2.4*	0.1	1.6	n/a	n/a
9 to 12 weeks	708	2.7**	3.8***	4.3***	5.0***	6.1***	5.1***	n/a	n/a
2nd quarter	1,899	-0.4	-0.3	1.1	1.9**	2.5***	1.6*	n/a	n/a
3rd quarter	1,244	-2.5***	-1.1	-1.4	-0.4	-0.3	-1.4	n/a	n/a
4th quarter	687	-6.9***	-3.2***	-1.9	0.0	1.7	2.8*	n/a	n/a
EI Benefits ($)
1 to 4 weeks	929	494***	149*	-165	15	-263	54	-210	284
5 to 8 weeks	735	805***	-296*	-453**	-610***	-289	-360	-2,009***	-1,204
9 to 12 weeks	708	1,263***	-184	-185	-191	-70	18	-613	650
2nd quarter	1,899	1,448***	-323***	-245**	-187*	70	-52	-737	711
3rd quarter	1,244	967***	-948***	-611***	-451***	-266**	-396**	-2,671***	-1,704**
4th quarter	687	1,877***	-1,040***	-541***	-484***	-368*	30	-2,402***	-526

* Significant at 10%; ** significant at 5%; *** significant at 1%

**Table G4. Incremental time of return to Employment for Employment Assistance Services participants based on timing of participation**
Cohorts (start of EAS-only after start of an EI claim)	1 to 4 weeks	5 to 8 weeks	9 to 12 week	2nd quarter	3rd quarter	4th quarter
Cohorts (start of EAS-only after start of an EI claim)	(N=929)	(N=735)	(N=708)	(N=1,899)	(N=1,244)	(N=687)
Time of Return to Employment (in weeks)	1.6***	-0.6	1.7***	-3.1***	-3.4***	-4.8***

* Significant at 10%; ** significant at 5%; *** significant at 1%

**Table G5. Cost-benefit results from the social perspective for Employment Assistance Services**
Total costs and benefits over participation (1 to 2 years) and 6 years post-program	Active claimants (n=7,444)
Program cost	-$741
Marginal social costs of public funds	$796
Employment earnings (including participant’s forgone earnings)	$13,724
Fringe benefit	$2,059
Net present value (By how much do the benefits exceed the costs within 6 years after participation?)	$15,837
Cost-benefit ratio (How much does it cost in EI part II funds to achieve $1 in benefit 6 years after participation?)	$0.00
Payback period (How many years after participation would it take for the benefits to recover the costs?)	2.1 years after participation

Appendix H. Overview of studies included in this report

**Table H1. Overview of studies included in this report**
Study	Evidence included in this summary report	Methods	Reference period	Observation period
Profile, Outcomes and Net Impacts of Employment Benefits and Support Measures (EBSM) Participants in Newfoundland and Labrador (Completed in 2014)	Incremental impacts for participants including youth and older workers Profile and socio-demographic characteristics of participants	Non-experimental method using propensity score matching in combination with Difference-in-Differences Statistical profiling	2002 to 2005 participants	7 years between 2002 and 2011 (that is, 2 years in program and 5 years post-program)
Effects of the Timing of Participation in Employment Assistance Services in Newfoundland and Labrador (Completed in 2014)	Incremental impacts	Non-experimental method using propensity score matching in combination with Difference-in-Differences Statistical profiling
Cost-Benefit Analysis of Employment Benefits and Support Measures delivered in Newfoundland and Labrador (Completed in 2016)	Cost-benefit analysis	Non-experimental method using propensity score matching in combination with Difference-in-Differences Statistical profiling		8 years between 2002 and 2013 (that is, 2 years in-program and 6 years post-program)
Analysis of National Employment Benefits and Support Measures (EBSM) Profile, Outcomes and Incremental Impacts for 2006 to 2008 Participants: Newfoundland and Labrador (Completed in 2015)	Incremental impacts Profile and socio-demographic characteristics of participants	Non-experimental method using propensity score matching in combination with Difference-in-Differences Statistical profiling	2006 to 2008 participants	5 years between 2006 and 2012 (that is, 2 years in-program and 3 years post-program)
Analysis of Employment Benefits and Support Measures (EBSM) Profile, Outcomes, and Incremental Impacts for EI Claimants Category “Long-Tenured Workers” in Newfoundland and Labrador (Completed in 2016)	Incremental impacts Statistical profile of socio-demographic characteristics	Propensity score matching and Difference-in-Differences Statistical profiling	2007 to 2009 participants	5 years between 2007 and 2013 (that is, 2 years in-program and 3 years post-program)
Study on JobsNL: Newfoundland and Labrador (Completed in 2016)	Program design and delivery Challenges and lessons learned	Six key informants interviews Literature and document review Questionnaire filled by Newfoundland and Labrador officials	Design and delivery at the time of the data collection (that is, 2015)
Study on Skills Development-Apprentice Supports in Newfoundland and Labrador (Completed in 2016)	Program design and delivery Challenges and lessons learned	Six key informants interviews Literature and document review Questionnaire filled by Newfoundland and Labrador officials	Design and delivery at the time of the data collection (that is, 2015)
Study of Newfoundland and Labrador Skills Development Program (Completed in 2016)	Program design and delivery Challenges and lessons learned	Seven key informants interviews Literature and document review Questionnaire filled by Newfoundland and Labrador officials

Footnotes

Footnote 1

Long-tenured workers covered in the evaluation are individuals who had a long-term attachment to the labour market but not necessarily a long tenure with the same employer.

Return to footnote 1 referrer

Footnote 2

While JobsNL is an intervention with a maximum duration of one year, participants may have been in the program during parts of two fiscal years.

Return to footnote 2 referrer

Footnote 3

Red Seal, 2014. Apprenticeship Completion, Certification and Outcomes. Ottawa: Red Seal.

Return to footnote 3 referrer

Footnote 4

Patrick Coe, 2013. “Apprenticeship programme requirements and apprenticeship completion rates in Canada.” Journal of Vocational Education and Training. 65(4): 575 to 605.

Return to footnote 4 referrer

Footnote 5

Former claimants who received maternity or parental benefits were not covered by the evaluation given the difficulty in finding a suitable comparison group.

Return to footnote 5 referrer

Footnote 6

Full details about the incremental impact methodology can be found in the following report: Stream 1 Study for 2013 to 2014: National Level Analysis of EBSM Incremental Impacts. Methodology Report. Evaluation Directorate. ESDC. September 16, 2013.

Return to footnote 6 referrer

Footnote 7

Government of Newfoundland and Labrador. Strategic Plan 2014 to 2017. Department of Advanced Education and Skills.

Return to footnote 7 referrer

Footnote 8

Government of Newfoundland and Labrador. Speech from the Throne. March 8, 2016.

Return to footnote 8 referrer

Footnote 9

Incremental impacts on earnings are estimated relative to pre-participation levels and to the comparison group. They are estimated using current dollars.

Return to footnote 9 referrer

Footnote 10

Red Seal. 2014. Apprenticeship Completion, Certification and Outcomes. Ottawa: Red Seal.

Return to footnote 10 referrer

Footnote 11

Patrick Coe. 2013. “Apprenticeship Programme Requirements and Apprenticeship Completion Rates in Canada.” Journal of Vocational Education and Training. 65(4): 575 to 605.

Return to footnote 11 referrer

Footnote 12

Sébastien LaRochelle-Côté and Sharanjit Uppal, "The Financial Well-Being of the Self-Employed," Perspectives on Labour and Income, vol. 23, no. 4, Winter 2011.

Return to footnote 12 referrer

Footnote 13

Government of Newfoundland and Labrador. Job Creation Partnerships Program Guidelines. Department of Advanced Education, Skills and Labour. April 1, 2014

Return to footnote 13 referrer

Footnote 14

While JobsNL is an intervention with a maximum duration of one year, participants may have been in the program during parts of two fiscal years.

Return to footnote 14 referrer

Footnote 15

Red Seal, 2014. Apprenticeship Completion, Certification and Outcomes. Ottawa: Red Seal.

Return to footnote 15 referrer

Footnote 16

Patrick Coe, 2013. “Apprenticeship programme requirements and apprenticeship completion rates in Canada.” Journal of Vocational Education and Training. 65(4): 575 to 605.

Return to footnote 16 referrer

Page details

Date modified:: 2022-04-14

Evaluation of the Canada – Newfoundland and Labrador Labour Market Development Agreement

On this page

Alternate formats

Executive summary

Introduction

Key findings

Effectiveness and efficiency of EBSMs

Main challenges about EBSM design and delivery

Skills Development

Skills Development – Apprentice Supports

JobsNL

Employment Assistance Services

Recommendations

Management response

1. Introduction

1.1 Labour Market Development Agreement background

Employment Benefits

Support Measures

1.2 Methodology

Incremental impacts analysis

Factors accounted for in the cost-benefit analysis

Strengths and limitations of the studies

1.3 Overview of the studies summarized in this report

2. Evaluation findings

2.1 Rationale and labour market context

LMDA investments align with provincial government priorities

2.2 Skills Development

2.2.1 Program description

2.2.2 Program delivery

2.2.3 Targeting to labour market demand

2.2.4 Profile of Skills Development participants

2.2.5 Incremental impacts

Active claimants

Former claimants

2.2.6 Cost-benefit results

2.2.7 Challenges and Lessons Learned about Skills Development Design and Delivery

2.3 Skills Development-Apprentices Supports

2.3.1 Program description

2.3.2 Profile of Skills Development–Apprentice Supports participants

2.3.3 Labour market outcomes

Active claimants

Former claimants

2.3.4 Challenges and lessons learned about Skills Development –Apprentice Supports design and delivery

2.4 JobsNL

2.4.1 Program description

2.4.2 Program delivery

2.4.3 Profile of JobsNL participants

2.4.4 Incremental impacts

Active claimants

2.4.5 Cost-benefit results

2.4.6 Challenges and lessons learned about JobsNL design and delivery

2.5 Self-Employment Assistance

2.5.1 Program description

2.5.2 Profile of Self-Employment Assistance participants

2.5.3 Challenges in measuring Self-Employment Assistance incremental impacts

2.6 Job Creation Partnerships

2.6.1 Program description

2.6.2 Program delivery

2.6.3 Profile of Job Creation Partnerships participants

2.6.4 Incremental impacts

Active claimants

Former claimants

2.6.5 Cost-benefit results

2.7 Employment Assistance Services

2.7.1 Program description and delivery

2.7.2 Profile of Employment Assistance Services participants

Socio-demographic characteristics

Active claimants

Former claimants

Labour Market barriers faced by Employment Assistance Services participants

2.7.3 Incremental impacts

2.7.4 Cost-benefit results

2.7.5 Challenges and lessons learned about Employment Assistance Services design and delivery

3. Comparison of key findings by program type

4. Conclusions

Skills Development

Skills Development – Apprentices supports

JobsNL

Employment Assistance Services

5. Recommendations