Horizontal Evaluation of the Youth Employment Strategy - Skills Link stream

About Skills Link

Under Skills Link, youth aged 15 to 30 were provided with opportunities to gain employment experiences and to acquire knowledge/skills through a mix of interventions. In particular, Skills Link targeted youth who faced barriers.

• Barriers included, but were not limited to, challenges faced by recent immigrant youth, youth with disabilities, youth who have not completed high school, and Indigenous youth

Skills Link interventions included an income support component to ensure that the participants had the means to support themselves, while they were taking steps to become employed. Income support was provided when no other support such as social assistance was available. It took the form of an allowance equal to the provincial/territorial minimum wages, up to a maximum amount per participant.

The Skills Link stream involved two federal departments and one Crown Corporation, who each delivered different initiatives (see Table 1 for more detailed information):

• Employment and Social Development Canada (ESDC)
• Indigenous and Northern Affairs Canada (INAC)
• Canada Mortgage and Housing Corporation (CMHC)

For the period 2013 to 2018, about 72% of the Skills link funding envelope was allocated to ESDC, with the remaining 25% and 3% allocated to INAC and CMHC respectively.

*For 2014 to 2015, INAC's spending reflects planned spending rather than actual spending.

Transitioning from the Youth Employment Strategy to the Youth Employment and Skills Strategy

As part of the modernization of the Youth Employment Strategy, all three streams –Summer Work Experience, Skills Link and Career Focus – underwent a re-design to respond to a range of labour market challenges faced by youth, particularly those facing barriers to employment, under a new Youth Employment and Skills Strategy (YESS) that came into effect in June 2019.

The new Strategy continues to be delivered by 11 departments and agencies across the Government of Canada. It aims to provide more flexible employment services tailored to each individual and enhanced supports for all young Canadians as well as to broaden eligibility and offer a wider range of programs to help young people gain the skills, abilities and experience they need to get good-quality jobs.

The transition from the Youth Employment Strategy to a modernized Youth Employment and Skills Strategy took place during the preparation of this evaluation report for Skills Link. Preliminary evidence for this and the other reports on the components of the Strategy were shared with the program area in order to support policy development as it became available. In addition, findings and observations from this report will serve policy development and inform the design and delivery of similar programs.

Scope and evaluation questions

1. Did Skills Link reach its eligible participants and what were the characteristics of the participants?
2. To what extent did Skills Link participants improve their labour market participation (employed, unemployed, self-employed, returned to school)?
3. What was the average cost per participant and what were the costs compared to the benefits of participating in Skills Link?
4. What impacts did Skills Link interventions have on supporting participants to obtain employment, decrease reliance on income support programs such as EI or on social assistance?
5. Did impacts vary by region, by target groups, by duration or type of intervention?

Data sources and timeframe

To report on recent trends in the number of participants as well as actual spending, we used information from the TBS Horizontal Initiatives Database as well as ESDC's Data Collection System and Common System for Grants and Contributions from 2013 to 2018 (including departmental annual Year-End reports when available). In addition, for the period of 2013 to 2018, information supplied by Indigenous and Northern Affairs Canada (INAC) was used.

Information on immediate outcomes or status within 12 weeks after participating in Skills Link was also drawn from these data sources. Similar information was not available for INAC. 

For the purposes of conducting the incremental impact analysis for ESDC Skills Link participants, information from the Common System for Grants and Contributions for the period from January 2010 to December 2011 was used to establish a profile of participants.

This cohort of participants was used in order to properly evaluate the impact of the program on participants. These cohorts provided five years of labour market outcomes after participation to assess the effectiveness of an intervention. By using these cohorts, the evaluation can leverage the most recent tax records released in 2018 for the 2016 tax year.

Other data sources were used including the Employment Insurance Status Vector and Records of Employment. Where appropriate, lines of evidence were enriched with other external sources of data and relevant literature to provide context to the analysis.

From 2013 to 2018, about 60,400 youth took part in Skills Link interventions, translating in an average cost of about $10,000 per participant.

• Over the period of 2013 to 2018, the level of actual spending fluctuated and reached a low of about $82M in 2015 to 2016 and a high of about $170M in 2017-2018 (Figure 2 - right axis)
• Similarly, as shown in figure 1, the number of participants reached a low of about 7,600 in 2015 to 2016 and a high of about 19,000 in 2017 to 2018
• Over this period, the average cost per participants trended differently depending on the department (Figure 2 - left axis)

• For ESDC, the cost per participant remained steady at around $10,500
• For INAC, the cost per participant fluctuated and reached a high of about $13,400 in 2016 to 2017 and a low of about $6,200 in 2013 to 2014 and 2017 to 2018
• For CMHC, the cost per participant trended upward from about $7,100 in 2013 to 2014 to about $11,800 in 2017 to 2018

• Lastly, prior to 2016 to 2017, actual spending tended to be consistently lower than planned spending. This reflected, in part, the re-allocation of federal funding between Skills Link and Career Focus as well as other factors such as delays in launching the calls for proposals

Text description of Figure 1

Department	2013 to 2014	2014 to 2015	2015 to 2016	2016 to 2017	2017 to 2018
ESDC	9603	6563	6024	8790	10659
INAC	2651	1255	1440	3944	7713
CMHC	136	122	129	702	659
Total	12390	7940	7593	13436	19031

Text description of Figure 2 MISSING

[MISSING DESCRIPTION MISSING]

• Sources: Horizontal Initiative Database and ESDC Data Collection System. As of 2015 to 2016, information regarding the First Nations and Inuit Skills Link Program was supplied directly by INAC. For 2014 to 2015, INAC's spending reflects planned spending rather than actual.

Indigenous and Northern Affairs Canada Intervention: First Nations and Inuit Skills Link Program

The First Nations and Inuit Skills Link Program supported activities that assist youth in acquiring the essential skills that will help them gain employment, function well in the workplace, and learn about job and career options. The program consisted of four specific initiatives:

• Mentored Work Placements
• Co-operative Education Placements and Internships;
• Career Promotion and Awareness
• Science and technology activities (for example, National Science Camp)

The following key activities underpinned these initiatives:

• wage subsidies for work placements and mentorship for youth who are not in school to enable them to develop employability skills and support their educational and career development
• activities designed to support First Nations and Inuit youth entrepreneurs gain self-employment
• training experiences that support youth in acquiring skills needed for work placements
• career development information, including awareness and support activities like career fairs and leadership projects
• career planning and counselling activities
• activities that promote science and technology as an educational or career choice, including science camps, computer clubs, and activities that connect science and technology to traditional Indigenous knowledge

Canada Mortgage and Housing Corporation Intervention: Housing Internship Initiative for First Nations and Inuit Youth

The Housing Internship Initiative for First Nations and Inuit Youth provided wage subsidies to Indigenous organizations such as First Nations Councils, Tribal Councils, self-government entities and Indigenous businesses to provide on-the-job training in the housing sector to youth between the ages of 15 and 30.

• Internship opportunities included administration or office positions, maintenance, construction, renovation, or client counseling, for a minimum of 8 weeks up to a maximum of 2 years – subject to funding.
• Through this internship, CMHC supported the development of skills and employment opportunities to help transition First Nation and Inuit youth from school to employment.

Between 2013 and 2018, an average of about 350 interns per year were provided with work experiences. Among participants, most interns were men and most were 20 years and older (see Figure 3). In addition:

• About 4 in 10 completed their secondary education prior to the internship; and
• About 1 in 8 (13%) were employed or self-employed 12 weeks following their internship, with about 4% who returned to school (see Figure 4).

Text description of Figure 3

Age group	Average share (in %)
Under 20 years old	17%
20 to 24 years old	41%
25 years old or older	42%

Text description of Figure 4

Status	Average share (in %)
Unknown status	33%
Returned to school	4%
Unemployed	6%
Employed/self-employed	13%

• Source: ESDC Data Collection System.
• Note: Percentages do not add up to 100% due to non-responses (43% not specified)

ESDC's Skills Link eligibility requirements

Skills Link interventions were delivered by regional and local organizations responding to calls for proposals. Following an assessment process, funding in the form of contributions was provided directly to employers, organizations or community coordinators.

• A community coordinator is a contribution recipient that received funding to enter into its own agreement with eligible participants and employers to undertake activities that further the objectives of Skills Link

For the ease of reporting, organizations in receipt of funding from ESDC will be referred to as ESDC service providers from hereon and throughout the report.

Profiles of selected cohorts

Between January 2010 and December 2011, about 29,000 youth took part in an ESDC Skills Link intervention ranging from 12 to 17 weeks in duration. Among participants:

• Close to 3 out of 5 were men (59%)
• Most were between the age of 20 and 24 (44%)

Relative to the Canadian youth population at-large, a higher proportion of participants self-identified as an Indigenous, while the proportion for those who self-identified as a person with disabilities was comparable.

• 20% of participants self-identified as Indigenous relative to about 6% for the Canadian youth at large (Census 2016 and National Household Survey, 2011)
• 11% of participants self-identified as a person with disability relative to about 13% for the Canadian youth at large (Statistics Canada, 2019). Results from the 2017 Canadian Survey on Disability are used rather than those from the 2012 Survey, since they include persons with cognitive and mental health related disabilities (Statistics Canada, 2018)

Lastly, close to 6 out of 10 participants had not completed high school (55%) and 25% received some social assistance benefits in the year prior to their participation.

• Source: Common System for Grants and Contributions.
• Note : Participants may belong to more than one sub-group.

ESDC participants across Canada

Geographically, the distribution of participants by province is roughly in line with the distribution of the Canadian youth across the country (see Table 4).

A majority of participants lived in Ontario and Quebec - with the smallest shares living in the Atlantic provinces (NB, NS and PEI) and the Territories.

• Along an urban-rural dimension, 85% of participants lived in urban communities while 15% resided in rural communities. This distribution is also roughly in line with the distribution of the Canadian youth living in rural communities across the country

Though not shown in Table 4, most participants who lived in the Territories were part of interventions taking the form of work experiences rather than skills training. More detail information on the profile of participants, at the intervention level, can be found in the technical report, which is available upon request (ESDC, June 2019).

• Source: ESDC (2019). Internal Technical Report.

Main finding #1

Immediately after the completion of their respective intervention, more than 7 in 10 participants reported to be employed or to have returned to school, exceeding the projected program target of 60%. Notably, 85% of participants who took part in an Entrepreneurship intervention reported to be employed or to have returned to school.

For ESDC, the desired outcomes of Skills Link interventions were to see at least 60% of participating youth employed or returned to school.

• Immediately after completing an intervention and according to ESDC service providers, about 48% of participants were employed and 25% returned to school, for a total of 73% (see Figure 5).  Across interventions, individuals who completed:

• An Individual Skills Enhancement (ISE) intervention were more likely to be in school (35%) than to be employed (26%)
• A Group-Based Employment Skills intervention (GBES) were more likely to be unemployed    

Text description of Figure 5

Intervention	Employed	In school	Unemployed
All	48%	25%	15%
GBES	45%	26%	18%
ESWE	52%	23%	13%
WE	57%	24%	9%
ISE	26%	35%	7%
ESE	70%	15%	4%

• Source: Common System for Grants and Contributions.

Notes:

• GBES: Group-Based Employability Skills, ESWE: Employability Skills through Work Experience, WE: Work Experience, ISE: Individual Skills Enhancement, and ESE: Employability Skills through Entrepreneurship.
• Percentages do not add up to 100% due to missing values or unknown status that varied from 11% for Group-Based Employment Skills intervention to 32% for the Individual Skills Enhancement intervention.
• Some youth may have not found work or decided not to return to school or were out of reach. In the latter situation, a follow-up after 12 weeks was required in order to verify if their status had changed.

During the 5-year period following an intervention, participating youth earned about $6,300 more per year relative to what they earned prior to their intervention.

As shown in Figure 6, during the five-year period following their participation:

• Youth earned an average of close to $6,300 more per year, compared to what they earned in the year prior to their participation

• About 6 in 10 (57%) participants were employed in the fifth year following their intervention

• The largest average increase in post-participation earnings (about $8,750 per year) was observed for participants who took part in Work Experience (WE) intervention
• A more modest average increase of about $4,000 per year was observed for participants who took part in Employability Skills through Entrepreneurship intervention (ESE)

Text description of Figure 6

Intervention	Average annual increase
Overall	$6,345
GBES	$5,890
ESWE	$6,735
WE	$8,745
ISE	$5,825
ESE	$4,030

• Source: Administrative Data Technical Report
• Note: GBES: Group-Based Employability Skills, ESWE: Employability Skills through Work Experience, WE: Work Experience, ISE: Individual Skills Enhancement, and ESE: Employability Skills through Entrepreneurship.

On average, participants earned less than $2,400 in the year prior to their intervention.

Clustering analysis of GBES participants points to a very heterogeneous group, facing very different barriers and pre/post-programs outcomes

For the purposes of examining the various segments of the population youth facing barriers to employment, clustering analysis was performed for participants who took part in Group-Based Employment Skills (GBES) interventions. As shown in Figure7, four distinct groups were identified, representing about 50% of participants. 

• Machine learning clustering algorithm consists of an iterative process that partitions a given data set into homogeneous groups (clusters) based on given features or characteristics, such that similar objects are kept in a group whereas dissimilar objects are in different groups

Text description of Figure 7

Among the youngest youth, with the lowest level of education, 67% were males; 76% were under 20 and 70% did not complete high-school. The highest educated immigrants were mostly made up of women (69%), 45% of whom completed their post-secondary education, and 70% were immigrants. Among youth who relied on social assistance, 61% were females, 21% had a disability, and 90% received social assistance income. Among youth who had the strongest labour market attachment, 60% were over 25 years old, 81% were employed prior to participating in the program, and 29% received EI benefits.

Relative to the pre/post-program outcomes observed for all GBES participants, certain groups of participants had distinct pre/post-program outcomes. For instance, relative to how much they earned or received in benefits in the year prior to their intervention:

• Highly educated immigrants (Cluster 2) saw the largest increase in post-participation earnings ($14,730 per year relative to $5,890 for all GBES participants as shown in Figure 6)
• Those relying on social assistance (Cluster 3) were the only group who saw a reduction in their post-participation receipt of social assistance ($2,030 per year)
• Those with stronger labour market attachment (Cluster 4) were the only group who saw a reduction in post-participation receipt of employment insurance benefits ($465 per year)

Main finding #2

Overall, when comparing results for participants with the results of a comparison group, the magnitude of the incremental impacts of Skills Link interventions on participants' labour market attachment was limited. This points to inherent challenges associated with helping youth facing barriers to employment who may require a wider and more comprehensive range of interventions.

As for the previous 2015 evaluation report, an incremental impact analysis was conducted for ESDC Skills Link participants. To that end, the Department relied on its Labour Market Data Platform using participants who received an intervention between January 2010 and December 2011. This period was selected to allow for the assessment of the interventions' impacts on labour market attachment over a period of at least five years following their completion up to calendar year 2016 (most recent available information). Building on previous evaluations, the incremental impact analysis was extended and conducted at the intervention level as well as at the sub-group level (for example, age and gender) for the two most prominent interventions.

Overall, following a Skills Link intervention, ESDC participants did not improve their labour market attachment, relative to a similar group of non-participants. As shown in Table 8, in the year when an intervention was received, the amount earned and the incidence of employment of participants was comparable to those observed for non-participants.

• Participants earned about $125 less than non-participants, and their incidence of employment was about 7.3 percentage point higher (bars in dark blue). The higher incidence of employment reflects, in part, the impact of participants who benefitted from a work placement as an intervention, which was not the case for the comparison group
• In addition, due to data limitations, the incidence of employment is determined based on whether an individual had any earnings during a calendar year. Therefore, it is possible to find instances where the net impact of an intervention is negative for earnings and positive for the incidence of employment; or vice versa. The former occurs when a relatively larger number of employed participants earn less than a relatively smaller number of employed individuals in the comparison group over a calendar year

• Source: Administrative Data Technical Report. pp = percentage points
• * Results are statistically not significant, still they are valid in terms of informing the direction of the impact (negative or positive).

Over the five-year period following their intervention, participants' outcomes did not improve relative to those observed for non-participants.

• Participants earned about $910 less a year than non-participants, and their incidence of employment was about 0.2 percentage point lower, on average (bar in light blue in Table 8).

Incremental impacts of Skills Link interventions on participants' reliance on income support programs such as Employment Insurance or on social assistance were of smaller magnitude and can be found in Annex E.

These limited results point to inherent challenges associated with helping youth with barriers to employment and with weak labour market attachment. These findings are also consistent with results found as part of the 2015 Evaluation as well as for similar active labour market programs targeting disadvantaged youth in other OECD countries (Kluve, 2014).

Main finding #3

Youth who received interventions in the form of work experiences had stronger labour market attachment, relative to youth who received interventions in the form of group-based workshops.

Building on the 2015 evaluation, incremental impact analysis was conducted at the intervention-level to assess the relative effectiveness of various interventions.

As shown in Table 9, youth who participated in Skills Link interventions that included a work experience component (Employability Skills through Work Experience and Work Experience interventions) had more positive labour market attachment impacts relative to other interventions. Still, the positive impact on participants' labour market attachment was not sustained over time and tended to revert back to those observed for the comparison group within five years following an intervention (bars in light blue).

Incremental Impacts on Employment Earnings and Incidence of Employment

For the year of participation (In-Year) and the following five years (Annual Average), by Intervention

• Source: Administrative Data Technical Report. pp = percentage points.
• * Results are statistically not significant, still they are valid in terms of informing the direction of the impact (negative or positive).

For example, as shown in Table 9, in the year when a Work Experience intervention was received, participants earned about $860 more than non-participants and their incidence of employment was about 17.2 percentage points higher. This was expected since the intervention involved a work placement associated with additional earnings, which was not the case for the comparison group.

Over the five-year period following their intervention, participants earned about $100 more per year than non-participants and their incidence of employment was about 5.9 percentage point higher, on average.

Similar to the results found for all Skills Link interventions (see Table 8), participants who took part in Group-Based Employment Skills (GBES) and Individual Skills Enhancement (ISE) did not improve their labour market attachment relative to their respective comparison groups in the years following an intervention. 

These findings are also consistent with international evidence, which suggests that active labour market programs targeting disadvantaged youth can help, if they are comprehensive – including job search assistance, counselling, training, and placement services (Kluve, 2014).

Robustness checks were performed to examine whether the relative ranking of the effects of each Skills link intervention were sensitive to the choice of the control group (see Annex D). Incremental impacts of Skills Link interventions on participants' reliance on income support programs such as Employment Insurance or on social assistance were of smaller magnitude and can be found in Annex E.

Main finding #4

Although the incremental impacts of interventions involving a workshop approach were generally negative, analysis at the sub-group level suggests that this type of interventions were relatively more effective for youth who had a lower level of education. In addition, impacts were relatively better for older youth (over the age of 19), particularly for interventions with a work experience component.

Lastly, an incremental impact analysis was conducted at the sub-group level (e.g., gender, age, level of education and rural/urban) for the two most prominent Skills Link interventions, that is Group-Based Employability Skills as shown in Table 10 and Employability Skills through Work Experience as shown in Table 11.  Selecting the two most prominent interventions provided for a sufficiently large number of participants to support the conduct of a more granular analysis at the sub-group level.

Annex A: Bibliography

Annex B: Evaluation approach

Annex C: Comparison between the making of a group of participants and their control group

Table 5 provides a comparison between various observed socio-demographic and economic characteristics for the group of participants who took part in Group-Based Employability Skill and their comparison group consisting of youth who received low-intensity employment assistance measures (EAS) as part of the Labour Market Development Agreement. The table shows how both groups had similar traits over a subset of socio-demographic and economic characteristics that were taken into account.

Annex D: Robustness check: Using a discrete subset of Group-Based Employability Skills (GBES) participants

• The relative ranking of the effects of each Skills link interventions was similar whether the comparison group was built from youth who received limited level of treatment through minor Employment Assistance Services (EAS) interventions or from youth who received a Group-Based Employability Skills interventions
• In particular, under both approach, Skills Link' interventions that included a work experience component (ESWE and WE interventions) led to better labour market attachment impacts
• For example, as shown in table 12, in the year when a WE intervention was received, participants earned about $1,940 more than comparable GBES participants and their incidence of employment was about 13.1 percentage point higher
• Over the five-year period following a Work Experience (WE) intervention, participants earned about $1,730 more a year than comparable GBES participants and their incidence of employment was about 5.6 percentage point higher on average

• Similarly, Individual Skills Enhancements (ISE) taking the form of short-term courses had a more sustained impact on labour market attachment, but to a lesser extent

• Source: Administrative Data Technical Report. pp = percentage points
• * Results are statistically not significant, still they are valid in terms of informing the direction of the impact (negative or positive)

Annex E: Incremental impacts of Skills Link interventions on participants' reliance on income support programs, by intervention.

As shown in Table 13, in the year when a Group-Based Employability Skills (GBES) intervention was received, participants received about $110 less than non-participants in Employment Insurance benefits and about $190 less in Social Assistance benefits.

Over the five-year period following their intervention, participants received about $80 less a year than non-participants in Employment Insurance benefits, and about $90 more in Social Assistance benefits.

• Source: Administrative Data Technical Report. pp = percentage points.
• * Results are statistically not significant, still they are valid in terms of informing the direction of the impact (negative or positive). (GBES = Group-Based Employability Skills; ISE = Individual Skills Enhancement; ESWE = Employability Skills through Work Experience; and WE = Work Experience)

Annex F: Incremental impacts of Group-Based Employability Skills on participants' reliance on income support programs, by subgroup

As part of Table 14, in the five-year period following a Group-Based Employability Skills intervention, the intervention led to:

• An average annual decrease of about $29 in post-participation receipt of Employment Insurance benefits and an decrease of about $493 in the receipt of Social Assistance benefits for participants who had completed their secondary studies

• Source: Administrative Data Technical Report. pp = percentage points
• * Results are statistically not significant, still they are valid in terms of informing the direction of the impact (negative or positive). Source: Administrative Data Technical Report. pp = percentage points

Annex G. Incremental impacts of Employability Skills through Work Experience interventions on participants' reliance on income support programs, by subgroup

As part of Table 15, in the five-year period following a Employability Skills through Work Experience intervention, the intervention led to:  

• An average annual increase of about $120 in post-participation receipt of Employment Insurance benefits and a decrease of about $330 in the receipt of Social Assistance benefits by participants that had completed their secondary study

• Source: Administrative Data Technical Report. pp = percentage points
• * Results are statistically not significant, still they are valid in terms of informing the direction of the impact (negative or positive). Source: Administrative Data Technical Report. pp = percentage points