Mid-cycle assessment of the Youth Employment and Skills Strategy

Barrier 1: high school not yet completed

Approximately 16% of Canada Summer Jobs and 22% of YESSP participants had not completed a high school diploma or equivalent at the time of their participation.

Barrier 2: being a new immigrant^{Footnote 3}

Approximately 2% of Canada Summer Jobs participants immigrated to Canada within 5 years of participating in a Canada Summer Jobs placement. For the YESSP, approximately 12% of participants self-reported as being new immigrants.

Barrier 3: being a visible minority

Approximately 21% of Canada Summer Jobs participants and 42% of YESSP participants self-identified as being a member of visible minority group.

Barrier 4: youth with disabilities

Just over 3% of Canada Summer Jobs participants and 24% of YESSP participants identified as being persons with disabilities.

Barrier 5: living and working in a rural area

For youth in Canada Summer Jobs, 19% reported living and working in rural areas. For YESSP participants just over 9% of participants reported living in rural or remote areas.

Canada Summer Jobs results

Since the implementation of the modernized strategy, the Canada Summer Jobs component has exceeded its target goal for paid work experience, except for the beginning of the COVID-19 pandemic in 2020. In fiscal year 2020 to 2021, the component fell short of its target job placements by 11,862. However, the component increased its target for job placements and exceeded them for fiscal year 2021 to 2022 and March to December 2022.^{Footnote 4}

Based on exit surveys for Canada Summer Jobs participants, almost 15% of respondents reported that their placement was their first work experience, while the majority reported having previously worked (85%).^{Footnote 5} In addition, from June 2019 to December 2022, 99% of employers responding to the survey indicated that Canada Summer Jobs participants developed skills during their job placement. Similarly, 97% of youth who responded to the most recent Canada Summer Jobs exit survey reported developing or strengthening transferable skills, and 89% self-reported they developed or strengthened employment skills.^{Footnote 6}

YESSP results

Outcomes for YESSP participants are derived from the Participant Information Templates completed at program exit. From April 2020 to December 2022, early outcomes of ESDC’s YESSP participants show that on average 68% are employed, and an additional 13% returned to school following their participation in this component. Approximately 8% were unemployed following their participation.

YESSP

Agreement holders must provide information in the Participant Information Template, such as intervention type(s), and project start and completion dates. The data assessment found a 100% response rate for this information. Specifically, the CSGC data for YESSP covers participants for whom a template was completed from April 2020 to December 2022, resulting in 77,978 interventions corresponding to 20,888 participants.

The data assessment identified several limitations, including:

• participant data is not available for the first year of the modernized Strategy (2019 to 2020)^{Footnote 8}
• data are only available for participants for whom a result date is reported (that is, a Participant Information Template is filled out and submitted to ESDC)
  • no information exists for participants who did not complete the Participant Information Template regardless of whether or not they completed an intervention

Historically, partner departments, agencies and Crown corporations uploaded their participant data to ESDC through the data collection system. A data collection system is not yet available for partner departments, agencies and Crown corporations to securely upload their participant data to ESDC. Alternative means of incorporating partners’ evaluation results will be investigated.

With no YESSP participants’ data available for fiscal year 2019 to 2020, YESSP participants in 2020 to 2021 will be the most recent cohort of participants that can be used for assessing the impact of the modernized Strategy.^{Footnote 9} As a result, the quantitative analysis for the modernized Strategy, as part of the horizontal evaluation, will be limited to the examination of short-term incremental impacts^{Footnote 10} between 6 and 15 months following participation.

Canada Summer Jobs

The Evaluation Directorate found that data for Canada Summer Jobs was available from June 2019 to December 2022, covering 100% of participants (N=368,891). As with the previous evaluation of Canada Summer Jobs, the analysis examines immediate outcomes reported in the CSGC through the information collected in the Participant Information Template and data collected by ESDC in the exit surveys for employers and participants. As well, short-term incremental impacts will be examined for Canada Summer Jobs participants.

Profile of participants

From June 2019 to December 2022, nearly 368,900 youth participated in Canada Summer Jobs and nearly 20,900 youth participated in YESSP.^{Footnote 17,Footnote 18} Table 2 details the characteristics of participants in both components. Information about sociodemographic groups is self-reported.^{Footnote 19}

Source: ESDC’s Common System for Grants and Contributions.

Province or territory of residence

Table 3 presents the geographic distribution of participants in Canada Summer Jobs from June 2019 to December 2022, and in YESSP from April 2020 to December 2022. For the Canada Summer Jobs component, most participants (64%) are located in Ontario and Québec. For the YESSP, most participants (52%) are from Ontario.

It is noted that there is a Canada-Québec agreement with the ministère de l’Emploi et de la solidarité sociale for the regional (YESSP) component. Since this agreement is in place, organizations are not eligible for the regional component, and can only apply for the national component. Quebec is responsible for reporting its data to ESDC. At the time of the mid-cycle assessment, this information was not available.

• Source: ESDC’s Common System for Grants and Contributions.
• Note: The reference period examined for Canada Summer Jobs participants is from June 2019 to December 2022, while for YESSP the reference period is from April 2020 to December 2022.

Table 4 presents the first official language of YESSP participants. From April 2020 to December 2022, approximately 95% of YESSP participants self-reported English as their first official language, while 4% reported French^{Footnote 23} as their first official language. Official Language data for Canada Summer Jobs participants is not available.

Source: ESDC’s Common System for Grants and Contributions.

Canada Summer Jobs Component

YESSP component

Combination of barriers most commonly faced by YESSP participants

Based on participants’ self-reported information, the most common combination of barriers that participants face is being:

• visible minority and new immigrant
• visible minority and a person with disabilities
• visible minority and high school not yet completed
• high school not yet completed and a person with disabilities

Table 6 below outlines the most common combination of barriers that YESSP participants report.

• Source: ESDC’s Common System for Grants and Contributions.
• Note: Not all combinations of barriers are listed due to small populations (<1%).

YESSP

As seen in Table 7, from June 2019 to December 2022, 20,889 YESSP participants took part in 77,982 interventions, with most participants receiving 2 or more interventions.^{Footnote 28}

• Source: ESDC’s Common System for Grants and Contributions.
• Note: The participant numbers are based on the start date for YESSP participation.
• Note: This table does not include participants from the YESSP Quebec regional component under the ministère de l’Emploi et de la solidarité sociale

Types of YESSP interventions

The YESSP component consists of 5 employment skills interventions and 3 support measures.

As can be seen in Table 8, the most common type of intervention for YESSP participants was Employment Skills Training (n=18,342), followed by Quality Employment Opportunity intervention (n=15,143). Fiscal year 2021 to 2022 had the highest number of participants and YESSP interventions. This increase is likely due to increased funding under the flexibilities granted in response to the Covid-19 pandemic.

• Source: ESDC’s Common System for Grants and Contributions (April 2020 to December 2022)
• Note: YESSP data has different start and end dates for each YESSP participant in YESSP projects. However, each intervention taken by individual participants has the same start and end date. This makes it impossible to define the length of each intervention received.
• Note: Fiscal year is based on the start date of interventions within the project.
• Note: Participants may take multiple interventions, as such the number of interventions is greater than the number of participants.

Canada Summer Jobs

The following information comes from Canada Summer Jobs exit surveys for participants and employers (from June 2019 to December 2022), as well as a review of ESDC documents. The exit surveys are conducted by ESDC’s Program Operations Branch. At the end of each funding cycle, the exit survey asks employers and youth (who have completed Canada Summer Jobs-funded work placements) to complete questionnaires on client satisfaction and experience with the Canada Summer Jobs component. The employer questionnaire became mandatory in fiscal year 2021 to 2022, while the youth questionnaire is voluntary but encouraged.^{Footnote 29}

As shown in Table 11, the results from June 2019 to the December 2022 found that 368,891 individuals participated in the Canada Summer Jobs component. On average, placements were 71 days in duration.

Source: ESDC’s Common System for Grants and Contributions.

As shown in Table 12, from June 2019 to December 2022, the Canada Summer Jobs component exceeded its target goal for paid work experiences for participants (except for the first year of the pandemic in 2020 to 2021). The component increased its target goal for paid work experiences and exceeded them from 2021 to 2022 and March to December 2022.

Source: Internal ESDC reporting documents.

Challenges associated with the implementation of the modernized strategy

According to program officials interviewed, the following challenges associated with data collection could prevent the Strategy from attaining its objectives:

While there is a horizontal Performance Information Profile in place, 2 data collection systems have traditionally been used for data collection. Specifically, one for ESDC’s data, and another one for partner departments, agencies and Crown corporations. Historically, partner departments, agencies and Crown corporations uploaded their participant data to ESDC through the data collection system. A data collection system is not currently available for partner departments, agencies and Crown corporations to securely uploaded their participant data to ESDC. Alternative means of incorporating partners’ evaluation results will be investigated.

Data standardization issues where collected data differs from partner to partner (for example, inconsistencies in partners collecting data on specific barriers faced by youth)

Key informants identified negative aspects regarding the communication of new component changes and a lack of a clear definition of youth barriers between partner departments and agencies

Challenges associated with the effects of the Covid-19 pandemic

Youths reported that the Covid-19 pandemic disrupted their education, with youth being increasingly concerned about their financial situation, even after the announcement of the Canadian Emergency Student Benefits. These factors likely contribute to lower mental health levels among youth, who have experienced the greatest decline in mental health since the beginning of the pandemic.^{Footnote 32},^{Footnote 33},^{Footnote 34}

According to a literature review conducted for the evaluation, the Covid-19 pandemic exacerbated challenges and impacts faced by youth who are facing barriers such as 2SLGBTQ+, visible minority, Indigenous and/or persons with disabilities.^{Footnote 35} For example:

• The pandemic has worsened educational outcomes, compromised mental health, increased cyberbullying and victimization, increased unemployment, and exacerbated family conflict for 2SLGBTQ+ youth
• Some vulnerable segments of black and racialized student populations may be at greater risk of experiencing declining social mobility

The pandemic exacerbated racial inequities in education and widening social inequities, disparities, and vulnerabilities within Black and racialized communities:

• Online platforms adopted across Canada in response to the pandemic may disadvantage Indigenous communities based on holistic and communal learning

Furthermore, there are challenges with internet connectivity and access to electronics and other devices.

For persons with disabilities, the pandemic resulted in a loss of access to disability-related supports and critical community services. The pandemic added a new level of strain for students reliant on assistive technologies to support in class instruction.

Lessons learned regarding the type of interventions that work for youth facing barriers

A literature review also identified lessons learned to help youth facing barriers, these include:

Early interventions for high-school youth in the education system appear to be very effective, especially for disadvantaged youth.^{Footnote 36} They increased education outcomes including subsequent university and college graduation and subsequent earnings.

Provision of timely and local labour market integration and job search assistance is of increasing importance for youth.^{Footnote 37} It is effective, especially for disadvantaged youth, and especially because of its low-cost. Labour market integration and job-search assistant should be targeted and designed for the needs of the different groups.

Micro credentials can be especially important for youth since they can be acquired in a more flexible manner that allows for targeting of specific requirements for different jobs and can be used for life-long learning.^{Footnote 38}

Lessons learned from the evaluation of the Labour Market Development Agreements^{Footnote 39}

The Horizontal Evaluation of the Labour Market Development Agreements found that youth in interventions such as Skills Development^{Footnote 40} and Targeted Wage Subsidies^{Footnote 41} improve their labour market attachment through increases in their incidence of employment and employment earnings compared to similar youth non-participants. They also decrease their dependence on government income support (that is, the combined use of Employment Insurance and social assistance benefits).

Canada Summer Jobs

For the Canada Summer Jobs component, the temporary measures and flexibilities were first announced in April 2020 and lasted until February 2022. The temporary flexibilities offered to employers in fiscal years 2020 to 2021 and 2021 to 2022 included:

Wage subsidies

All funded employers were eligible to receive a wage subsidy reimbursement of up to 100% (75% for 2021 to 2022) of the provincial or territorial minimum hourly wage. Previously, private and public sector employers were only eligible to receive up to 50% of the provincial or territorial minimum hourly wage.

Part-time employment

All funded employers could offer part-time placements (for example, fewer than 30 hours per week). Previously, all Canada Summer Jobs-funded employment had to be full time (a minimum of 30 hours per week for at least 6 weeks). This continued in 2021 to 2022.

Employment period

All funded employers could offer job placements between May 15, 2020 and February 28, 2021. Employers could offer part-time employment to youth who wanted to work during the academic year. This continued in 2021 to 2022.

Changes to project and job activities

All funded employers were provided with the flexibility to amend project and job placement activities to support the delivery of critical services. This continued in 2021 to 2022.

Following fiscal year 2021 to 2022, temporary flexibilities offered for Canada Summer Jobs were ended due to low uptake of the flexibilities.

As described in Table 13, project applications from Canada Summer Jobs employers remained near pre-pandemic levels, with approved projects in fiscal year 2020 to 2021 (26,857) being slightly lower than approved projects in 2019 to 2020 (close to 29,500). The average job placement duration of Canada Summer Jobs job placements increased during the pandemic years (8.6 weeks in 2020 to 2021 and 9.3 weeks in 2021 to 2022).

Source: Internal ESDC reporting documents.

Canada Summer Jobs employers use of temporary flexibilities during the pandemic

In fiscal year 2020 to 2021, many approved Canada Summer Jobs employers took advantage of the temporary flexibilities provided by ESDC because of the Covid-19 pandemic. By fiscal year 2021 to 2022, the use of temporary flexibilities fell off considerably, in particular in terms of employment period and part-time employment. During the pandemic, project applications from employers remained near pre-pandemic levels, with approved projects in 2020 to 2021 (26,857) being slightly lower than approved projects in 2019 to 2020 (nearly 29,500). The average duration of Canada Summer Jobs work experiences increased slightly during the pandemic years, from an average of 8.5 weeks in 2019 to 2020 to 8.6 weeks in 2020 to 2021 and 9.3 weeks in 2021 to 2022.

YESS responsiveness to the needs of Canada Summer Jobs employers

In the fall of 2022, a survey of Canada Summer Jobs employers was conducted by ESDC for the evaluation. While the survey had non-representative results and a low response rate of 7.25% (n=881), the responses can provide some indications of the effects of the Covid-19 temporary measures. The results for the ESDC Canada Summer Jobs survey for employer use of temporary flexibilities (n=546) are as follows:

• of employers surveyed, 62% stated that they used the temporary flexibilities
• for survey respondents, 81% indicated that the temporary flexibilities either mostly or completely met their organizations needs
• for survey respondents who used the temporary flexibilities 90% reported that the Canada Summer Jobs application process with the inclusion of the temporary flexibilities was easy or very easy

Approximately 48% (n=546) of respondents who used the temporary flexibilities used a combination of these in their Canada Summer Jobs project.

YESSP^{Footnote 43}

The ESDC YESSP data is stored in the Common System for Grants and Contributions (CSGC). Based on the data assessment, records are only available for YESSP participants from April 2020 to December 2022

Data limitation: There are only a handful of YESSP participants from fiscal year 2019 to 2020. The lack of participant records for the first year of YESSP is attributed to the lack of signed agreements in the first year of the Strategy.

The YESSP data is collected via the Participant Information Template. Agreement holders must provide information in the template such as intervention type(s), project start date, and project completion date. The data assessment found 100% response rates for this information. Specifically, the CSGC data for YESSP covers participants for whom a template was completed from April 2020 to the December 2022 period, resulting in 77,978 interventions corresponding to 20,888 participants.

Data limitation: Data are only available for participants for whom a result date is reported (that is, a Participant Information Template is filled out and submitted to ESDC). No information exists for participants who did not complete the Participant Information Template regardless of whether or not they completed YESSP interventions. This may limit the estimation of incremental impacts and limit conducting a cost-benefit analysis. Peer reviewers have consistently advised to consider all participants who started program participation to assess program impact and effectiveness.

The new YESSP is intended to meet an individual's holistic needs through available supports, services, and resources to promote a positive result. As such, a participant can receive supports concurrently with any intervention. In particular, supports can vary and be provided before, during and after the intervention, within the project start and end dates. There are challenges for the implementation of new performance measurement frameworks.

Data limitation: Intervention start and end dates are not uniquely captured; only the start and end dates of participation in the full YESSP project is recorded. Because of this data collection approach and the way intervention dates are captured, there are challenges to assigning outcomes to a “main” intervention type, as is traditionally examined in other ESDC labour market program evaluations (for example, the Labour Market Development Agreements). This lack of start and end dates per intervention may limit the capacity to estimate incremental impacts with the perspective of informing which interventions work best for whom.

With no YESSP participants’ data for fiscal year 2019 to 2020, participants in 2020 to 2021 will be the earliest cohort of participants that can be used for impact analysis for the modernized Strategy. This will limit the scope of the quantitative analysis that can be conducted as Canada Revenue Agency taxation data has a 2-year lag period from year of filing to year of availability for analysis (that is, the 2021 tax data will only be available in Autumn 2023). As a result, the quantitative analysis for the modernized Strategy, as part of horizontal evaluation, will be limited to the examination of immediate incremental impacts^{Footnote 44} (between 6 and 15 months post-participation for individuals who started their participation between April and December 2020.

Cost-benefit analysis is not possible at this time given the limited number of years available to assess the benefits associated with participation in the modernized Strategy.^{Footnote 45} As such, the YESS horizontal evaluation will not be able to fully assess the impact and effectiveness of the modernised Strategy.

As previously mentioned, the CSGC database does not contain data for Quebec specific projects. There is a Canada-Québec agreement with the ministère de l’Emploi et de la solidarité sociale for the ESDC YESSP Regional component. Because this agreement is in place, organizations were not eligible for that component; they could only apply for the national component. Quebec is responsible for reporting its data to ESDC. At the time of the mid-cycle assessment, this information was not available.

Data limitation: as the Quebec specific data are not available in the CSGC database, any analysis of YESSP will not include Quebec.^{Footnote 46}

For participants, the collection of sociodemographic and personal information such as Social Insurance Number, birth date, gender and postal code through the Participant Information Template is done at the onboarding stage and is voluntary. While voluntary, the data analysis found high response rates for the fields presented in Table 14.

Source: ESDC’s Common System for Grants and Contributions (April 2020 to December 2022 YESSP participants).

Partner YESSP departments, agencies and Crown corporation data

Historically, partner departments, agencies and Crown corporations uploaded their participant data to ESDC through a data collection system. A data collection system is not currently available for partner departments, agencies and Crown corporations to securely uploaded their participant data to ESDC. Alternative means of incorporating partners’ evaluation results will be investigated.

Data limitation: due to the lack of available data and given time constraints to respect the Financial Administration Act and Treasury Board requirements, the quantitative analysis for YESSP participants will be limited to ESDC participants.

Each partner department and agency is responsible for the management and implementation of their respective YESS initiatives and also for the decision on whether to conduct their own evaluation for their initiative(s). Given that, this Evaluation strategy aims, in part, to guide any activities conducted by partner departments and agencies to provide and inform a more comprehensive horizontal assessment of the YESS program.

If a partner department or agency plans to conduct an evaluation of its own YESS initiative, the Evaluation Directorate will use the findings to inform the Horizontal Evaluation report. Furthermore, ESDC evaluation will use available complementary information, documentation or analysis held by partner departments or agencies regarding their YESS initiatives or programs. Relatedly, partner departments and agencies are also responsible to provide Employment and Social Development Canada with administrative data and information on their initiatives.

Canada Summer Jobs^{Footnote 47}

The Evaluation Directorate found that data for Canada Summer Jobs was available for the period of June 2019 to December 2022, covering 100% of participants (N=368,891). As with the previous evaluation of Canada Summer Jobs, the analysis will examine immediate outcomes reported in the CSGC through the information collected in the Participant Information Template and data collected by ESDC in the exit surveys for employers and participants.

Canada Summer Jobs

From June 2019 to December 2022, nearly 368,900 youth participated in the Canada Summer Jobs component. During that period, almost 15% of Canada Summer Jobs participants reported that their placement was their first work experience. Canada Summer Jobs also exceeded its target goal for paid work experience with the exception of the beginning of the COVID-19 pandemic in 2020. In 2020 to 2021, the component fell short of its target job placement creation by 11,862 job placements, but in the following 2 years the component increased its target goal for job placement creation and exceeded them from fiscal year 2021 to 2022 and March to December 2022. In fiscal year 2021 to 2022, the component created 23,882 job placements over target and was 13,223 job placement creations over target from March to December 2022.

On average, from June 2019 to December 2022, 99% of employers responding to the Canada Summer Jobs exit survey indicated that participants developed skills during their job placement. Similarly, youth surveyed in the most recently available Canada Summer Jobs exit survey showed that 97% self-reported they developed or strengthened transferable skills, and 89% self-reported they developed or strengthened employment skills.

In fiscal year 2020 to 2021, many approved Canada Summer Jobs employers took advantage of the temporary flexibilities provided by ESDC because of the Covid-19 pandemic. By fiscal year 2021 to 2022, the use of temporary flexibilities fell off considerably, in particular in terms of employment period and part-time employment. During the pandemic, project applications from employers remained near pre-pandemic levels, with approved projects in 2020 to 2021 (26,857) being slightly lower than approved projects in 2019 to 2020 (nearly 29,500). The average duration of Canada Summer Jobs work experiences increased slightly during the pandemic years, from an average of 8.5 weeks in 2019 to 2020 to 8.6 weeks in 2020 to 2021 and 9.3 weeks in 2021 to 2022.

YESSP

From April 2020 to December 2022, nearly 20,900 youth participated in the YESSP. The YESSP is a more intensive component focusing on youth facing barriers. Most YESSP participants receive 2 or more component interventions. From April 2020 to December 2022, the average duration of combinations of interventions declined from a high of 141 days in 2020 to 2021 to 83 days from March to December 2022. A data assessment on barriers that youth mainly face for the YESSP and Canada Summer Jobs was conducted based on 5 known barriers to youth employment including:

• youth who have not yet completed high school
• youth who are a new immigrant
• youth who are a visible minority
• youth who are a person with disability
• youth who are living in rural or remote areas

Combinations of barriers was also examined based on participants’ self-reported information, the most common combination of barriers that participants face are being:

• visible minority and new immigrant (9%)
• visible minority and being a person with disabilities (5%)
• visible minority and high school not yet completed (5%)
• high school not yet completed and being a person with disabilities (5%)

Early outcomes on the modernized Strategy for the YESSP found that on average 68% are employed, and an additional 13% returned to school following component participation. Approximately 8% were unemployed following their participation.

In regard to the implementation of the new performance measurement framework components, for YESSP participants specifically, it is now voluntary for participants to provide personal information such as Social Insurance Number, birth date, gender and postal code through the Participant Information Template. While voluntary, the data analysis found high response rates for key fields. Historically, partner YESS departments, agencies, and Crown corporations uploaded their participant data to ESDC through a data collection system. A data collection system is not currently available for partner departments, agencies and Crown corporations to securely uploaded their participant data to ESDC. Alternative means of incorporating partners’ evaluation results will be investigated.