Horizontal Evaluation of the Youth Employment Strategy - Career Focus Stream
On this page
Request other formats online or call 1 800 O-Canada (1-800-622-6232). If you use a teletypewriter (TTY), call 1-800-926-9105. Large print, braille, audio cassette, audio CD, e-text diskette, e-text CD and DAISY are available on demand.
Main findings and observations
Three main findings were drawn from this evaluation:
- immediately after the completion of their respective interventions, a larger share of participants was still employed as opposed to having returned to school
- for ESDC participants, Career Focus had a positive and lasting impact on their labour market attachment. Over the five-year post-participation period, the average annual earnings of participants were $5,535 higher than non-participants with similar characteristics. Positive impacts were relatively larger for men
- over a 10-year period, Career Focus (ESDC part) yielded a positive return on investment for individuals and society as a whole
From these main findings, the following observations can be drawn.
- youth interventions taking the form of a wage-subsidy can play a role in facilitating the entry of post-secondary graduates into the labour market, with associated positive long-term impacts
- youth targeted by Career Focus typically face lesser barriers to employment relative to youth targeted by Skills Link. This may explain, in part, the more positive outcomes found for Career Focus relative to Skills Link
This evaluation report presents key findings and observations regarding the Career Focus stream of the Youth Employment Strategy (hereafter referred to as the Strategy). This report was completed in compliance with the Financial Administration Act and the Policy on Results.
- In addition to the Career Focus stream, the Strategy also consisted of two other separate streams: Skills Link and Summer Work Experience. Although all streams involved youth aged 15 to 30, their respective objectives and segment of youth targeted were different, resulting in the need to present findings in separate reports.
Employment and Social Development Canada (ESDC) is the lead department working in collaboration with ten other departments and agencies on this horizontal initiative. From the program design and delivery perspective, every department is accountable for the management and implementation of their respective programs to achieve results. As the lead department, ESDC performed an oversight role, which included the coordination for an evaluation.
To that end, this report builds on the previous 2015 evaluation report. It provides a profile of participants and their immediate outcomes following participation, for those contributing departments and agencies, for which the information was available. These results are then complemented by an incremental impact and a cost-benefit analysis for ESDC Career Focus participants, who benefitted from an intervention between January 2010 and December 2011.
- The incremental impact analysis compares participants’ labour market outcomes with those of a comparison group to learn about what would have happened in the absence of the program. Results of the incremental impact analysis are then used to inform the cost-benefit analysis. This type of analysis requires the linking of participants’ administrative records to the Canada Revenue Agency and Employment Insurance databases and this could only be done for ESDC participants
- The cohort of January 2010 to December 2011 was selected to allow for the assessment of interventions’ impacts on labour attachment outcomes for a period of at least five years following their completion up to calendar year 2016 (most recent tax records released in 2018 at the time of the evaluation)
- The specific methodologies used to assess the ESDC Career Focus initiative performance are described in Annex B and in the technical report which is available upon request
Given the similarities between the target population of the various initiatives under the Career Focus umbrella, participating organizations can draw from key findings and observations from this report to inform their own initiative; taking into account their respective policy, design and delivery features.
About Career Focus
Career Focus provided funding for employers and organizations to design and deliver a range of activities that enable youth to make more informed career decisions and develop their skills. In particular, Career Focus aimed to:
- increase the supply of highly qualified people
- facilitate the transition of highly skilled young people to a rapidly evolving labour market
- promote the benefits of advanced studies
- demonstrate federal leadership by investing in the skills required to meet the needs of the knowledge economy
The main activity, under Career Focus, was to provide Career-related work experiences, through which participants would gain employment experience and skills related to their field of study or career goal. This was generally achieved via ‘wage subsidies’ to selected employers or organizations.
The Career Focus stream involved eight federal departments and agencies, who each delivered different initiatives (see Table 1 for more detailed information):
- Employment and Social Development Canada (ESDC)
- National Research Council of Canada (NRCC)
- Innovation, Science and Economic Development Canada (ISEDC)
- Environment and Climate Change Canada (ECCC)
- Global Affairs Canada (GAC)
- Natural Resources Canada (NRCan)
- Agriculture and Agri-Food Canada (AAFC)
- Canadian Heritage (PCH)
For the period 2013 to 2018, the bulk of the funding for Career Focus was allocated to the following four (4) participating federal organizations:
- About 86% of the funding envelope was allocated amongst ESDC (43%), NRCC (22%), ISEDC (12%) and ECCC (9%).
- Commensurate with the funding allocation, about 88% of all participants received support from ESDC (35%), NRCC (25%), ISEDC (18%), and ECCC (10%).
Transitioning from the Youth Employment Strategy to the Youth Employment and Skills Strategy
As part of the modernization of the Youth Employment Strategy, all three streams, Summer Work Experience, Skills Link and Career Focus, underwent a re-design to respond to a range of labour market challenges faced by youth, particularly those facing barriers to employment, under a new Youth Employment and Skills Strategy (YESS) that came into effect in June 2019.
The new Strategy continues to be delivered by 11 departments and agencies across the Government of Canada. It aims to provide more flexible employment services tailored to each individual and enhanced supports for all young Canadians as well as to broaden eligibility and offer a wider range of programs to help young people gain the skills, abilities and experience they need to get good-quality jobs.
The transition from the Youth Employment Strategy to a modernized Youth Employment and Skills Strategy took place during the preparation of this evaluation report for Career Focus. Preliminary evidence for this and the other reports on the components of the Strategy were shared with the program area in order to support policy development as it became available. In addition, findings and observations from this report will serve policy development and inform the design and delivery of similar programs.
Changes in program design
Under the modernized strategy, the Career Focus stream no longer exists independently but has been merged with the Skills Link and Summer Work Experience streams in order to offer a more integrated and flexible set of program interventions that would be more responsive to the needs of individual youth. Some YESS federal partner departments will continue to deliver initiatives similar in design to those that were previously offered under Career Focus. Lastly, the Goal Getters program has also been added as an additional program area to help youth facing barriers to complete high school and transition to post-secondary education and/or employment.
Changes to the performance framework
In addition, the modernized strategy includes a new performance framework whereby all federal partners will track common, standardized outcomes that are indicative that a youth is moving along a continuum towards employment. New outcomes include job-readiness and career advancement, which involve tracking if young Canadians are acquiring the skills employers are looking for through modernized YESS programming. The new performance framework also focuses on collecting more in-depth information about the types and combinations of interventions youth receive.
Scope and evaluation questions
- Is there a demand for Career Focus?
- Did Career Focus reach its eligible participants? What are the characteristics of Career Focus participants?
- What impacts does Career Focus have on supporting participants to obtain employment, decrease reliance on Employment Insurance or Social Assistance and improve income?
- Do results vary by region or target groups?
- What was the average cost per participant and what was the cost/benefit of participating in Career Focus?
Data sources and timeframe
To report on recent trends in the number of participants as well as actual spending, we used information from the TBS Horizontal Initiatives Database as well as ESDC’s Data Collection System and Common System for Grants and Contributions from 2013 to 2018 (including departmental annual Year-End reports when available).
Information on immediate outcomes or status within 12 weeks after participating in a Career Focus intervention was also drawn from these two data sources.
For the purposes of conducting the incremental and cost-benefit analysis for ESDC Career Focus participants, information from the Common System for Grants and Contributions for the period from January 2010 to December 2011 was used to establish a profile of participants.
This cohort of participants was used in order to properly evaluate the impact of the program on participants. These cohorts provided five years of labour market outcomes after participation to assess the effectiveness of an intervention. By using these cohorts, the evaluation can leverage the most recent tax records released in 2018 for the 2016 tax year.
Other ESDC databanks included the Employment Insurance Status Vector and Records of Employment.
Where appropriate, lines of evidence were enriched with other external sources of data and relevant literature to provide context to the analysis.
|Department||Intervention||Participants (#)||Percentage of total participants||Spending ($M)||Percentage of total spending|
|ESDC||Career Focus offered career-related work experiences to post-secondary graduates.||8,884||35%||167.1||43%|
|NRCC||Youth Employment offers financial assistance to innovative small-to-medium enterprises to hire post-secondary science, engineering, technology, business and liberal arts graduates.||6,463||25%||87.5||22%|
|ISEDC||Computer For Schools Technical Work Experience offers youth practical work internships through the refurbishment of donated digital devices across the country.
Youth Internships provided experience and capabilities related to the use of digital technologies.
|ECCC||Science Horizons Youth Internship offers opportunities in the science, technology, engineering, and mathematics (STEM) fields by providing hands-on experience with potential employers.||2,561||10%||34.4||9%|
|GAC||International Youth Internship offers the opportunity to gain professional experience through international development work.||927||4%||26.2||7%|
|NRCan||Science and Technology Internship provides funding to organizations within the natural resources sectors that hire recent graduates in natural sciences.||821||3%||11.5||3%|
|AAFC||Career Focus offers agricultural internships to recent post-secondary graduates and meaningful agriculture career-oriented work experience and skills acquisition through mentoring and coaching.||552||2%||8||2%|
|PCH||Young Canada Works at Building Careers in English and French offers international internships to help develop knowledge of French and English while acquiring and developing skills in key areas of the global labour market and language-based industries involving Canada’s official languages.
Young Canada Works at Building Careers in Heritage supports internships in heritage fields and arts administration and practice.
- Source: TBS Horizontal Initiatives Database and ESDC Data Collection System.
- * See Annex C for a more detailed description of Interventions.
Career Focus: Horizontal perspective
From 2013 to 2018, about 25,500 youth took part in a Career Focus intervention, translating in an average cost of about $15,350 per participant.
- As shown in Figures 1 and 2 , the number of participants in Career Focus interventions - and associated actual amount spent - generally increased between 2013 to 2018.
- For instance, the total number of youth increased from 3,707 in 2013 to 2014 to 6,290 in 2017 to 2018, with a commensurate increase in actual spending from $53M to $93M over the same period.
- The average cost per participant remained roughly the same over this period and stood at about $14,300 in 2013 to 2014 and $14,800 in 2017 to 2018.
- Despite the observed increase in total spending, actual spending tended to be consistently lower than planned spending (with the exemption of 2017 to 2018) over this period.
- This reflected, in part, the reallocation of federal funding between Skills Link and Career Focus and other factors such as delays in launching the calls for proposals.
Text description of Figure 1
|Department||2013 to 2014||2014 to 2015||2015 to2016||2016 to 2017||2017 to 2018|
- Sources: TBS Horizontal Initiatives Database and ESDC Data Collection System.
Text description of Figure 2
|Total spending and cost per participant||2013 to 2014||2014 to 2015||2015 to 2016||2016 to 2017||2017 to 2018|
|total actual spending (in $ millions)||$M 53||$M 64||$M 82||$M 99||$M 93|
|average cost per participant (in $)||$14,260||$15,390||$17,130||$15,060||$14,860|
- Source: TBS Horizontal Initiatives Database and ESDC Data Collection System.
In most organizations, the majority of participants were women and most were 25 years old and older.
- Generally, the distribution of Career Focus’ participants across gender was roughly in line with the portrait of post-secondary graduates in Canada (see Figure 3).
- Among recent post-secondary graduates, a larger share were women (Statistics Canada, 2013)
- Notably, for the National Research Council of Canada, the Youth Employment program consisted predominantly of men (66%). This most likely reflects the relatively higher proportion of men graduating from engineering, mathematics and computer science programs (Statistics Canada, 2013)
- On the other hand, for Global Affairs Canada, the International Youth Internship program consisted predominantly of women (78%). This most likely reflects the relatively higher proportion of women graduating from humanities-related programs (Statistics Canada, 2013)
- Generally, Career Focus participants were 25 years old and older, with no or a very small number of participants aged between 15 and 19 years old
- Still, about 14% of participants in the Innovation, Science and Economic Development Canada and 5% of participants in the Agriculture and Agri-Food Canada interventions were younger than 20 years old
Text description of Figure 3
|15 to 19||3%||2%||14%||0%||0%||0%||5%|
|20 to 24||43%||47%||44%||35%||37%||38%||41%|
|25 to 29||48%||44%||36%||55%||54%||54%||46%|
|30 and over||6%||6%||6%||9%||9%||8%||8%|
- Source: TBS Horizontal Initiatives Database; ESDC Data Collection System and Common System for Grants and Contributions. Similar data for Canadian Heritage was not available.
- * For ESDC, data is for the period from January 2010 to December 2011.
- Note: Figures for gender may not add to 100%, due to the small shares of participants who did not specify their gender.
While underrepresented groups were not generally targeted by Career Focus, some interventions were more successful in reaching them.
- Interventions delivered under the Career Focus stream generally did not specifically aim to reach underrepresented groups
- Still, interventions delivered by NRCan in the area of natural resources, ISEDC in the area of computer science and technology, AAFC in the area of agriculture and by ESDC reached relatively higher shares of Indigenous People (see Figure 4)
- Similarly, interventions delivered by GAC in the area of international development, ISEDC in the area of computer science and technology and NRCC reached relatively higher shares of visible minorities
- Lastly, ISEDC interventions reached a relatively higher share of youth with disability
Text description of Figure 4
|Person with a disability||1%||1%||5%||2%||1%||2%||1%|
- Source: TBS Horizontal Initiatives Database and ESDC Data Collection System.
- Notes: Data is for participants who self-identified; Data for Canadian Heritage was not available over the reference period.
- * For ESDC, data is for the period from January 2010 to December 2011.
Career Focus - ESDC
ESDC’s Career Focus eligibility requirements
Career Focus activities were delivered by regional and local organizations responding to calls for proposals. Following an assessment process, funding in the form of contributions was provided directly to employers/organizations or community coordinators.
- A community coordinator is a contribution recipient that receives funding to enter into its own agreements with eligible participants and employers to undertake activities that further the objectives of the Career Focus program
Eligible Career Focus activities include Career-related work experiences, through which participants can gain employment experience and skills related to their field of study or career goals. The duration of the work experience normally varied between six and twelve months. The maximum cost per participant was $20,000.
- Amounts needed to accommodate participants with disabilities would be in addition to the $20,000 maximum
To participate in Career Focus, participants must be:
- aged between 15 and 30 (inclusive) at the time of intake/selection
- a Canadian citizen, or permanent resident, or a person who has been granted refugee status in Canada
- legally entitled to work according to the relevant provincial/territorial legislation and regulations
- not in receipt of Employment Insurance (EI) benefits
Eligible service providers included and continue to include:
- Not-for-profit organizations
- Municipal governments
- Indigenous organizations (including band councils, tribal councils and self-government entities)
- For-profit organizations
- Provincial and territorial goverments, institutions, agencies and Crown Corporations
To be eligible for Career Focus funding, proposed projects had to involve a minimum of eight youth participants. Exceptions were made for rural and remote regions.
Profile of Career Focus participants – ESDC cohorts 2010 and 2011
Between January 2010 and December 2011, about 2,745 youth took part in an ESDC Career Focus intervention . As shown in Figure 5 , among participants:
- close to 3 out of 5 participants were women (56%)
- most participants were between the age of 25 and 30 (54%)
Relative to the Canadian youth population at-large, a relatively lower proportion of participants self-identified as a visible minority or a person with a disability, while the share of participants who self-identified as an Indigenous person was roughly comparable.
- 5% of participants self-identified as Indigenous relative to about 6% for the Canadian youth at large in 2016 (Census 2016)
- 12% of participants self-identified as a visible minority relative to about 27% of the Canadian youth-at-large (Statistics Canada, 2019)
- 1% of participants self-identified as having a disability relative to about 13% for the Canadian youth at large (Statistics Canada, 2019). Results from the 2017 Canadian Survey on Disability are used rather than those from the 2012 Survey, since they include persons with cognitive and mental health related disabilities (Statistics Canada, 2018)
Lastly, about 9 out of 10 participants had completed their post-secondary degree (90%) and about 8 out of 10 were single.
Text description of Figure 5
|gender||Percentage of participants|
|Age||Percentage of participants|
|15 to 19||3%|
|20 to 24||44%|
|25 to 30||54%|
|Auto-identification||Percentage of participants|
|Persons with disability||1%|
|Location||Percentage of participants|
|Marital status||Percentage of participants|
|Education||Percentage of participants|
- Source: ESDC Common System for Grants and Contributions.
- Note: Participants may belong to more than one category. Whether a participant lived in a rural or urban area is determined based on the first three digits of their postal code.
Career Focus participants across Canada
Geographically, the distribution of participants by province was roughly in line with the distribution of the Canadian youth across the country (see Table 2 ).
A majority of participants lived in Ontario and Quebec – with the smallest shares living in the
Atlantic regions and the North.
- Along an urban-rural dimension, 86% of participants lived in urban communities while 14% came from rural communities. This distribution is also roughly in line with the distribution of the Canadian youth living in rural communities across the country. (see Figure 5 )
|Newfoundland and Labrador||1.7%|
|Prince Edward Island||1.2%|
Most participating organizations/employers were in the not-for-profit sector and the majority had fewer than 10 employees
Between January 2010 and December 2011, 369 projects were funded by ESDC. Based on the information available on the organizations/employers who received funding, most organizations/employers were from the not-for-profit organizations (see Figure 6 ).
- Not-for-profit organizations included local communities, charitable, voluntary and provincial non-governmental organizations (NGOs)
Text description of Figure 6
|Sector||Share of organizations|
As shown in Figure 7 most organizations/employers who received funding were small or of medium size, with:
- about 52% having fewer than 10 employees
- 24% with 11 to 50 employees
Only 7% of organizations/employers had 100 employees or more.
Text description of Figure 7
|Number of employees||Share of organizations|
|Fewer than 10||52%|
|11 to 50||24%|
|51 to 100||5%|
|100 and more||7%|
- Source: ESDC Common System for Grants and Contributions.
- Note: Totals may not add to 100% due to rounding.
Main finding #1: Immediate outcomes
Immediately after the completion of their respective interventions, a larger share of participants was still employed as opposed to having returned to school
In line with the objective of the program, after the completion of their intervention, the share of participants who were still employed was larger than the share of participants who went back to school, as shown in Figure 9 .
Text description of Figure 8
|Return to school||3%||12%||16%||3%||6%||4%||13%||5%|
- Source: TBS Horizontal Initiatives Database; Departmental Performance Reports / Departmental Results Reports; and
- ESDC Data Collection System.
- * Figures were supplied directly by the organization in January 2020.
- ** Results reflect employment status within 2 weeks of the interns’ return to Canada from overseas.
For most organizations, the status of a relatively large share of participants - immediately after the completion of a Career Focus intervention - was not specified. As a result, it is not possible to assess with certainty whether the desired immediate outcomes of Career Focus interventions were reached.
- For certain departments, information presented in year-end reports was not complete. In particular, year-end reports were generally limited to those participants who had completed their intervention during the fiscal year and responded to the exit survey prior to the publication of their reports.
Outcomes observed immediately after the completion of an intervention were generally in line with the participants’ level of education and whether they had graduated or not. For example:
- a relatively larger share of participants in interventions delivered by NRCC, ISEDC, and AAFC went back to school after the completion of their intervention. These results are consistent with the relatively larger share of their participants not having completed post-secondary studies (see Figure 10 )
- on the other hand, in organizations with interventions mainly consisting of participants who had completed their post-secondary studies, the share of participants going back to school was small (for example, ESDC, ECCC, GAC and NRCan)
Text description of Figure 9
|Level of education||ESDC||NRCC||ISED||ECCC||GAC||NRCan||AAFC|
- Source: TBS Horizontal Initiatives Database; Departmental Performance Reports / Departmental Results Reports; and
- ESDC Data Collection System.
- * Data for Canadian Heritage was not available.
Main finding #2: Incremental Impact Analysis (ESDC participants)
For ESDC participants, Career Focus had a positive and lasting impact on their labour market attachment.
Over the 5-year post-participation period, the average annual earnings of participants was $5,535 higher than for non-participants with similar characteristics. Positive impacts were relatively larger for men.
As for the previous 2015 evaluation report, an incremental impact analysis was conducted for ESDC Career Focus participants. To that end, the Department relied on its Labour Market Data Platform using participants who received an intervention between January 2010 and December 2011. This period was selected to allow for the assessment of their impacts on labour attachment outcomes over a period of at least five years following the completion up to calendar year 2016 (most recent available information). Building on previous evaluations, the incremental impact analysis was extended and conducted at the sub-group level (for example, age and gender).
Incremental impact analysis compares participant results to those of a comparison group to learn about what would have happened in the absence of the program.
The comparison group consisted of a subset of individuals aged 15 to 30 who have the same characteristics as Career Focus participants, but had received only a limited level of treatment through minor Employment Assistance Services (EAS) interventions as part of the Labour Market Development Agreements (LMDA).
It should be noted that results from an incremental impact analysis are dependent on the observable characteristics of the participants at the time, and that it is on this set of characteristics that a comparison group is built. In practice, this means that when an incremental impact analysis is conducted for women who received a Career Focus intervention, a separate control group consisting only of women with similar observable characteristics was built. The same approach was applied for men and for each respective age category.
Therefore, caution should be exercised in over-generalizing the results.
See Annex B for more details on the methodology.
Following a Career Focus intervention, ESDC participants improved their labour market attachment through increases in earnings, incidence of employment and decrease in their use of social assistance benefits relative to a similar group of non-participants.
In particular, as shown in Table 3 , in the 5-year period following their intervention, participants’ work experience led to an average annual increase of:
- $5,535 in post-participation earnings
- 3.9 percentage points in the incidence of employment
While Career Focus led to a reduction in the receipt of social assistance benefits, the average annual decrease following the intervention was small.
Among participants, incremental positive impacts were generally larger for men and for older youth (age between 25 and 30 years old) relative to those found for women and younger participants.
- This suggests that young men may benefit more from measures seeking to facilitate the transition from post-secondary education into the labour market.
|Subgroup||Employment earnings (annual average, in $)||Incidence of employment (annual average, in percentage points)||Social assistance benefits (annual average, in $)|
|Career Focus (overall)||$5,535||3.9||-$125|
|20 to 24 years old||$5,880||2.3*||-$145|
|25 to 30 years old||$4,395||4.9||-$160|
- *Results are statistically not significant, still they are valid in terms of informing the direction of the impact (negative or positive).
Main finding #3: Cost-benefit analysis
Over a 11-year period, Career Focus (ESDC part) yielded a positive return on investment for society (participant + government).
The incremental impact analysis, presented earlier, assesses the effectiveness of interventions focusing on the first five years following an intervention (up to 2016). As a result, this type of analysis does not inform the longer term impacts that such interventions may have on participants nor whether the benefits associated with the interventions outweigh its costs. To assess the longer-term impacts associated with Career Focus, a cost-benefit analysis was conducted. To that end, observed incremental impacts five years following a Career Focus intervention were projected for an additional five years (for detailed methodology, see cost-benefit analysis technical report). This analysis reports on the net present value and the cost-benefit ratio from the government, the individual and the society’s perspectives.
- From the government’s perspective, costs are incurred up-front and consist of program costs. On the other hand, benefits accrue over time in the form of increased taxed revenues and decreased outlays from support programs (for example, social assistance)
- From the individual’s perspective, benefits accrue over time and take the form of higher after-tax earnings which may contribute to improved social outcomes
- From society’s perspective, the total net benefits of both the government’s and the individual’s perspectives are taken into account (government + individual)
Participants to ESDC Career Focus Intervention
The total social benefit (government + participant) is $46,293, yielding an average annual social rate of return of 18% over 11 years (1 year in-program, 10 years post-participation consisting of 5 years observed and 5 years projected).
- For participants, the total net benefit is $42,272
- For governments, the total net benefit is $4,021
Overall, the report showed that youth interventions taking the form of a wage-subsidy can play a role in facilitating their integration into the labour market, with associated positive long-term impacts. In particular, in the case of Career Focus, we found that:
- immediately after the completion of their respective interventions, a larger share of participants were still employed as opposed to having returned to school
- for ESDC participants, Career Focus had a positive and lasting impact on their labour market attachment, with generally larger positive impacts for men
- over a 10 year period, Career Focus (ESDC) yielded a positive return on investment for individuals and society as a whole
Findings found under this evaluation are consistent with those found in previous evaluations, The key results from the 2009 and 2015 evaluations are described in detail in Annex D.
That being said, while youth interventions taking the form of a wage-subsidy can play a role in facilitating the entry of post-secondary graduates into the labour market, this youth segment typically face lesser barriers to employment relative to the segment of youth targeted by Skills Link. This may explain, in part, the more positive outcomes found for Career Focus relative to Skills Link.
Annex A: Bibliography
Data sources: Employment and Social Development Canada. Administrative data known as the Common System for Grants.
Internal Sources : Technical Studies (not published, available on demand)
- ESDC (June 2019). Net Impact Results of the Youth Employment Strategy. Evaluation Directorate
- ESDC (June 2019). Cost-Benefit Analysis for the Youth Employment Strategy. Evaluation Directorate
- Canadian Institute for Health Information (2015). Trends in Income-Related Health Inequalities in Canada. Technical Report
- Employment and Social Development Canada (June 2015) Youth Employment Strategy Skills Link.
- Employment and Social Development Canada Departmental performance report.
- Gebel, M., & Vossemer, J. (2014) The impact of employment transitions on health in Germany: a difference-in-differences propensity score matching approach. Social Science and Medicine, 108, 128–136
- Heckman, J., Ichimura, H., & Todd, P. (1997). Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme. Review of Economic Studies, 64(4), 605-654. doi: 10.2307/2971733
- Human Resources and Social Development Canada. 2008-09 Departmental Performance Report.
- Lechner, M., & Wunsch, C. (2013). Sensitivity of matching-based program evaluations to the availability of control variables. Labour Economics, 21, 111-121
- Machin S. and C. Meghir (2000). Crime and Economic Incentives. The Institute for Fiscal Studies. Working Paper 00/17
- Nilsson A. and J. Agell (2003). Crime, unemployment and labor market programs in turbulent times. Institute for Labour Market Policy Evaluation (IFAU). Working Paper 2003:14
- Saunders, M; Barr, B; McHale, P; Hamelmann, C. (2017). Key policies for addressing the social determinants of health and health inequities. Health Evidence Network Synthesis Report 52. Regional Office for Europe. World Health Organization
- Statistics Canada (2018). Canadian Survey on Disability, the evolution of disability data in Canada: Keeping in step with a more inclusive Canada
- Statistics Canada (2019). A Portrait of Canadian Youth
- Statistics Canada (2013). Gender differences in science, technology, engineering, mathematics and computer science (STEM) programs at university
- Stuart, E.A., Huskamp, H.A., Duckworth, K., Simmons, J., Song, Z., Chernew, M., & Barry, C.L. (2014). Using propensity scores in difference-in-differences models to estimate the effects of a policy change. Health Services Outcomes Research Methodology, 14(4), 166–182. doi: 10.1007/s10742-014-0123-z
Annex B: Evaluation approach
Lines of evidence
Administrative data / Incremental impact methodology
The incremental impact analysis is conducted using the integrated Labour Market Program Data Platform (LMPDP), which contains program administrative data collected at the intervention and project level through ESDC’s Common System for Grants and Contributions database, linked to the EI benefits databank and Canada Revenue Agency (CRA) taxation files.
The data were thoroughly assessed and transformed into high-quality analytical files that included a large number of variables relevant to individuals’ labour market experiences, including socio-demographic characteristics of both participant and comparison cases (for example, age, gender, education, marital status, disability), as well as their province of residence, background qualifications (for example, occupational group, industry codes), and labour market history (for example, use of EI benefits, employment/self-employment earnings, use of social assistance, incidence of employment in the five-year pre-participation period).
The current evaluation examines the incremental impacts for participants who started a specific Career Focus intervention between January 2010 and December 2011, and follows their subsequent labour market trajectories for up to five consecutive years up to 2016, in accordance with the current availability of CRA data.
The proposed methodology is built on the same framework as the evaluation of the Labour Market Development Agreements (LMDA). The procedure for estimating incremental impacts is based on a quasi-experimental approach that measures the effectiveness of the program by comparing the actual outcomes for the participants in the interventions to their counterfactual outcomes (for example, the outcomes they would have experienced in the absence of the intervention). Because one cannot measure a given individual’s outcomes under both participation and non-participation in a Career Focus intervention, one needs an appropriate comparison group to “stand in” as the counterfactual.
In line with the same strategy applied in many previous ESDC evaluations, the comparison group consisted of a subset of individuals aged 15 to 30 who have the same characteristics as Career Focus participants, but had received only a limited level of treatment through minor Employment Assistance Services (EAS) interventions as part of the Labour Market Development Agreements (LMDA).
For all counterfactual comparisons, the incremental impact results were produced using a state-of-the-art combination of difference-in-differences estimation and propensity score kernel matching methods (Heckman et al., 1997). Given the breadth of background variables available in the linked data files, one can be reasonably confident that the impact estimates are robust (Lechner & Wunsch, 2013). For validation purposes, the results were also produced with alternative techniques (in other words inverse probability weighting and nearest–neighbour matching). This triangulation exercise confirmed that the incremental impact results were insensitive to the choice of estimation method.
The main advantage of this evaluation methodology is that the universe of the comparison group is very large, so it is possible to obtain close matches to program participants in terms of their observable characteristics. Based on consultation with external academic experts (peer reviewers), a carefully selected subset of EAS clients was the best choice in terms of data availability and the fact that ESDC has considerable experience in using it as a comparison group.
One potential limitation is the possibility of unobserved variables (for example, area of study, motivation) that may result in some degree of overestimation of the incremental impact results. Also, it should be noted the income tax data are only available on an annual basis with a two-year lag, which means outcomes can only be tracked on a yearly basis.
The reference period for the cost-benefit analysis consists of participants who started their intervention between January 2010 and December 2011. This analysis examined labour market outcomes during the year of participation in the intervention and then continued to observe the labour market outcomes for five years after the participants had completed their intervention. Observed outcomes over the five-year post-participation period are then projected of an additional five years, for a total period of 11 years.
The cost-benefit analysis reports on the incremental present value and the cost-benefit ratio from the individual’s, the government’s and the society’s (individual + government) perspectives.
An accounting framework determined who bore a particular cost or benefit. All benefits were discounted (by a rate of 5%) after the participation start year.
Limitations: The cost-benefit analysis is based on major quantifiable costs and benefits available in the administrative database and Employment Insurance Monitoring and Assessment Report. It does not include intangible benefits such as higher well-being for individuals related to better health outcomes and reduced criminality.
In addition to producing improvements in labour market indicators, it is likely that active labour market programming such as Career Focus also has positive influences on additional social outcomes such as health and crime. According to research evidence, labour market participation and higher income is associated with better health outcomes (for example, Canadian Institute for Health Information, 2015; Saunders & al. 2017) and lower risk of criminal activity (for example, Nilsson & Agell, 2003; Machin & Meghir, 2000). Therefore, expanding the cost-benefit framework to include potential savings resulting from improved health outcomes (in other words, reduced health system expenditures) and decreases in criminal justice system contact (in other words, reductions in the costs of policing, courts, and correctional services) could yield additional positive impacts for both individuals and the government beyond what is directly measured.
This report was informed by the review of relevant documents, including program documents from the participating Departments.
Annex C: Description of Career Focus Interventions
Career Focus activities are designed to:
- enable youth to acquire and enhance skills, which include but are not limited to employability skills and advanced employability skills
- provide work experiences, mentoring and coaching
- support youth entrepreneurs to gain self-employment
- help youth obtain skills acquired while participating in exchanges between post-secondary institutions, including higher education institutions in other countries where reciprocal exchange of people, knowledge and expertise will occur
- support youth in making informed career decisions, promote the value of education, and promote youth as the labour force of the future
- support research and innovative projects to identify better ways of helping youth prepare for, return to, and keep employment and to be productive participants in the labour force
Of the three YES streams, Career Focus has the largest number (7) of federal departments or agencies involved in its delivery apart from ESDC:
- Agriculture and Agri-Food Canada
- Canadian Heritage
- Environment and Climate Change Canada
- Global Affairs Canada
- Innovation Science and Economic Development Canada
- National Research Council Canada
- Natural Resources Canada
Agriculture and Agri-Food Canada: the Career Focus Program (CFP) is intended to create approximately 50 agricultural internships annually for recent post-secondary graduates across the country. Applications are received from employers who can offer meaningful agriculture career-oriented work experience and skills acquisition through mentoring and coaching to interns. Internships must be between 4 and 12 months in length and must last long enough to give the employee significant work experience.
To be eligible as an intern, an individual must be a Canadian citizen or landed immigrant between the ages of 15 and 30; be unemployed or under-employed; and have graduated in the last three years from an agriculture-related program in a university, college, CEGEP or a provincial program focusing on biological, agricultural and veterinary science or applied technology.
Canadian Heritage: there are two program components within the Career Focus Stream - Young Canada Works at Building Careers in English and French (YCW-BCEF), and Young Canada Works at Building Careers in Heritage (YCW-BCH).
YCW-BCEF aims to provide unemployed or under-employed college or university graduates under 30 years old with international internships to help them develop their knowledge of French and English while acquiring and developing skills in key areas of the global labour market and language-based industries involving Canada’s official languages. It is delivered by one national non-for-profit youth organization.
YCW-BCH supports internships in heritage fields and arts administration and practice for unemployed or under-employed college and university graduates aged 30 years and under. The program aims at increasing the interns’ professional skills and practical experience in career-oriented jobs both in Canada and abroad; at helping them make the transition to heritage and arts-oriented trades and professions, as well as earn money for additional studies. It is delivered nationally by 6 professional service organizations, each specializing in one sector of the Canadian not-for-profit heritage community (museums, archives, libraries, heritage sites, and arts organizations).
Environment and Climate Change Canada: the Science Horizons Youth Internship program began in 1997, with the original launch of the Youth Employment Strategy. The program objective is to create opportunities for youth in the science, technology, engineering, and mathematics (STEM) fields by providing them with hands-on experience with potential employers.
Global Affairs Canada: developed in 1997, the International Youth Internship Program (IYIP) was designed to offer young Canadians, between the ages of 19 and 30, the opportunity to gain professional experience through international development work. To be eligible, interns must be a post-secondary graduate, graduate of a degree or diploma program in a university, college, post-secondary school of technology, post-secondary institute or a CEGEP, out of school, or unemployed or underemployed.
The IYIP contributes to Canada’s Youth Employment Strategy by providing a large spectrum of Canadian graduates with a valuable international development work experience. IYIP supports internships that are designed and implemented in consultation with both Canadian and developing country organizations. The internships carry out sustainable development efforts that meet locally defined needs in Global Affairs Canada’s sectors of focus and aim to provide and share valuable knowledge and skills in order to ensure sustainability.
Innovation Science and Economic Development Canada: The Technical Work Experience Program (TWEP) is a companion program to the Computer For Schools program (a national, partnership-based program that refurbishes surplus computers – from federal departments, provincial-territorial governments and the private sector – for use by the program’s beneficiaries), and supports youth interns working in refurbishment centres across the country. As part of the Youth Employment Strategy, TWEP helps bridge youth into the workplace and provides much-needed technical skills training. Candidates are engaged in the refurbishment process, and develop skills in computer repair and software testing while cultivating softer skills such as time management and team building.
The Youth Internships Program (YIP) was a national initiative funded through the YES. It was intended to provide Canadian youth with work experience and digital skills to help end the cycle of no experience/ no work, to encourage the development of digital skills in youth and to encourage not-for-profit groups across Canada to develop and use digital technologies in their organizations. The overall objective of YIP was to provide Canadian youth with experience and capabilities related to the use of digital technologies, thereby making them more productive and competitive in a digital and knowledge-based economy and more successful in the job market. YIP was sunset at the end of Fiscal Year 2016 to 2017.
National Research Council of Canada: The Youth Employment Program, launched in 1997, provides financial assistance to innovative small-to-medium enterprises (SMEs) in Canada to hire post-secondary science, engineering, technology, business and liberal arts graduates. SMEs benefit from skills and knowledge of graduates while providing them with a valuable work experience that will open doors to the future. Graduates work on innovative projects within the SME environment and may participate in research, development and commercialization of technologies.
In some cases, SMEs may be collaborating with the National Research Council of Canada institute or with Innovation, Science and Economic Development’s Communications Research Centre. In addition to meeting the needs of innovative SMEs, this program facilitates the transition of highly skilled young people to a rapidly changing labour market. To qualify, SMEs must be incorporated and for-profit, ready to enhance their innovation capacity, and willing to establish a trusting relationship with the National Research Council of Canada.
Natural Resources Canada: The Science and Technology Internship Program (STIP) provides funding to organizations within the natural resources sectors that hire recent graduates in natural sciences. It helps organizations to develop Canada’s knowledge economy and increase the pool of versatile and highly qualified youth who will be tomorrow’s leaders in research and development (R&D). Since 1997, Natural Resources Canada has helped support some 600 internships. Annually, STIP creates some 50 to 80 internships that benefit both organizations and recent graduates in natural sciences.
Annex D: Key findings from previous evaluations
- 2009 Summative evaluation
- Participants expressed high levels of satisfaction with the opportunities for skill development offered by Career Focus. Employers also concurred with participants that the program assisted considerably in enhancing important employability skills.
- 2015 Summative evaluation
Over half of Career Focus survey respondents (59%) identified their decision to return to school as being influenced by their participation in the program.
When compared to participants from the limited treatment group, results from the incremental impact analysis indicated that Career Focus participants experienced significantly higher cumulative employment earnings over the six year period.
Report a problem or mistake on this page
- Date modified: