Evaluation of the Canada-Alberta Labour Market Development Agreement
On this page
- List of abbreviations
- List of charts
- List of tables
- Executive summary
- Management response
- Introduction
- Labour Market Development Agreements
- Methodology
- Evaluation findings
- Supplemental studies
- Conclusions and recommendations
- References
- Appendix A. List of 7 studies included in the Canada-Alberta synthesis report
Alternate formats
Evaluation of the Canada-Alberta Labour Market Development Agreement [PDF - 984 KB]
Large print, braille, MP3 (audio), e-text and DAISY formats are available on demand by ordering online or calling 1 800 O-Canada (1-800-622-6232). If you use a teletypewriter (TTY), call 1-800-926-9105.
List of abbreviations
- CEAS
- Career Employment Assistance Services
- EBSM
- Employment Benefits and Support Measures
- EI
- Employment Insurance
- ESDC
- Employment and Social Development Canada
- FLS/OT
- Foundational Learning Supports/Occupational Training
- LMDA
- Labour Market Development Agreements
- SA
- Social assistance
List of charts
- Chart i. Change in probability of being employed in participants relative to non-participants (annual average)
- Chart ii. Employment earnings of participants relative to non-participants (annual average)
- Chart iii. Change in dependence on government income support (annual average)
- Chart 1. Change in probability of being employed in participants relative to non-participants (annual average)
- Chart 2. Employment earnings of participants relative to non-participants (annual average)
- Chart 3. Change in the use of EI benefits (annual average)
- Chart 4. Change in the use of SA benefits (annual average)
- Chart 5. Change in dependence on government income support (annual average)
- Chart 6. Average earnings for active claimant participants in Workplace Training Program
- Chart 7. Average earnings for former claimant participants in Workplace Training Program
- Chart 8. Average earnings for active claimant participants in FLS/OT-Apprentices
- Chart 9. Average earnings for former claimant participants in FLS/OT-Apprentices
List of tables
- Table i. Share of LMDA funding and average cost per Action Plan Equivalent per participant in Alberta for 2010 to 2012
- Table ii. Number of years for the benefits to exceed FLS/OT costs
- Table 1. Share of LMDA funding and average cost per Action Plan Equivalent per participant in Alberta
- Table 2. Profile of active and former EI claimant participants in 2010 to 2012 in Alberta
- Table 3. Employment and earning levels, and use of SA in the year before participation in EBSMs
- Table 4. Profile of female participants in Alberta in 2010 to 2012
- Table 5. Incremental impacts for female participants (annual average)
- Table 6. Profile of male participants in Alberta in 2010 to 2012
- Table 7. Incremental impacts for male participants (annual average)
- Table 8. Profile of youth participants in Alberta in 2010 to
- Table 9. Incremental impacts for youth participants (annual average)
- Table 10. Profile of older worker participants in Alberta in 2010 to 2012
- Table 11. Incremental impacts for older worker participants (annual average)
- Table 12. Profile of recent immigrant participants in Alberta in 2010 to 2012
- Table 13. Incremental impacts for recent immigrant participants in CEAS (annual average)
- Table 14. Profile of visible minority participants in Alberta in 2010 to 2012
- Table 15. Incremental impacts for visible minority participants in FLS/OT (annual average)
- Table 16. Incremental impacts for FLS/OT participants in Calgary (annual average)
- Table 17. Incremental impacts for FLS/OT participants outside of Calgary (annual average)
- Table 18. Cost-benefit results for active and former EI claimant participants
- Table 19. Profile of FLS/OT- Apprentices participants in Alberta in 2010 to 2012
Executive summary
The Canada-Alberta Labour Market Development Agreement (LMDA) is a bilateral agreement between Canada and Alberta for the design and delivery of Employment Benefits and Support Measures (EBSMs).
The objective of EBSMs is to assist individuals to obtain or keep employment through various active employment programs, including training or employment assistance services. Successful delivery of EBSMs is expected to result in participants receiving needed services, a quick return to work, and savings to the Employment Insurance (EI) account.
Programs and services delivered by Alberta have to correspond to the EBSM categories defined under the EI Act. The following is a brief description of the EBSMs examined in the evaluation:
Foundational Learning Supports / Occupational Training (FLS/OT) helps participants obtain employment skills by giving them financial assistance that enables them to select, arrange and pay for classroom training.
Workplace Training Program helps participants obtain on-the-job work experience by providing employers with financial assistance to help with the wages of participants.
Self-Employment provides financial assistance and business planning advice to participants to help them start their own business.
Integrated Training is contract and tuition-based training with work-experience program. The program has a sub-component called Immigrant Bridging that helps skilled immigrants gain employment in their original occupation or a related occupation.
Career Employment Assistance Services (CEAS) help participants through activities such as individual counselling, action planning, help with job search skills, job-finding clubs, job placement services, the provision of labour market information, and case management and follow-up.
Labour Market Partnerships aim to support an industry-led approach to human resource development tailored to meet the labour market needs of industry sectors.
The LMDA investment
In fiscal year 2020 to 2021, Canada transferred nearly $192.4 million to Alberta.Footnote 1
Table i provides an overview of the share of funding allocated to EBSMs and the average cost per participant in Alberta. The average cost per participant is calculated based on the 2010 to 2012 data from the EI Monitoring and Assessment Reports. The 2010 to 2012 period corresponds with the cohort of participants selected for incremental impacts and cost-benefit analysis in the LMDA evaluation.
Employment Benefits and Support Measures | Average share of funding | Average cost – active claimants | Average cost – former claimants |
---|---|---|---|
Foundational Learning Supports / Occupational Training | 66% | $4,816 | $4,915 |
Career Employment Assistance Services | 26% | $123 | $123 |
Integrated Training | 5% | $9,503 | $9,485 |
Self-Employment | 2% | $8,806 | $9,057 |
Workplace Training Program | 1% | $8,812 | $8,891 |
Labour Market Partnerships | 1% | n/a | n/a |
Sources: EI Monitoring and Assessment Reports for fiscal years 2010 to 2011 and 2011 to 2012.
Note: Total spending do not add up to 100% due to rounding.
Compared to the 2010 to 2012 period, the LMDA budget allocation varied for few programs and services in 2020 to 2021. For example, investments in FLS/OT decreased from 66% to 53%. Moreover, investments in CEAS increased from 26% to 33% of total allocation.
Evaluation objectives
Building on the success of previous LMDA evaluation cycles, the aim of this evaluation is to fill in knowledge gaps about the effectiveness, efficiency, as well as design and delivery of EBSMs in Alberta.
Evaluation methodology
The findings in this report are drawn from 7 separate evaluation studies carried out at the provincial level. These studies examine issues related to program effectiveness, efficiency, and design and delivery. A mix of qualitative and quantitative methods are used, including:
- incremental impact analysis for participants who began an intervention between 2010 and 2012
- outcome analysis
- cost-benefit analysis (including savings to health care)
- key informant interviews with 43 Alberta program officials, contract services coordinators, training providers, and project holders
- questionnaires completed by provincial officials
- document and literature reviews
The incremental impacts are estimated for 2 types of EI claimants:
- active claimants are participants who started an EBSM intervention while collecting EI benefits
- former claimants are participants who started an EBSM intervention up to 3 years after the end of their EI benefits
Main findings
Nearly 73,000 EI active and former claimants began participating in Canada-Alberta LMDA programs and services between 2010 and 2012.
Effectiveness and efficiency of EBSMs
Overall, incremental impacts demonstrate that participants in FLS/OT improve their labour market attachment (employment and earnings) and reduce their dependence on government income supports (that is, the combination of EI and social assistance) compared to similar non-participants.
FLS/OT also improves the labour market attachment of most subgroups of active and former EI claimant participants. Most former EI claimants also reduce their dependence on government income supports following participation.
A regional analysis of FLS/OT found that former claimant participants outside of Calgary improved their labour market attachment and reduced their dependence on government income supports compared to similar non-participants. Active claimant participants in Calgary and outside of Calgary increase their incidence of employment relative to their respective comparison groups. Other results were mixed and not statistically significant.
Moreover, the social benefits of participating in FLS/OT exceed the costs of investments over time.
Active EI claimants in CEAS alone have a small negative, but not statistically significant, impact on the probability of being employed, as well as decreases in employment earnings. These participants also increase their dependence on government income supports.
Chart i presents the incremental impacts on the incidence of employment for active and former claimant participants. The estimates can be interpreted as a chance in the probability of being employed following participation. For example, participation in FLS/OT increases the probability of being employed by 6.7 percentage points for active EI claimants relative to non-participants.
Text description – Chart i
Program name | Incidence of employment for active claimants (percentage points) | Incidence of employment for former claimants (percentage points) |
---|---|---|
Foundational Learning Supports / Occupational Training | 6.7 | 5.7 |
Career Employment Assistance Services | -0.5* | n/a |
Note: Impacts are estimated over 4 post-program years for FLS/OT and 5 years post-program for CEAS.
* The impact is not statistically significant. However, CEAS participants experienced a decrease in the incidence of employment in years 3 and 5 post-participation (1 and 1.1 percentage points respectively).
Chart ii presents the annual average change in employment earnings for active and former claimants over the post-participation period. For example, former EI claimants in FSL/OT earn, on average, $1,705 more than similar non-participants.
Text description – Chart ii
Program name | Employment earnings of active claimants ($) | Employment earnings of former claimants ($) |
---|---|---|
Foundational Learning Supports / Occupational Training | 1,2851 | 1,705 |
Career Employment Assistance Services | -1242 | n/a |
Note: Impacts are estimated over 4 post-program years for FLS/OT and 5 years post-program for CEAS.
1 The impact is not-statistically significant. However, a statistically significant increase in earnings ($1,724) was found in year 2 post-program participation.
2 The impact is not-statistically significant. However, a statistically significant decrease in earnings (-$488) was found in year 1 post-program.
Chart iii presents the change in dependence on government income supports for active and former claimants over the post-participation period. For example, former EI claimants in FSL/OT reduced their dependence on government income support by 2.4 percentage points.
Text description – Chart iii
Program name | Dependence on government income support for active claimants (percentage points) | Dependence on government income support for former claimants (percentage points) |
---|---|---|
Foundational Learning Supports / Occupational Training | -0.7* | -2.4 |
Career Employment Assistance Services | 2.6 | n/a |
Note: Impacts are estimated over 4 post-program years for FLS/OT and 5 years post-program for CEAS.
*The impact is not statistically significant over the entire post-program period. However, participants did reduce their dependence on government income support by 1.4 percentage points in year 3 post-program.
Table ii presents the number of years required for the social benefits to exceed FLS/OT program costs. Social benefits to participation exceed investment costs in a period ranging from 7.2 for former claimants to 16 years for active claimants.
Indicator | FLS/OT active claimants (10 years post-program) | FLS/OT former claimants (10 years post-program) |
---|---|---|
Payback period (years after end of participation) | 7.2 | 16 |
Supplemental studies
A series of supplemental studies address information gaps previously identified in LMDA evaluations regarding the design and delivery, challenges and lessons learned for Self-Employment, Integrated Training, Labour Market Partnerships and FLS/OT-apprentices. Using a mix of qualitative and quantitative methods, questions regarding design and delivery, lessons learned, and challenges are examined in detail. When relevant, key considerations are included to help guide future program and policy discussions.
Self-Employment study
The Self-Employment program aims to assist participants in creating employment for themselves by providing them with a range of services including:
- assistance with business plan development
- counselling, coaching and mentoring
- entrepreneurial training and workshops
According to key informants, program participants started businesses in a variety of industries, including construction, food and beverage industry, marketing, health and personal services, and other services including bookkeeping.
Among the factors that contributed to launching and maintaining self-employment businesses, key informants identified:
- adequate planning including having clear objectives, tools needed to launch and manage the business, setting and meeting deadlines
- participant level of commitment and persistence
- adequate assessment by the training provider such as viability of the business, entrepreneurial skills, and motivation
- mentoring and peer support from other participants
- ability to network
Key informants identify the following reasons why program participants fail at starting and maintaining their self-employment businesses:
- insufficient financial resources
- lack of commitment or persistence
- lack of personal or family support
- lack of self-confidence
- economic conditions
- having unrealistic expectations or underestimating the level of required efforts
- poor labour market research, poor business idea, and low demand for the product or service offered
- poor management skills
- personal family or health issues
Integrated Training studyFootnote 4
Alberta implements a contract and tuition-based training with work-experience program through Integrated Training.
The objectives of Integrated Training are to:
- enable unemployed or marginally employed adult Albertans to secure and maintain employment
- provide training and occupation-related skills recognized by industry/employers
The Immigrant Bridging program (a sub-component of Integrated Training) helps skilled immigrants gain employment in their original occupation or a related occupation. The design and delivery of Integrated Training allows Alberta to address a variety of barriers to employment experienced by its residents. Integrated Training can also be used to address labour market needs by targeting sub-groups of individuals, occupations or economic sectors in demand, and to a limited extent, communities.
In addition to gaining work experience, key informants identified a variety of other benefits that can be expected from Integrated Training programs. Participants are expected to develop new skills and job search abilities. For immigrant participants, benefits associated with transitioning to a new labour market are also expected.
Benefits for employers who provide training with work experience are associated with:
- gaining a potential source of trained employees
- helping people in need
- informing the design of training programs to suit their labour needs
Key informants identify the following as contributing factors to participant success:
- participant assessment that is rigorous
- work experience opportunities that have a duration catered to participants’ needs
- ensuring program flexibility and supports are in place to assist persons with mental health issues, disabilities, lack of confidence, and multiple barriers to employment
- training providers most likely to succeed are those with experience delivering Integrated Training and those with the staff capacity to address the barriers to employment of participants
Labour Market Partnerships study
The Labour Market Partnerships program aim to assist employers, communities and/or industries to address their labour force adjustments and human resource needs. It includes a wide range of funded activities, such as:
- coordination to facilitate community problem-solving relating to Indigenous employment
- career fairs
- conduct a labour market environmental scan
- develop an industry-based workforce plan to ensure employee skills are maintained
- design and deliver workshops to improve the ability of immigrant serving organizations to integrate immigrants into the labour force
- initiate discussions regarding local challenges and opportunities relating to new industries emerging in the community
- analyse potential training needs and employment opportunities
Labour Market Partnerships projects targeted current and/or forecasted skills and/or labour shortages. These projects also targeted specific populations (for example, women, Indigenous peoples, and newcomers).
Generally, funded projects target labour market issues associated with:
- technological changes in the industry
- businesses downsizing/closure
- limited employment opportunities in Indigenous, small and remote communities
- barriers to employment experienced by a target population
Alberta’s Ministry of Labour and Immigration confirmed that program officials conducted activities to support the formation and maintenance of partnerships as a part of the program design and delivery. The Ministry and project holders explained that partners’ expertise and contributions are all essential to project implementation and success.
FLS/OT-Apprentices study
The objective of the program is to help apprentices become skilled tradespeople and to increase their labour market attachment. Program participants have generally chosen a career and are already attached to the labour market. The apprenticeship process involves on-the-job learning and technical training in a classroom setting.
The evaluation found that active EI claimant participants increase their average earnings from $22,746 in the fifth year pre-program to $66,665 in the fifth year after the program start year. Former EI claimants increase their average earnings from $24,792 in the fifth year pre-program to $67,559 in the fifth year after the program start year. After participating in the program, both active and former claimants also decrease their dependence on government income supports.
Recommendations
Since 2012, 15 qualitative and quantitative evaluation studies have been used to address issues and questions related to EBSM design, delivery and effectiveness. The quantitative studies successfully assessed the effectiveness and efficiency of EBSMs by producing incremental impacts and cost-benefit analysis. The qualitative studies identified specific challenges, lessons learned, and best practices associated with the design and delivery of EBSMs. Each study included key considerations for program and policy development or recommendations.
In addition, the recently completed evaluation of the Workforce Development Agreements complements the LMDA qualitative studies. This evaluation was also supported by literature reviews and provided unique insights into challenges and lessons learned to assist persons with disabilities, immigrants and those further removed from the labour market.
Most results from this evaluation stem from the conduct of advance causal analysis whereby impacts found could be attributed to a specific EBSM. These analyses are predicated on having access to high quality administrative records, thereby confirming the importance of the capacity to leverage and integrate relevant administrative data.
From these main findings, 2 key recommendations emerge.
Recommendation #1: Alberta is encouraged to share and discuss lessons learned, best practices and challenges associated with the design and delivery of programs and services. Discussions are encouraged with ESDC, at the bilateral or multilateral levels, as well as with service delivery network if necessary.
Recommendation #2: Alberta is encouraged to pursue efforts to maintain and strengthen data collection provisions in support of reporting, performance measurement and data-driven evaluations at the national and provincial levels.
Management response
Introduction
The Government of Alberta would like to thank all those who participated in the Cycle III Evaluation of the Canada-Alberta LMDA. In particular, the Government of Alberta acknowledges the contributions of key informants, ESDC and the work of the LMDA Evaluation Steering Committee in conducting the assessment of EBSMs in Alberta, funded under the LMDA.
Alberta Labour and Immigration, on behalf of the Government of Alberta, considered the evaluation findings in this report and provided the following Management Response.
Alberta management response
The Evaluation Report indicated that Alberta is meeting the objective of assisting individuals to obtain or keep employment through various active employment programs, including training or employment assistance services under this agreement. Overall, findings demonstrate improvement in the labour market attachment of participants and reduce dependence on government income supports.
Alberta is committed to continue to achieve strong results and look for ways to improve on the programs it provides under the LMDA and is appreciative of the new evidence and information that is available. Information from the Bilateral Evaluation Report will be helpful and will be considered as program and service changes occur.
Alberta is currently addressing both key recommendations from the main findings.
Recommendation #1: Alberta is encouraged to share and discuss lessons learned, best practices and challenges associated with the design and delivery of programs and services. Discussions are encouraged with ESDC, at the bilateral or multilateral levels, as well as with service delivery network if necessary.
- Lessons learned, best practices and challenges associated with the design and delivery of programs and services are being discussed and shared with service delivery network
- Jurisdictional scan was also conducted on the implementation of Labour Market Partnerships that encouraged sharing of best practices and lessons learned with other jurisdictions, together with ESDC
Recommendation #2: Alberta is encouraged to pursue efforts to maintain and strengthen data collection provisions in support of reporting, performance measurement and data-driven evaluations at the national and provincial levels.
- Alberta also implemented a co-located database where participant-level data for most of the programs under LMDA are stored. A cross-ministry working group was created as a mechanism to govern and maintain the integrity of the data collected. These data are currently used not only for reporting purposes, but also in the conduct of data-driven evaluations at the provincial level
Introduction
Employment and Social Development Canada (ESDC) worked jointly with Alberta and 11 other provinces and territories to complete the third evaluation cycle (2018 to 2023) for the Canada-Alberta Labour Market Development Agreement (LMDA).
The first evaluation cycle of the Canada-Alberta LMDA was conducted from 1998 to 2012. It involved the conduct of separate formative and summative evaluations under the guidance of a bilateral Joint Evaluation Committee.
Building on lessons learned and best practices from the first cycle, the second cycle of LMDA evaluations was undertaken between 2012 and 2017. The second cycle was designed and implemented under the guidance of a federal-provincial/territorial LMDA Evaluation Steering Committee. The work was supported by bilateral discussions at the Joint Evaluation Committee.
The third LMDA evaluation cycle builds on the success of the second cycle. The aim is to fill in knowledge gaps about the effectiveness, efficiency, and design and delivery of Employment Benefits and Support Measures (EBSMs). The evaluation cycle was designed and implemented under the guidance of a federal-provincial/territorial LMDA Evaluation Steering Committee composed of ESDC and 12 participating provinces and territories.
This report presents a summary of the findings from 7 studies specific to Alberta.
Labour Market Development Agreements
The LMDAs are bilateral agreements between Canada and each province and territory for the design and delivery of EBSM programs and services. They were established under Part II of the 1996 Employment Insurance (EI) Act.
In fiscal yearFootnote 5 2020 to 2021, Canada transferred nearly $192.4 million to Alberta.Footnote 6 Under the Agreement, Alberta is responsible for the design and delivery of programs and services aimed at assisting individuals to prepare for, obtain, and maintain employment.
LMDA programs and services are classified under 2 categories:
- Employment benefitsFootnote 7, Footnote 8 fall into 5 sub-categories:
- Skills Development (Foundational Learning Supports/Occupational Training)
- Targeted Wage Subsidy (Workplace Training Program)
- Self-Employment
- Job Creation Partnerships (Integrated Training)
- Targeted Earnings Supplements
- Support measures fall under 3 sub-categories:
- Employment Assistance Services (Career Employment Assistance Services)Footnote 9
- Labour Market Partnerships
- Research and Innovation
Alberta has the flexibility to adapt EBSMs to its jurisdiction’s context as long as they are consistent with Part II of the EI Act.Footnote 10
The objective of EBSMs is to assist individuals to obtain or keep employment through various active employment programs, including training or employment assistance services. Successful delivery of EBSMs is expected to result in participants receiving needed services, a quick return to work, and savings to the EI account.
Programs and services examined in this study include employment benefits and support measures.
Employment benefits
Employment benefits programs and services examined in this study include:
Foundational Learning Supports/Occupational Training (FLS/OT) help participants obtain employment skills by giving them financial assistance that enables them to select, arrange and pay for classroom training.
Workplace Training Program subsidizes the wages of individuals whom employers would not ordinarily hire.
Integrated Training is a contract and tuition-based training with work-experience program. The program has a sub-component called Immigrant Bridging that helps skilled immigrants gain employment in their original occupation or a related occupation.
Self-Employment provides financial assistance and business planning advice to EI-eligible participants to help them start their own business. This financial assistance is intended to cover personal living expenses and other expenses during the initial stages of the business.
Support measures
Support measures programs and services examined in this study include:
Career Employment Assistance Services (CEAS) support individuals as they prepare to enter or re-enter the workforce or assist them to find a better job. Services can include job search services, career development and counselling, and résumé writing assistance. These services are referred to as ‘light touch intervention’ due to their short duration. They can be provided on a one-on-one basis, or in a group setting. A typical intervention lasts less than 1 day, but a participant may receive multiple short interventions over a few weeks. These services are generally provided in combination with more intensive Employment Benefit interventions.Footnote 11
Labour Market Partnerships provide funding to help employers, employee and employer associations, and communities improve their capacity to deal with human resource requirements and implement labour force adjustments. These partnerships involve developing plans and strategies and implementing labour force adjustment measures.
Eligible participants covered in this study
The outcomes incremental impacts are estimated for active and former EI claimants:
- active claimants are participants who started an EBSM intervention while collecting EI benefits
- former claimants are participants who started an EBSM intervention up to 3 years after the end of their EI benefits Footnote 12
Average EBSM share of funding and cost per Action Plan Equivalent
Table 1 provides an overview of the share of funding allocated to EBSMs and the average cost per Action Plan Equivalent per participant in Alberta. It is noted that the average cost per participant is calculated based on the 2010 to 2012 data from the EI Monitoring and Assessment Reports. The 2010 to 2012 period corresponds with the cohort of participants selected for incremental impacts and cost-benefit analysis in the Canada-Alberta LMDA evaluation.
From the 2010 to 2012 period to the 2020 to 2021 fiscal year, investments in FLS/OT decreased by 13 percentage points. The largest increases in funding are noted for CEAS (+7 percentage points) and for the Workplace Training Program (+8 percentage points).
Employment Benefits and Support Measures | Share of funding (2010 to 2012) | Share of funding (2020 to 2021) | Average cost - active EI claimants (2010 to 2012) | Average cost - former EI claimants (2010 to 2012) |
---|---|---|---|---|
Foundational Learning Supports / Occupational Training | 66% | 53% | $4,816 | $4,915 |
Career Employment Assistance Services | 26% | 33% | $123 | $123 |
Integrated Training | 5% | <1% | $9,503 | $9,485 |
Self-Employment | 2% | 1% | $8,806 | $9,057 |
Workplace Training Program | 1% | 9% | $8,812 | $8,891 |
Labour Market Partnerships | 1% | 2% | n/a | n/a |
Sources: EI Monitoring and Assessment Reports for fiscal years 2010 to 2011, 2011 to 2012 and 2020 to 2021.
Note: Total spending do not add up to 100% due to rounding.
Methodology
This section presents key aspects of the quantitative analyses carried out as part of the LMDA studies.
All quantitative analyses are based on administrative data from the EI Part I (EI claim data) and Part II (EBSM participation data). The EI Part I and II data are then linked to the T1 and T4 taxation files from the Canada Revenue Agency. Incremental impact and cost-benefit analyses are based on 100% of participants in Alberta who began their EBSM participation in 2010 to 2012.
The 2010 to 2012 timeframe was selected to assess the impacts of EBSMs in the years following participation. Impacts were assessed over a period of at least 4 years after program completion up to the 2017 calendar year (most recent available information at the time of this evaluation).
Incremental impacts analysis Footnote 15
Program effectiveness is assessed by estimating incremental impacts from EBSM participation on participants’ labour market experience. That is, earnings from employment and self-employment, incidence of employment, use of EI, use of social assistance (SA), and dependence on government income supports after participation.
The role of the incremental impact analysis is to isolate the effects of participation from other factors. To achieve this, the incremental impact analysis compares the labour market experience of participants before and after their participation with that of similar non-participants. Figure 1 presents an example of incremental impact calculation.
Text description – Figure 1
Incremental impacts are the difference in the pre-post participation outcomes between participants and similar non-participants. For example:
- if the average annual earnings of participants are $30,000 before participation and $38,000 after participation, the change in earning for participants is an increase of $8,000
- if the average annual earnings in the comparison group are $31,000 before participation and $36,000 after participation, the change in earnings for the comparison group is an increase of $5,000
- the incremental impact, which is the change attributed to program participation, is $8,000 minus $5,000, which equals an increase of $3,000
The main estimator used is propensity score kernel matching technique combined with difference-in-differences estimator. Moreover, 3 different state-of-the-art estimation techniques (Inverse Probability Weighting, Nearest Neighbour and Cross-sectional Matching) were carried out separately for each type of EBSMs and EI claimants in order to validate the impact estimates.
As for previous LMDA evaluation studies, the Action Plan Equivalent is the unit of analysis used. Action Plan Equivalents regroup all EBSMs received by an individual within less than 6 months between the end of an EBSM and the start of the next. Action Plan Equivalents are categorized based on the longest EBSM they contain, except for CEAS-only Action Plan Equivalents which include only CEAS interventions.
The matching of participants and comparison group members used up to 75 socio-demographic and labour market variables observed over 5 years before participation. Two different comparison groups were used to measure impacts for active and former EI claimants
For active claimants, incremental impacts were measured relative to a comparison group of active claimants who were eligible to, but did not, participate in EBSMs during the reference period.
For former claimants, the comparison group was created using individuals who participated in CEAS only during the reference period.Footnote 16 In other words, the experience of former claimants in FLS/OT interventions is compared to the experience of former claimants who received CEAS only. This is a conservative approach given the fact that participation in CEAS can lead to limited effects on labour market outcomes.
Due to this difference in measurement, incremental impacts estimated for active claimant participants should not be directly compared to those of former claimant participants.
Impacts are generated over 4 years for FLS/OT, while a fifth year is estimated for participants in CEAS. Footnote 17
Factors accounted for in the cost-benefit analysis Footnote 18, Footnote 19
Building on the results of the incremental impacts, program efficiency is assessed through a cost-benefit analysis. The analysis compares the participants’ cost of participating and the government’s cost of delivering the program to the benefits associated with the program. Overall, this analysis provides insights on the extent to which the program is efficient for the society (that is, for both participants and the government).
Sources of data and information
The analysis considers all the quantifiable costs and benefits directly related to EBSM delivery and participation that can be measured given the information available. The analysis is comprehensive in that it accounts for the vast majority of possible direct costs and benefits.
However, the analysis does not account for all costs and benefits. For example, there are factors that can lead to an understatement of the benefits (for example, positive spillovers to other family members) and other factors that can lead to an overstatement of the benefits (for example, effects on skill prices or displacement).
This study relied on integrated data from the EI Part I and II Databank and Income Tax records from the Canada Revenue Agency. Information about earnings, use of EI, and use of SA was taken from the study of incremental impacts. Footnote 20 The program costs were calculated using information available in the EI Monitoring and Assessment Reports.
Relative to the previous cycle of evaluation, the methodology has been extended to incorporate one of the indirect health benefits associated with increased labour market attachment. In particular, the methodology includes an estimate of the change in public health care cost due to the decline in health care utilization resulting from program participation.
Data on average public healthcare costs by income quintiles are taken from the report Lifetime Distributional Effects of Publicly Financed Health Care in Canada (2013) by the Canadian Institute for Health Information.
Program costs are measured using information on LMDA expenditures and new interventions reported in the EI Monitoring and Assessment Report. Other costs and benefits are assessed using integrated administrative data from the EI Part I and II databank and the Canada Revenue Agency.
Incremental impacts measured over the second year of participation and up to 5 post-program years are discounted by 3% to bring them to a common base with the program cost and benefits incurred in the program start year. This 3% rate accounts for the interest the government could have collected if the funds used to pay for the program had been invested. Incremental impacts are estimated using 2010 constant dollars and this accounts for inflation.
Costs and benefits accounted for in the calculations
Program cost: cost incurred by the government for delivering the program (that is, administration and direct program costs calculated from data reported in the EI Monitoring and Assessment Reports).
Marginal social cost of public funds: loss incurred by society when raising additional revenues such as taxes to fund government spending. The value is estimated as 20% the program cost, sales taxes, income taxes, impacts on EI and impacts on SA paid or collected by the government.
Foregone earnings: estimated net impacts on participants’ earnings during the participation period. During labour market program participation, some individuals have lower earnings than what they would have received if they had not participated.
Employment earnings: incremental impacts on participants’ earnings during and after participation. In-program earnings represent the foregone earnings for participants.
Fringe benefits: the employer-paid health and life insurance as well as pension contributions. They are estimated as 15% of incremental impacts on earnings.
Federal and provincial income taxes: incremental impacts on federal, provincial and territorial taxes paid by participants.
Sales taxes: the sales taxes paid by participants estimated as incremental impacts on earnings multiplied by the propensity to consume (97%), the proportion of household spending on taxable goods and services (52%) and the total average federal and provincial sales tax rate (11%).
Social assistance and Employment Insurance benefits collected: incremental impacts on SA and EI benefits use by participants following participation.
Canada Pension Plan contribution and Employment Insurance premiums: these contributions and premiums were identified from the Canada Revenue Agency data and then, the incremental impacts on Canada Pension Plan contributions and EI premiums were estimated.
Public health care costs savings: estimated impact of participation in EBSMs on public health care costs shown as an average change per participant over the post-program period examined.
Strengths and limitations of the studies
One of the key strengths from the studies is that all quantitative analyses are based on administrative data rather than survey responses. Compared to survey data, administrative data are not subject to recall errors or response bias.
The propensity score models used to match participants and non-participants for the incremental impact analyses are judged to be robust. In part this is because they were based on 5 years of pre-participation data. Moreover, these models are based on a vast array of variables including sociodemographic characteristics, location, skill level related to last occupation, and indicators of labour market attachment.
However, the matching process can be further refined for specific subgroups if the following information is available in the future:
- persons with disabilities: the type and severity of the disability, and the capacity/willingness to work full-time
- recent immigrants: the country of origin, the proficiency in English or French, and the relevance of credentials and work experience
- visible minorities: place of birth; individuals who are born outside of Canada face different challenges compared to those born in Canada
Refining the matching process for population subgroups could broaden the scope for greater Gender-based Analysis Plus.
Sensitivity analysis and the use of alternative estimation methods have increased confidence in the incremental impact estimates. However, one limitation with the propensity score matching techniques is that no one can be fully sure the impacts are not influenced by factors not captured in the data.
The cost-benefit analysis accounted for all quantifiable costs and benefits directly attributable to the EBSMs and could be estimated with the available administrative data. It is furthered strengthened by incorporating one of the indirect benefits, which is the change in public health benefits care expenditure associated with program participation. However, the analysis did not account for non-quantifiable factors that can lead to an understatement of the benefits (for example, positive spillovers to other family members) and factors that can lead to an overstatement of the benefits (for example, effects on skill prices or displacement).
In some studies that use qualitative data collection methods, the number of key informants interviewed is relatively small. Responses provided by key informants reflect their own experience and their own region and may not be fully representative of the entire province and territory.
In the Self-Employment study, only a 13.5% response rate was achieved. Due to the low response rate, survey results are not representative of the program participants in Alberta. Outcomes for the 29 valid responses will not be presented in the synthesis report.
Overview of the studies summarized in this report
The findings in this report are drawn from 7 separate studies conducted at the national level. These studies examine issues related to program effectiveness, efficiency, design/delivery and used a mix of qualitative and quantitative methods. Appendix A presents an overview of these studies. The studies are:
- Examination of the medium-term outcomes from 2010 to 2017
- Estimation of the medium-terms incremental impacts from 2010 to 2017
- Cost-Benefit Analysis of Employment Benefits and Support Measures in Alberta
- Cost-Benefit Analysis of Employment Benefits and Support Measures in Alberta: Incorporating Public Health Care Costs Savings in the Context of the Labour Market Programs Evaluation
- Design and delivery of the Integrated Training program in Alberta
- Design and delivery of the Self-Employment program in Alberta
- Design and delivery of the Labour Market Partnerships program in Alberta
Evaluation findings
Profile of participants
Nearly 73,000 EI active and former claimants participated in LMDA programs and services between 2010 and 2012 in Alberta.
The profile of participants is presented in Table 2 by gender, age, sociodemographic group, and marital status. Information about their educational attainment, occupation and industry is based on the latest job they held prior to applying for EI benefits. Information about sociodemographic groups is self-reported.
Categories | Active claimants | Former claimants |
---|---|---|
Number of participants | 32,010 | 40,925 |
Gender: Female | 47% | 50% |
Gender: Male | 53% | 50% |
Age: 30 and under | 30% | 32% |
Age: 31 to 54 | 58% | 56% |
Age: 55 and over | 13% | 12% |
Sociodemographic group: Indigenous people | 8% | 11% |
Sociodemographic group: Persons with disabilities | 3% | 4% |
Sociodemographic group: Visible minorities | 8% | 8% |
Sociodemographic group: Recent immigrants | 5% | 4% |
Marital status: Married or common-law | 33% | 28% |
Marital status: Widow / divorced / separated | 16% | 18% |
Marital status: Single | 46% | 48% |
Education or skills level: High school or occupational training | 35% | 37% |
Education or skills level: On-the-job training | 23% | 27% |
Education or skills level: College, vocational education or apprenticeship training | 33% | 28% |
Education or skills level: University degree | 3% | 3% |
Note: Values may not equal 100% due to rounding or missing information.
Top 3 occupational groups for EBSMs participants:Footnote 21
Active claimants:
- skilled crafts and trades workers = 16%
- semi-skilled manual workers; other manual workers = 13% each
- clerical personnel; intermediate sales and service personnel = 11% each
Former claimants:
- other manual workers = 14%
- semi-skilled manual workers; other sales and service personnel; intermediate sales and service personnel = 13% each
- skilled crafts and trades workers = 12%
Top 3 industries for EBSMs participants
Active claimants:
- construction = 18%
- retail trade = 11%
- manufacturing = 10%
Former claimants:
- construction = 15%
- retail trade = 13%
- manufacturing = 10%
As presented in Table 3, in the year before program participation, former claimants have lower levels of employment and earnings than active claimants. Former claimants increase their use of SA.
Pre-EBSM participation employment characteristics | Active claimants | Former claimants |
---|---|---|
Average employment earnings | $29,755 | $16,285 |
Percentage employed | 98% | 84% |
Percentage on SA | 11% | 20% |
Incremental impacts for active and former EI claimants
Main findings: Overall, incremental impacts demonstrate that participants in FLS/OT improve their labour market attachment (employment and earnings) and reduce their dependence on government income supports (that is, the combination of EI and SA) compared to similar non-participants. Active EI claimants in CEAS alone have a small negative, but not statistically significant, impact on the probability of being employed, as well as decreases in employment earnings. These participants also increase their dependence on government income supports.
In Alberta, incremental impacts were estimated for active and former EI claimant participants in FLS/OT, and for active claimants in CEAS. Moreover, incremental impacts for Integrated Training/Immigrant Bridging participants are included under the incremental impacts for FLS/OT due to the employment benefit’s reclassification as part of FLS/OT by Alberta. Incremental impacts for Workplace Training Program were not produced due to the small number of participants.
For FLS/OT, the incremental impact results presented below are generally consistent with those found as part of the second LMDA evaluation cycle.
Incidence of employment
Chart 1 presents the incremental impacts on the incidence of employment for active and former claimants by type of program.Footnote 22 The estimates can be interpreted as a change in the probability of being employed following participation.
Active and former EI claimants who participate in FLS/OT increase their incidence of employment relative to similar non-participants.
For active claimants who participated in CEAS-only, the average annual incremental impact was not statistically significant. However, participants experienced statistically significant decreases in their incidence of employment in years 3 and 5 post-participation (-1 and -1.1 percentage points respectively). CEAS are relatively modest activities such as counselling, job search assistance and case management, which focus on quicker returns to work for participants. CEAS supports that are not provided with longer interventions, are not expected to increase participants’ skills or influence their employment levels to a large extent.Footnote 23
Text description – Chart 1
Program name | Incidence of employment for active claimants (percentage points) | Incidence of employment for former claimants (percentage points) |
---|---|---|
Foundational Learning Supports / Occupational Training | 6.7 | 5.7 |
Career Employment Assistance Services | -0.5* | n/a |
Note: Impacts are estimated over 4 post-program years for FLS/OT and 5 years post-program for CEAS.
* The impact is not statistically significant. However, CEAS participants experienced a decrease in the incidence of employment in years 3 and 5 post-participation (-1 and -1.1 percentage points respectively).
Employment earnings
Chart 2 presents the average annual increase in employment earnings for active and former EI claimants in the post-participation period.
Active EI claimants in FLS/OT experience positive, but not statistically significant, impact on their average annual average employment earnings relative to similar non-participants. While the impact is not statistically significant for the entire post-program period, participants experience a statistically significant increase in earnings ($1,724) in year 2 post-program participation.
Active EI claimants who participated in CEAS experience a negative, but not statistically significant, impact on their annual average employment earnings relative to similar non-participants. While the impact is not statistically significant for the entire post-program period, a statistically significant decrease in employment earnings of -$488 was found in year 1 post-program participation. However, as noted above, these services are relatively modest activities and, by themselves, are not expected to lead to greatly influence labour market outcomes. In addition, these services do not aim to upgrade participants’ level of education and/or skills and are therefore not necessarily expected to assist them in securing higher paying jobs. Securing higher paying jobs is more dependent on participants’ prior levels of education and skills.
Former EI claimants who participated in FLS/OT increase their employment earnings relative to similar participants who received CEAS-only services.
Text description – Chart 2
Program name | Employment earnings of active claimants | Employment earnings of former claimants |
---|---|---|
Foundational Learning Supports / Occupational Training | $1,2851 | $1,705 |
Career Employment Assistance Services | $1242 | n/a |
Note: Impacts are estimated over 4 post-program years for FLS/OT and 5 years post-program for CEAS.
1 The impact is not-statistically significant. However, a statistically significant increase in earnings ($1,724) was found in year 2 post-program participation.
2 The impact is not-statistically significant. However, a statistically significant decrease in earnings (-$488) was found in year 1 post-program.
Use of EI benefits
Chart 3 presents the annual average incremental impacts on the use of EI benefits for active and former EI claimant participants.
Active EI claimant participants in FLS/OT have not statistically significant increase in their annual average use of EI benefits in the post-program period, relative to similar non-participants. However, a statistically significant increase of $405 was found in year 2 post-program.
Active EI claimant participants in CEAS reduce their annual average receipt of EI benefits.
Former EI claimants who participate in FLS/OT increase their use of EI benefits in the post-program period. In the case of FLS/OT, the increase in the use of EI by former claimant participants is consistent with previous evaluations. For FLS/OT former claimants the increase is not necessarily a negative impact. Following their participation in these interventions, former EI claimants are more likely to requalify for EI benefits due to their stronger labour market attachment demonstrated by increases in employment and earnings.
Text description – Chart 3
Program name | Use of EI benefits for active claimants | Use of EI benefits for former claimants |
---|---|---|
Foundational Learning Supports / Occupational Training | $100* | $161 |
Career Employment Assistance Services | $260 | n/a |
Note: Impacts are estimated over 4 post-program years for FLS/OT and 5 years for CEAS.
*The impact is not-statistically significant. However, a statistically significant increase of $405 was found in year 2 post-program.
Use of SA benefits
As shown in Chart 4, active and former EI claimants who participate in FLS/OT decrease their use of SA benefits in the post-program period. While the annual average for active claimants is not statistically significant over the entire post-program period, participants decreased their use of SA benefits in years 3 and 4 post-program by $211 and $148 respectively.
Active EI claimants who participated in CEAS increased their use of SA benefits compared to similar non-participants.
Text description – Chart 4
Program name | Use of SA benefits for active claimants | Use of SA benefits for former claimants |
---|---|---|
Foundational Learning Supports / Occupational Training | -$97* | $299 |
Career Employment Assistance Services | $391 | n/a |
Note: Impacts are estimated over 4 post-program years for FLS/OT and 5 years for CEAS.
*The impact is not statistically significant over the entire post-program period as an annual average. However, participants decreased their use of SA benefits in years 3 and 4 post-program by $211 and $148 respectively.
Dependence on income support
As shown in Chart 5, active and former EI claimants in FLS/OT reduce their overall level of dependence on income support (combined EI and SA benefits). While the finding for active FLS/OT participants is not statistically significant over the entire post-program period, participants did reduce their dependence by 1.4 percentage points in year 3 post-program.
Active EI claimants in CEAS increase their use of government income support.
Text description – Chart 5
Program name | Change in the dependence on government income support for active claimants (percentage points) | Change in the dependence on government income support for former claimants (percentage points) |
---|---|---|
Foundational Learning Supports / Occupational Training | -0.7* | -2.4 |
Career Employment Assistance Services | 2.6 | n/a |
Note: Impacts are estimated over 4 post-program years for FLS/OT and 5 years post-program for CEAS.
*The impact is not statistically significant over the entire post-program period. However, participants did reduce their dependence on government income support by 1.4 percentage points in year 3 post-program.
Incremental impacts by subgroups of participants
Female participants
Nearly 35,625 EI active and former claimant participants in Alberta between 2010 and 2012 are female, representing nearly 49% of participants.
The profile of female participants is presented in Table 4 by age, sociodemographic group, and marital status. Information about their educational attainment, occupation and industry is based on the latest job they held prior to applying for EI benefits. Information about sociodemographic groups is self-reported.
Categories | Active claimants | Former claimants |
---|---|---|
Number of participants | 14,972 | 20,654 |
Age: 30 and under | 29% | 35% |
Age: 31 to 54 | 59% | 55% |
Age: 55 and over | 13% | 10% |
Sociodemographic group: Indigenous people | 10% | 12% |
Sociodemographic group: Persons with disabilities | 3% | 4% |
Sociodemographic group: Visible minorities | 6% | 6% |
Sociodemographic group: Recent immigrants | 4% | 4% |
Marital status: Married or common-law | 31% | 28% |
Marital status: Widow / divorced / separated | 22% | 23% |
Marital status: Single | 43% | 44% |
Education or skills level: High school or occupational training | 43% | 44% |
Education or skills level: On-the-job training | 20% | 23% |
Education or skills level: College, vocational education or apprenticeship training | 25% | 22% |
Education or skills level: University degree | 4% | 4% |
Note: Values may not equal 100% due to rounding or missing information.
Top 3 occupational groups for female participants Footnote 24
Active claimants:
- intermediate sales and service personnel = 19%
- clerical personnel = 18%
- other sales and service professionals = 14%
Former claimants:
- intermediate sales and service personnel = 22%
- clerical personnel; other sales and service professionals = 17% each
- semi-professionals and technicians = 7%
Top 3 industries for female participants
Active claimants:
- retail trade = 15%
- accommodation and food services = 12%
- healthcare and social assistance = 11%
Former claimants:
- retail trade = 17%
- accommodation and food services = 13%
- healthcare and social assistance = 10%
Main findings: Female active and former EI claimant participants in FLS/OT improve their labour market attachment through increases in their incidence of employment and earnings. Only former claimant participants decrease their dependence on government income supports (that is, the combined use of EI and SA benefits). In the case of CEAS, female participants are found to increase their annual average dependence on government income supports following participation, due to increases in SA benefits.
Table 5 presents the detailed incremental impacts. For example, the results reveal that relative to the comparison groups, female active EI claimant participants in FLS/OT have higher annual average earnings (+ $2,489) and incidence of employment (+ 6.6 percentage points).
Female former EI claimant participants in FLS/OT have higher annual average earnings (+ $1,710) and incidence of employment rate (+ 6.1 percentage points). They also decreased their annual average use of government income supports (-2.6 percentage points), due mainly to a decrease in the use of SA benefits (- $420 per year).
Female active EI claimant participants in CEAS increase their use of government income supports (+ 3 percentage points) due to increases in SA benefits receipt ($410 per year). The annual average results for incidence of employment and earnings are not statistically significant.
Indicator | FLS/OT active claimants | FLS/OT former claimants | CEAS active claimants |
---|---|---|---|
Incidence of employment (percentage points) | 6.6*** | 6.1*** | 0.32 |
Employment earnings ($) | 2,489*** | 1,710*** | 1463 |
EI benefits ($) | -761 | 325*** | -214*** |
SA benefits ($) | 6 | -420*** | 410*** |
Dependence on income support (percentage points) | 0.3 | -2.6*** | 3*** |
n= | 1,048 | 1,313 | 13,873 |
Statistical significance level *** 1%; ** 5%; * 10%, other values are not statistically significant.
Note: Impacts are estimated over 4 post-program years for FLS/OT and 5 years for CEAS.
1 While the annual average impact on EI benefits (- $76) is not statistically significant over the entire post-participation period, active EI claimants in FLS/OT decreased their EI benefits use in year 1 post-program participation (- $342).
2 While the annual average impact on incidence of employment (0.3 percentage points) is not statistically significant over the entire post-participation period, participants increased their incidence of employment by 1.1 percentage points in in year 1 post-program participation.
3 While the annual average impact on employment earnings ($146) is not statistically significant over the entire post-program period, participants decreased their employment earnings (-$701) in year 1 post-program participation.
Male participants
Nearly 37,300 EI active and former claimant participants in Alberta between 2010 and 2012 are male, representing about 51% of participants.
The profile of male participants is presented in Table 6 by age, sociodemographic group, and marital status. Information about their educational attainment, occupation and industry is based on the latest job they held prior to applying for EI benefits. Information about sociodemographic groups is self-reported.
Categories | Active claimants | Former claimants |
---|---|---|
Number of participants | 17,038 | 20,271 |
Age: 30 and under | 30% | 29% |
Age: 31 to 54 | 57% | 58% |
Age: 55 and over | 13% | 13% |
Sociodemographic group: Indigenous people | 7% | 11% |
Sociodemographic group: Persons with disabilities | 3% | 4% |
Sociodemographic group: Visible minorities | 8% | 9% |
Sociodemographic group: Recent immigrants | 5% | 4% |
Marital status: Married or common-law | 35% | 27% |
Marital status: Widow / divorced / separated | 11% | 12% |
Marital status: Single | 47% | 51% |
Education or skills level: High school or occupational training | 28% | 29% |
Education or skills level: On-the-job training | 26% | 31% |
Education or skills level: College, vocational education or apprenticeship training | 40% | 34% |
Education or skills level: University degree | 2% | 2% |
Note: Values may not equal 100% due to rounding or missing information.
Top 3 occupational groups for male participants
Active claimants:
- skilled crafts and trades = 27%
- semi-skilled manual workers = 20%
- other manual workers = 19%
Former claimants:
- skilled crafts and trades; other manual workers = 22% each
- semi-skilled manual workers = 21%
- other sales and service personnel = 9%
Top 3 industries for male participants
Active claimants:
- construction = 28%
- manufacturing = 14%
- administrative and support, waste management and remediation services = 9%
Former claimants:
- construction = 24%
- manufacturing = 15%
- administrative and support, waste management and remediation services = 9%
Main findings: Male active and former EI claimant participants in FLS/OT improve their labour market attachment through increases in their incidence of employment and employment earnings. Former male participants also decrease their dependence on government income supports (that is, the combined use of EI and SA benefits). Male active claimant participants in CEAS decrease their incidence of employment and increase their dependence on government income support relative to similar non-participants.
Table 7 presents the detailed incremental impacts. For example, the results reveal that relative to the comparison groups, former claimants in FLS/OT increase their annual average incidence of employment (+ 5.6 percentage points). They also decreased their use of government income support (- 3.8 percentage points), due to their decreased use of SA benefits (- $585). While not statistically significant for average annual amount, male participants also increased their employment earnings over the medium-term. Statistically significant increases in employment earnings were found in the third and fourth post-program years (+ $2,641 and $3,040 respectively).
Active male claimant participants in CEAS only activities have lower average annual incidence of employment (- 1.2 percentage points). They also increase their use of government income support (+ 2.1 percentage points), due to the increase in the use of SA benefits (+ $322 per year).
Indicator | FLS/OT active claimants | FLS/OT former claimants | CEAS active claimants |
---|---|---|---|
Incidence of employment (percentage points) | 7.2*** | 5.6*** | -1.2** |
Employment earnings ($) | 2,561* | 1,0312 | 144 |
EI benefits ($) | 2391 | 128 | -280*** |
SA benefits ($) | 44 | -585*** | 322*** |
Dependence on income support (percentage points) | -0.3 | -3.8*** | 2.1*** |
n= | 914 | 849 | 16,076 |
Statistical significance level *** 1%; ** 5%; * 10%, other values are not statistically significant.
Note: Impacts are estimated over 4 post-program years for FLS/OT and 5 years for CEAS.
1 While the annual average impact on EI benefits ($239) is not statistically significant over the entire post-program period, participants increased their use of EI benefits in years 2 and 3 post-program participation (+ $476 and $550 respectively).
2 While the annual average impact on employment earnings (+ 1,031) is not statistically significant over the entire post-participation period, participants increased their earnings in years 3 and 4 post-program participation (+ $2,641 and $3,040 respectively).
Youth participants
Nearly 22,500 participants, between 2010 and 2012, were 30 years of age or younger when they began their program participation, representing about 31% of participants.
The profile of youth participants is presented in Table 8 by gender, sociodemographic group, and marital status. Information about their educational attainment, occupation and industry are based on the latest job they held prior to applying for EI benefits. Information about sociodemographic groups is self-reported.
Categories | Active claimants | Former claimants |
---|---|---|
Number of participants | 9,472 | 13,015 |
Gender: Female | 45% | 55% |
Gender: Male | 55% | 45% |
Sociodemographic group: Indigenous people | 8% | 12% |
Sociodemographic group: Persons with disabilities | 2% | 3% |
Sociodemographic group: Visible minorities | 5% | 5% |
Sociodemographic group: Recent immigrants | 4% | 3% |
Marital status: Married or common-law | 25% | 23% |
Marital status: Widow / divorced / separated | 5% | 9% |
Marital status: Single | 66% | 64% |
Education or skills level: High school or occupational training | 30% | 36% |
Education or skills level: On-the-job training | 25% | 30% |
Education or skills level: College, vocational education or apprenticeship training | 40% | 29% |
Education or skills level: University degree | 2% | 2% |
Note: Values may not equal 100% due to rounding or missing information.
Top 3 occupational groups for youth participants Footnote 25
Active claimants:
- skilled crafts and trade workers = 26%
- other manual workers = 15%
- intermediate sales and service personnel; other sales and service personnel = 11% each
Former claimants:
- other manual workers = 16%
- intermediate sales and service personnel; skilled crafts and trade workers = 15% each
- other sales and service personnel = 14%
Top 3 industries for youth participants Footnote 26
Active claimants:
- construction = 25%
- retail trade = 12%
- accommodation and food services; manufacturing = 9% each
Former claimants:
- construction = 17%
- retail trade = 15%
- accommodation and food services = 11%
Main findings: Youth former EI claimants in FLS/OT improve their labour market attachment through increases in their employment earnings and incidence of employment. They also decrease their dependence on government income support. Youth active EI claimant participants in CEAS increase their dependence on government income supports and have mixed and not statistically significant results for their labour market attachment. Youth active EI claimant participants in FLS/OT have short-term increases in employment earnings. The majority of their results are mixed and not statistically significant.
Table 9 presents the detailed incremental impacts. For example, the results reveal that relative to the comparison groups, youth former EI claimants who participated in FLS/OT have higher average annual incidence of employment rate (+ 4.4 percentage points). They also reduce their use of government income supports (- 2.5 percentage points), due to their lesser use of SA benefits (- $406 per year). While not statistically significant for average annual amount, youth had a statistically significant increase in their employment earnings in the fourth post-participation year (+ $1,981).
Youth active EI claimants who participated in CEAS only activities have higher annual average government income support reliance rate (+ 1.9 percentage points), due to their increased use of SA benefits (+ $290). While the average annual impact on the incidence of employment (-0.7 percentage point) is not statistically significant over the entire post-program period, participants decreased their incidence of employment in years 3 and 5 post-participation by 1.4 and 1.7 percentage points.
Indicator | FLS/OT active claimants | FLS/OT former claimants | CEAS active claimants |
---|---|---|---|
Incidence of employment (percentage points) | 0.6 | 4.4** | 0.75 |
Employment earnings ($) | 2,6321 | 1,3674 | 574 |
EI benefits ($) | 2642 | 473*** | -236*** |
SA benefits ($) | 57 | -406*** | 290*** |
Dependence on income support (percentage points) | 1.53 | -2.5** | 1.9*** |
n= | 595 | 758 | 8,843 |
Statistical significance level *** 1%; ** 5%; * 10%, other values are not statistically significant.
Note: Impacts are estimated over 4 post-program years for FLS/OT and 5 years for CEAS
1 While the annual average impact on employment earnings (+ $2,632) is not statistically significant over the entire post-participation period, participants increased their employment earnings in year 2 post-program participation (+ $3,239).
2 While the annual average impact on the use of EI benefits is not statistically significant over the entire post-program period, participants increased their use of EI benefits in year 2 by $524.
3 While the annual average impact on the dependence on income support is not statistically significant over the entire post-program period, participants increased their dependence in year 3 by 2.5 percentage points.
4 While the annual average impact on employment earnings (+ $1,367) is not statistically significant over the entire post-participation period, participants increased their earnings in year 4 post-program participation by $1,981.
5 While the average annual impact on the incidence of employment (-0.7 percentage point) is not statistically significant over the entire post-program period, participants decreased their incidence of employment in years 3 and 5 post-participation by 1.4 and 1.7 percentage points respectively.
Older worker participants
Nearly 4,100 EI active claimant participants, between 2010 and 2012, were 55 years of age or older when they began their program, representing about 6% of participants.
The profile of older worker participants is presented in Table 10 by gender, sociodemographic group, and marital status. Information about their educational attainment, occupation and industry are based on the latest job they held prior to applying for EI benefits. Information about sociodemographic groups is self-reported.
Categories | Active claimants |
---|---|
Number of participants | 4,075 |
Gender: Female | 47% |
Gender: Male | 53% |
Sociodemographic group: Indigenous people | 6% |
Sociodemographic group: Persons with disabilities | 4% |
Sociodemographic group: Visible minorities | 5% |
Sociodemographic group: Recent immigrants | 1% |
Marital status: Married or common-law | 37% |
Marital status: Widow / divorced / separated | 30% |
Marital status: Single | 29% |
Education or skills level: High school or occupational training | 40% |
Education or skills level: On-the-job training | 24% |
Education or skills level: College, vocational education or apprenticeship training | 25% |
Education or skills level: University degree | 3% |
Note: Values may not equal 100% due to rounding or missing information.
Top 3 occupational groups for older worker active claimants:
- semi-skilled manual workers = 17%
- other sales and service professionals = 14%
- intermediate sales and service personnel = 12%
Top 3 industries for older worker active claimants:
- retail trade = 12%
- construction = 11%
- manufacturing = 10%
Main findings: Incremental impacts reveal that older workers who are active EI claimant participants in CEAS increase their average annual employment earnings following program participation. However, they also increase their dependence on government income supports (that is, the combined use of EI and SA benefits) relative to similar non-participants. Due to insufficient numbers, incremental impacts are estimated only for active EI claimant participants in CEAS.
Table 11 presents the detailed incremental impacts. For example, the results reveal that relative to the comparison group, older workers who are active EI claimant participants in CEAS have higher annual earnings (+$984). A not statistically significant increase is also found in incidence of employment. These participants also increase their annual average dependence on government income supports (+ 2 percentage points) through increases in SA benefits (+ $500 per year).
Indicator | CEAS active claimants |
---|---|
Incidence of employment (percentage points) | 1 |
Employment earnings ($) | 984* |
EI benefits ($) | -146* |
SA benefits ($) | 500*** |
Dependence on income support (percentage points) | 2** |
n= | 3,857 |
Statistical significance level *** 1%; ** 5%; * 10%, other values are not statistically significant.
Note: Impacts are estimated over 5 years for CEAS.
Recent immigrants Footnote 27
In Alberta, 1,600 active EI claimant participants in LMDA programs and services, between 2010 and 2012, were recent immigrants, representing about 2% of participants.
The profile of recent immigrant participants is presented below by gender, age and marital status. Information about their educational attainment, occupation and industry are based on the latest job they held prior to applying for EI benefits.
Categories | Active claimants |
---|---|
Number of participants | 1,600 |
Gender: Female | 42% |
Gender: Male | 58% |
Age: 30 and under | 22% |
Age: 31 to 54 | 74% |
Age: 55 and over | 3% |
Marital status: Married or common-law | 70% |
Marital status: Widow / divorced / separated | 10% |
Marital status: Single | 17% |
Education or skills level: High school or occupational training | 31% |
Education or skills level: On-the-job training | 31% |
Education or skills level: College, vocational education or apprenticeship training | 27% |
Education or skills level: University degree | 6% |
Note: Values may not equal 100% due to rounding or missing information.
Top 3 occupational groups for recent immigrant active claimant participants:
- other sales and service professionals = 18%
- other manual workers = 14%
- semi-skilled manual workers = 13%
Top 3 industries for recent immigrant active claimant participants: Footnote 28
- manufacturing = 20%
- administrative and support, waste management and remediation services = 11%
- retail trade; accommodation and food services = 10% each
Main findings: Recent immigrants who are active EI claimant participants in CEAS increased their annual average dependence on income supports (that is, the combination of EI and SA benefits) relative to similar non-participants. The incremental impacts on the labour market attachment were not statistically significant.
Due to insufficient numbers, incremental impacts are estimated only for active EI claimant participants in CEAS.
Table 13 presents the detailed incremental impacts. The results reveal that relative to the comparison group, recent immigrants who are active EI claimant participants in CEAS, increase their annual average dependence on government income supports (+ 1.6 percentage points) through increases in SA benefits (+ $254 per year). The impacts on the labour market attachment are not statistically significant.
Indicator | CEAS active claimants |
---|---|
Incidence of employment (percentage points) | 2.5 |
Employment earnings ($) | 613 |
EI benefits ($) | -441 |
SA benefits ($) | 254*** |
Dependence on income support (percentage points) | 1.6** |
n= | 1,398 |
Statistical significance level *** 1%; ** 5%; * 10%, other values are not statistically significant.
Note: Impacts are estimated over 5 years for CEAS.
1 While the annual average impact on EI benefits is not statistically significant over the entire post-participation period, active EI claimants in CEAS decreased their EI benefits in the first-year post-program by a statistically significant amount of - $402.
Visible minorities
Nearly 3,100 EI former claimant participants in Alberta, between 2010 and 2012, self-identified as being visible minorities, representing about 4% of total participants.
The profile of participants is presented below by gender, age, and marital status. Information about their educational attainment, occupation and industry are based on the latest job they held prior to applying for EI benefits.
Categories | Former claimants |
---|---|
Number of participants | 3,082 |
Gender: Female | 43% |
Gender: Male | 57% |
Age: 30 and under | 23% |
Age: 31 to 54 | 68% |
Age: 55 and over | 10% |
Marital status: Married or common-law | 39% |
Marital status: Widow / divorced / separated | 19% |
Marital status: Single | 37% |
Education or skills level: High school or occupational training | 37% |
Education or skills level: On-the-job training | 30% |
Education or skills level: College, vocational education or apprenticeship training | 25% |
Education or skills level: University degree | 4% |
Note: Values may not equal 100% due to rounding or missing information.
Top 3 occupational groups for visible minority former claimant participants:
- other sales and service personnel = 16%
- semi-skilled manual workers = 15%
- other manual workers = 14%
Top 3 industries for visible minority former claimant participants: Footnote 29
- manufacturing = 16%
- administrative and support, waste management and remediation services = 11%
- retail trade; construction = 10% each
Main findings: Visible minorities who are former claimant participants in FLS/OT have higher incidence of employment (+ 4.2 percentage points). They did, however, decreased their employment earnings in the short-term.
Due to insufficient numbers, incremental impacts are estimated only for former claimant participants in FLS/OT. Table 15 presents the detailed incremental impacts.
Former EI claimant participants in FLS/OT who were visible minorities, increased their incidence of employment by 4.2 percentage points. While the annual average impact on employment earnings is not statistically significant over the entire post-participation period, participants decreased their earnings in years 1 and 2 post-program participation by $4,136 and $3,070 respectively.
Indicator | FLS/OT former claimants |
---|---|
Incidence of employment (percentage points) | 4.2* |
Employment earnings ($) | -1,6681 |
EI benefits ($) | -1682 |
SA benefits ($) | 21 |
Dependence on income support (percentage points) | 0.1 |
n= | 285 |
Statistical significance level *** 1%; ** 5%; * 10%, other values are not statistically significant.
Note: Impacts are estimated over 4 post-program years for FLS/OT.
1 While the annual average impact on employment earnings is not statistically significant over the entire post-participation period, former EI claimants in FSL/OT decrease their earnings in years 1 and 2 post-program participation (- $4,136 and - $3,070 respectively).
2 While the annual average impact on EI benefits is not statistically significant over the entire post-participation period, former EI claimants in FSL/OT decreased their EI benefits in year 2 post-program participation (- $495).
Incremental impacts by region for FLS/OT participants
An additional analysis was conducted to examine the incremental impacts for FLS/OT active and former participants by region (that is, for participants in Calgary, and for participants outside of Calgary).
FLS/OT participants in Calgary
Main findings
Active EI claimant participants in FLS/OT in Calgary increased their annual average incidence of employment by 5 percentage points compared to similar non-participants. Other estimates were not statistically significant.
The findings for Former EI claimants in Calgary are mixed and not statistically significant.
Indicator | FLS/OT active claimants | FLS/OT former claimants |
---|---|---|
Incidence of employment (percentage points) | 5** | 2.32 |
Employment earnings ($) | 1,493 | -1503 |
EI benefits ($) | -120 | -1004 |
SA benefits ($) | -70 | -1855 |
Dependence on income support (percentage points) | -1.21 | -1.3 |
n= | 666 | 822 |
Statistical significance level *** 1%; ** 5%; * 10%, other values are not statistically significant.
Note: Impacts are estimated over 4 post-program years for FLS/OT.
1 While the annual average impact on dependence on income supports is not statistically significant over the entire post-participation period (- 1.2 percentage points), these participants decreased their use of government income supports in the year 4 post-program participation by 3.3 percentage points.
2 While the average annual impact on the incidence of employment (2.3 percentage points) is not statistically significant over the entire post-program period, participants increased their incidence of employment in year 3 post-participation by 3.9 percentage points.
3 While the annual average impact on employment earnings is not statistically significant over the entire post-program period, these participants decreased their earnings in year 1 post-program participation by $2,322.
4 While the annual average impact on the use of EI benefits is not statistically significant over the entire post-participation period, these participants decreased their use of EI benefits in year 1 post-program participation by $262.
5 While the annual average impact on the use of SA is not statistically significant over the entire post-participation period, these participants decreased their use of SA in year 1 and 2 post-program participation by $248 and $275 respectively.
FLS/OT participants outside of Calgary
Main findings
Incremental impacts reveal that former EI claimant participants in FSL/OT improve their labour market attachment with average annual increases in incidence of employment (+ 8 percentage points) and employment earnings (+ $3,459). They also reduce their use of government income supports (- 3.5 percentage points), due to decreases in SA benefits (- $492 per year).
Active EI claimant participants in FSL/OT increase their incidence of employment (5.6 percentage points) relative to similar non-participants. The remaining results are not statistically significant.
Indicator | FLS/OT active claimants | FLS/OT former claimants |
---|---|---|
Incidence of employment (percentage points) | 5.6*** | 8*** |
Employment earnings ($) | 959 | 3,459*** |
EI benefits ($) | 105 | 493*** |
SA benefits ($) | -21 | -492*** |
Dependence on income support (percentage points) | -0.3 | -3.5** |
n= | 1,301 | 1,337 |
Statistical significance level *** 1%; ** 5%; * 10%, other values are not statistically significant.
Note: Impacts are estimated over 4 post-program years for FLS/OT.
Cost-benefit analysis
This analysis is based on the EBSM medium-term incremental impacts previously described in this report. Costs and benefits are examined over the participation period of 1 or 2 years and 10 years after the end of participation. Footnote 30
The cost-benefit analysis addresses the following questions:
- Are the benefits from EBSMs exceeding the costs within 10 years for FLS/OT after the end of participation?
- How much is the benefit for the government and society if the government spends $1 in EI part II funding?
- How many years does it take the benefits to recover the costs?
The cost-benefit results are generated separately for active and former EI claimants who participated in FLS/OT. The analysis was not conducted for active claimant participants in CEAS given that the incremental impacts on employment earnings are negative.
The following results are presented from the social perspective, that is, the government and individual combined. This allows for a sound assessment of program effectiveness in achieving its objectives of helping unemployed individuals to obtain and maintain employment and to generate EI savings.
Table 18 presents the cost-benefit results for active and former EI claimant participants in FLS/OT.
Indicator | FLS/OT active claimants (10 years post-program) | FLS/OT former claimants (10 years post-program) |
---|---|---|
Net present value | -$7,326 | $7,189 |
Benefit cost ratio | -$0.52 | $2.46 |
Payback period (years after end of participation) | 16 | 7.2 |
Social return | -152% | 146% |
Savings to public health care | $106 | $152 |
The information below provides examples of the net present value, the benefit-cost ratio, the payback period, the social rate of return and savings to health care costs.
Foundational Learning Supports/Occupational Training (FLS/OT) Footnote 31
During the 2010 to 2012 period, FLS/OT represents almost 66% of EBSM expenditures under the LMDAs in Alberta. The average duration of an FLS/OT Action Plan Equivalent is 45 weeks for active claimants and 55 weeks for former claimants.
As shown in Table 18, over the 10-year post-program period, the benefit for active EI claimants is -$7,326 lower than the costs, yielding a social return of -152% on investment. This means that if the government spends $1 on FLS/OT for active EI claimants, it generates a loss of -$0.52 for society. It takes 16 years for the benefits to recover the costs of programming. A savings of $106 per participant was found in public health care costs over the 10-year post-program period.
The benefit for former EI claimants is +$7,189 higher than the costs, yielding a social return of 146% over the 10-year post-program period. This means that if the government spends $1 on programming, it generates $2.46 of gain for society. From a social perspective it takes 7.2 years for society to recover the costs of FLS/OT for former EI claimant participants. A savings of $152 per participant was found in public health care costs per participant over the 10-year post-program period.
Outcomes for active and former EI claimants
Incremental impacts for Workplace Training Program participants were not produced due to the small number of participants. However, outcomes can still be examined for participants to describe the average changes that occur from before to after program participation.
The labour market outcomes are based on individuals who began their participation during the 2010 to 2012 period. Statistics focus on 5 years before and 4 years after participation.
Workplace Training Program participant outcomes
Approximately 245 active and former EI claimant participants, between 2010 and 2012, participated in Workplace Training Program.
Active claimants
As shown in Chart 6, Workplace Training Program participants increase their average earnings from $20,589 in the fifth year pre-program to $27,187 in the fourth year after participation.
Text Description – Chart 6
Year pre-post participation | Employment earnings for active claimants ($) |
---|---|
5 years pre-program | 20,589 |
4 years pre-program | 21,945 |
3 years pre-program | 26,201 |
2 years pre-program | 29,428 |
1 year pre-program | 31,468 |
Program start year | 18,462 |
Second participation year | 27,488 |
1 year post-program | 29,908 |
2 years post-program | 32,100 |
3 years post-program | 30,096 |
4 years post-program | 27,187 |
The proportion of employed participants declines from 98% in the program start year but remains around 92% on average during the post-program period. The proportion of participants on EI Part I decreases from 98% in the program start year to 38% in the fourth year after participation. Participants decrease their dependence on income support from 42% in the program start year to 10% in the fourth year after participation.
Former claimants
As shown in Chart 7, former EI claimant participants in Workplace Training Program increase their average earnings from $19,050 in the fifth year pre-program to $22,074 in the fourth year after participation.
Text Description – Chart 7
Year pre-post participation | Employment earnings for former claimants ($) |
---|---|
5 years pre-program | 19,050 |
4 years pre-program | 22,933 |
3 years pre-program | 24,987 |
2 years pre-program | 23,305 |
1 year pre-program | 15,127 |
Program start year | 15,270 |
Second participation year | 24,006 |
1 year post-program | 24,351 |
2 years post-program | 25,386 |
3 years post-program | 22,884 |
4 years post-program | 22,074 |
The proportion of employed participants declines from 94% in the program start year but remains on average around 84% in the post-program period. The proportion of participants on EI Part I decreases from 57% in the program start year to 27% in the fourth year after participation. Participants decrease their dependence on income support from 25% in the program start year to 17% in the fourth year after participation.
Supplemental studies
Self-Employment Footnote 32
The Self-Employment program in Alberta aims to assist participants in creating employment for themselves by providing them with a range of services including:
- assistance with business plan development
- counselling, coaching and mentoring
- entrepreneurial training and workshops
The application process is structured and aimed to ensure that participants, who are either unemployed or underemployed, are suited for self-employment, have a viable business idea and the financial resources to launch a business.
Alberta has the flexibility to design and deliver the program to meet its labour market needs. The program is delivered through fee-for-service contracts with training providers that can be:
- a larger organization already delivering several employment training/programs and services
- an organization specialized in delivering the Self-Employment program
Alberta allocates between 1% and 2% of its LMDA funding to the Self-Employment program between fiscal years 2013 to 2014 and 2018 to 2019.
Type of businesses created and success factors
According to key informants, program participants started businesses in a variety of industries, including construction, food and beverage industry, marketing, health and personal services, and other services including bookkeeping.
Among the factors that contributed to launching and maintaining their business, key informants included:
- adequate planning including having clear objectives, tools needed to launch and manage the business, setting and meeting deadlines
- mentoring and peer support from other participants
- participant level of commitment and persistence
- adequate assessment by the training provider such as viability of the business, entrepreneurial skills, and motivation
- ability to network
- economic conditions
- business training received
Key informants also noted some of the reasons as to why program participants fail at starting and maintaining their businesses such as:
- insufficient financial resources
- lack of commitment or persistence
- lack of personal or family support
- lack of self-confidence
- economic conditions
- having unrealistic expectations or underestimating the level of required efforts
- poor labour market research, poor business idea, and low demand for the product or service offered
- participants who are difficult to coach
- poor management skills
- personal family or health issues
Challenges and lessons learned
Key informants identify the following challenges related to program design and delivery:
- contracting provisions create challenges for training providers
- rural or northern areas introduce additional challenges to program delivery
- participants may need funding for living expenses that exceed 26 weeks
Key informants identify the following lessons learned related to program design and delivery:
- participant selection process:
- conduct short interviews with potential participants, following the provision of orientation sessions, to validate their level of commitment to become self-employed
- using organizations with demonstrated knowledge of local labour market and capacity to achieve results
- types of training and support provided:
- support the development of solid business plan demonstrating the capacity of the business idea to succeed
- for example, by making available a template for the business plan
- invite experienced guest speakers for training sessions
- provide intensive classroom training, ongoing mentorship, and opportunities for networking
- teaching participants basic sales skills
- ensuring that training modules have adequate reference material available to participants
- support the development of solid business plan demonstrating the capacity of the business idea to succeed
- supporting access to financing:
- importance of informing participants about where and how to access funds, as well as to manage their expectations in that regard
- importance of having a training provider with strong relationships with financing organizations to assist participants in accessing or applying for funding
- introducing participants to lenders throughout the program helps give them the confidence needed to then pursue financing later, on their own, when needed
- program delivery:
- using a diversity of instructors with different business backgrounds and subject matter expertise
- delivering the program in cohorts has led to a network of support among participants that continued after the end of the program
- having instructors with entrepreneurship background to teach future entrepreneurs works well
Integrated Training Footnote 33
The Integrated Training program has been reclassified as part of FLS/OT by Alberta. This supplemental study was conducted prior to the reclassification and provides information about the program design and delivery.
Program objective
Alberta implements a contract and tuition-based training with work-experience program through Integrated Training.
The objectives of Integrated Training are to:
- enable unemployed or marginally employed adult Albertans to secure and maintain employment
- provide training and occupation-related skills recognized by industry/employers
The Immigrant Bridging program (a sub-component of Integrated Training) helps skilled immigrants gain employment in their original occupation or a related occupation.
Program delivery
The design and delivery of Integrated Training allows Alberta to address a variety of barriers to employment experienced by its residents. Integrated Training can also be used to address labour market needs by targeting sub-groups of individuals, occupations or economic sectors in demand, and to a limited extent, communities.
The program may be delivered to individuals by classroom or through the simultaneous delivery of Integrated Training modules during work placements.
Alberta’s Ministry of Labour and Immigration is responsible for contract-based delivery of Integrated Training and Immigrant Bridging programs by third-party training providers, while the Ministry of Advanced Education is responsible for tuition-based delivery of Integrated Training and Immigrant Bridging programs.
All contract-based and tuition-based Integrated Training programs include a component with occupation-related skills, 1 or more work experience placement(s), and a component with employability and/or essential skills. When necessary, programs must also include academic competencies relevant to the occupational outcome and/or English as a second language.
Integrated Training must be provided to participants on a full-time basis up to a maximum of 52 weeks. An exception to this full-time requirement is made for persons with disabilities.
A financial assessment of participants decides the level of supports they are eligible to receive.
Program managers reported that the amount allocated to Integrated Training is influenced by labour market needs, previous success of the program, and the projected ability of training providers to fill student positions.
In addition to gaining work experience, key informants identified a variety of other benefits that can be expected from Integrated Training programs. Participants are expected to develop new skills and job search abilities. For immigrant participants, benefits associated with transitioning to a new labour market are also expected. Benefits for employers who provide training participants with work experience are associated with gaining a potential source of trained employees, helping people in need, and informing the design of training programs to suit their labour needs.
Challenges and lessons learned
In the current design of Integrated Training, key informants identified delivery challenges related to finding and retaining qualified staff for training providers, participant recruitment, monitoring, securing work experience opportunities, and addressing participants’ barriers to employment.
Key informants identified the following as contributing factors to participant success: Footnote 34
- a rigorous assessment process if participants
- work experience opportunities that have a duration catered to participants’ needs
- ensuring program flexibility and supports are in place to assist persons with mental health issues, disabilities, lack of confidence, and multiple barriers to employment
- training providers most likely to succeed are those with experience delivering Integrated Training and those with the staff capacity to address the barriers to employment of participants
Labour Market Partnerships Footnote 35
Program design
The Labour Market Partnerships program aim to assist employers, communities and/or industries to address their labour force adjustments and human resource needs. It includes a wide range of funded activities, such as:
- coordination to facilitate community problem-solving relating to Indigenous employment
- career fairs
- conduct a labour market environmental scan
- develop an industry-based workforce plan to ensure employee skills are maintained
- design and deliver workshops to improve the ability of immigrant serving organizations to integrate immigrants into the labour force
- initiate discussions regarding local challenges and opportunities relating to new industries emerging in the community
- analyse potential training needs and employment opportunities
With $3.6 million in 2020 to 2021, Labour Market Partnerships program represents nearly 2% of total expenditure under the Canada-Alberta LMDA.
Funded organizations
Funded organizations include non-profit organizations, industry and business associations, Indigenous organizations, businesses, and municipal governments.
Targeted labour market issues
Labour Market Partnerships projects targeted current and/or forecasted skills and/or labour shortages. These projects also targeted specific populations (for example, women, Indigenous peoples, and newcomers).
Generally, funded projects target labour market issues associated with:
- technological changes in the industry
- businesses downsizing/closure
- limited employment opportunities in Indigenous, small and remote communities
- barriers to employment experienced by a target population
All projects reviewed aligned with the Labour Market Partnerships program objectives and eligible activities.
Partnerships
Alberta’s Ministry of Labour and Immigration confirmed that program officials conducted activities to support the formation and maintenance of partnerships as a part of the program design and delivery. The Ministry and project holders explained that partners’ expertise and contributions are all essential to project implementation and success.
The document review of 20 projects confirmed that:
- partnerships were established to support the delivery of all projects
- partners made financial and/or in-kind contributions
- the most common forms of in-kind contributions were expertise, as well as staff time to project administration and delivery, office/event space and usage of equipment
Project activities delivered with the support of partners included labour market and human resource research, career/job awareness, workforce strategy/plan development, community engagement to resolve labour market issues, and human resource tools development.
Performance measurement
As per the Labour Market Transfer Agreements’ Performance Measurement Strategy, Alberta collects data on:
- number of employers that participate in the industry, Indigenous, and regions streams of the Labour Market Partnerships program
- under-represented groups targeted under Labour Market Partnerships
Project holders collected additional data associated with project outputs and outcomes.
Challenges and lessons learned
Alberta program officials and key informants identified challenges related to project holder recruitment and setting up Labour Market Partnerships projects (costs or lack of expertise associated with developing a project proposal, restrictive project eligibility and criteria), as well as to program administration and monitoring (difficulties in developing and collecting measurable outcomes).
Actions of program officials and project characteristics that are conducive to the success of the program included:
- government officials supporting project holders during the application process and throughout the project implementation
- experience and expertise of the partners
- ensuring that appropriate partners are selected to participate in the project
- good understanding of the program requirements
- government officials increasing awareness about the Labour Market Partnerships program for potential partners
Key considerations for Labour Market Partnerships program and policy development
The following considerations emerged as part of the Labour Market Partnerships study:
Consideration #1: Considering that the current performance indicators do not reflect the diversity of activities funded under Labour Market Partnerships, it is important for ESDC and Alberta to discuss current Labour Market Partnerships funded activities to make recommendations on how to best report on results.
Consideration #2: It is essential to share lessons learned about successful Labour Market Partnerships projects. Particularly, for projects targeted to employers (such as workplace or employer-sponsored training), and those assisting communities and economic sectors dealing with labour market adjustment issues (contraction or expansion).
Foundational Learning Supports/Occupational Training ApprenticesFootnote 36
The objective of the program is to help apprentices become skilled tradespeople and to increase their labour market attachment. Program participants have generally chosen a career and are already attached to the labour market. The apprenticeship process involves on-the-job learning and technical training in a classroom setting.
Apprentices who have worked enough hours to qualify for EI can apply to receive EI Part I benefits while on training. The program provides financial assistance to EI eligible apprentices to help them offset the costs they incur while they attend technical training. The level of funding is based on the needs of apprentices, the location of the training, and any fees paid by the apprentices.Footnote 37
The profile of participants is presented in Table 19 by gender, age, sociodemographic group, and marital status. Information about their educational attainment, occupation and industry is based on the latest job they held prior to applying for EI benefits. Information about sociodemographic groups is self-reported.
Categories | Active claimants | Former claimants |
---|---|---|
Number of participants | 6,404 | 10,411 |
Gender: Female | 8% | 7% |
Gender: Male | 92% | 93% |
Age: 30 and under | 76% | 77% |
Age: 31 to 54 | 24% | 23% |
Age: 55 and over | 1% | <1% |
Sociodemographic group: Indigenous people | 5% | 4% |
Sociodemographic group: Persons with disabilities | 2% | 1% |
Sociodemographic group: Visible minorities | 7% | 7% |
Sociodemographic group: Recent immigrants | 2% | 1% |
Marital status: Married or common-law | 32% | 33% |
Marital status: Widow / divorced / separated | 3% | 3% |
Marital status: Single | 63% | 62% |
Education or skills level: High school or occupational training | 3% | 4% |
Education or skills level: On-the-job training | 5% | 5% |
Education or skills level: College, vocational education or apprenticeship training | 91% | 91% |
Education or skills level: University degree | <1% | <1% |
Top 3 occupational groups for FLS/OT-Apprentices
Active claimants:
- skilled crafts and trades workers = 85%
- other manual workers = 5%
- skilled sales and service personnel = 3%
Former claimants:
- skilled crafts and trades workers = 84%
- other manual workers = 4%
- skilled sales and service personnel = 3%
Top 3 industries for FLS/OT-Apprentices
Active claimants:
- construction = 55%
- manufacturing = 12%
- other services (except public administration) = 8%
Former claimants:
- construction = 50%
- manufacturing = 13%
- other services (except public administration) = 9%
Labour market outcomes
The labour market outcomes are based on individuals who began their participation during the 2010 to 2012 period. Statistics focus on 5 years before program participation and 5 years after the program start year.
Active claimants
As shown in Chart 8, program participants increase their average earnings from $22,746 in the fifth year pre-program to $66,665 in the fifth year after the program start year.
The proportion of employed participants declines slightly from the program start year (99%) but remains around 96% for the annual post-program average. The proportion of participants on EI Part I decreases from 100% in the program start year to 30% in the fifth year after the program start year. Participants decrease their dependence on income support from 19% in the program start year to 5% in the fifth year after participation.
Text Description – Chart 8
Year pre-post participation | Employment earnings for active claimants ($) |
---|---|
5 years pre-program | 22,746 |
4 years pre-program | 26,290 |
3 years pre-program | 28,884 |
2 years pre-program | 32,453 |
1 year pre-program | 39,853 |
Program start year | 40,825 |
1 year post-program | 52,836 |
2 years post-program | 61,290 |
3 years post-program | 66,910 |
4 years post-program | 67,989 |
5 years post-program | 66,665 |
Former claimants
As shown in Chart 9, program participants increase their average earnings from $24,792 in the fifth year pre-program to $67,559 in the fifth year after the program start year.
Text Description – Chart 9
Year pre-post participation | Employment earnings for active claimants ($) |
---|---|
5 years pre-program | 24,792 |
4 years pre-program | 29,323 |
3 years pre-program | 32,621 |
2 years pre-program | 36,824 |
1 year pre-program | 41,694 |
Program start year | 49,988 |
1 year post-program | 59,330 |
2 years post-program | 65,004 |
3 years post-program | 68,677 |
4 years post-program | 69,070 |
5 years post-program | 67,559 |
The proportion of employed participants declines slightly after the program start year (99%) but remains around 96%. The proportion of participants on EI Part I decreases from 76% in the program start year to 23% in the fifth year after the program start year. Participants decrease their dependence on income support from 9% in the program start year to 4% in the fifth year after participation.
Conclusions and recommendations
The Canada-Alberta LMDA is the largest annual investment in active labour market programs and services in the province. Based on the findings presented in this report, the EBSMs are meeting the objective of assisting individuals to obtain or keep employment through various active employment programs, including training or employment assistance services.
Summary of findings
Overall, incremental impacts demonstrate that participants in FLS/OT improve their labour market attachment (employment and earnings) and reduce their dependence on government income supports (that is, the combination of EI and SA) compared to similar non-participants.
FLS/OT also improves the labour market attachment of most subgroups of active and former EI claimant participants. Most former EI claimants also reduce their dependence on government income supports following participation.
A regional analysis of FLS/OT found that former claimant participants outside of Calgary improved their labour market attachment and reduced their dependence on government income supports compared to similar non-participants. Active claimant participants in Calgary and outside of Calgary increase their incidence of employment relative to their respective comparison groups. Other results were mixed and not statistically significant.
Moreover, the social benefits of participating in FLS/OT exceeds the costs of investments over time.
Active EI claimants in CEAS alone have a small negative, but not statistically significant, impact on the probability of being employed, as well as decreases in employment earnings. These participants also increase their dependence on government income supports.
A series of supplemental studies address information gaps previously identified in LMDA evaluations for Self-Employment, Integrated Training, Labour Market Partnerships and FLS/OT-apprentices. Each study identified lessons learned, best practices and challenges, and issued when relevant considerations for policy design and development. Overall, it was found that:
- the Self-Employment program helps carefully selected participants to create employment for themselves by providing them with a range of services
- in Alberta the focus of Integrated Training is to assist participants in acquiring work experience, leading to the acquisition of new employment-related skills or the improvement in current skills
- after participating in FLS/OT, apprentices increase their employment earnings and decrease their dependence on government income supports
- Alberta uses Labour Market Partnerships to assist employers, communities and/or industries to address their labour force adjustment and human resource needs
- the current performance indicators do not reflect the diversity of funded activities; therefore, it is important for ESDC and provinces/territories to discuss current funded activities to make recommendations on how to best report on results
Recommendations
Since 2012, 15 qualitative and quantitative evaluation studies have been used to address issues and questions related to EBSM design, delivery and effectiveness. The quantitative studies successfully assessed the effectiveness and efficiency of EBSMs by producing incremental impacts and cost-benefit analysis. The qualitative studies identified specific challenges, lessons learned, and best practices associated with the design and delivery of EBSMs. Each study included key considerations for program and policy development or recommendations.
The recently completed evaluation of the Workforce Development Agreements complements the LMDA qualitative studies. This evaluation was also supported by literature reviews and provided unique insights into challenges and lessons learned to assist persons with disabilities, immigrants and those further removed from the labour market.
Most results from this evaluation stem from the conduct of advance causal analysis whereby impacts found could be attributed to a specific EBSM. These analyses are predicated on having access to high quality administrative records, thereby confirming the importance of the capacity to leverage and integrate relevant administrative data.
From these main findings, 2 key recommendations emerge:
Recommendation #1: Alberta is encouraged to share and discuss lessons learned, best practices and challenges associated with the design and delivery of programs and services. Discussions are encouraged with ESDC, at the bilateral or multilateral levels, as well as with service delivery network if necessary.
Recommendation #2: Alberta is encouraged to pursue efforts to maintain and strengthen data collection provisions in support of reporting, performance measurement and data-driven evaluations at the national and provincial levels.
References
Allin, S., Corscadden, L., Gapanenko, K., and Grignon, M. Lifetime Distributional Effects of Publicly Financed Health Care in Canada. CIHI,2013.
Athey, S., and Imbens, G. Machine Learning Methods Economists Should Know About. Annual Review of Economics, 2019.
Douwere G. and Huju Liu. The Entry into and Exit out of Self-employment and Business Ownership in Canada. Statistics Canada, 2018.
Employment and Social Development Canada. Cost-Benefit Analysis: Incorporating Public Health Care Costs Savings in the Context of the Labour Market Programs Evaluation. Internal document, 2022.
Employment and Social Development Canada. Cycle II of the Evaluation of the Labour Market Development Agreements: Cost-Benefit Analysis of Employment Benefits and Support Measures. Internal document, 2015.
Employment and Social Development Canada. Design and delivery of the Integrated Training program in Alberta. Internal report, 2019.
Employment and Social Development Canada. Employment Insurance Monitoring and Assessment Reports. 2011 to 2022.
Employment and Social Development Canada. Evaluation of the Labour Market Development Agreements, Design and delivery of the Self-Employment program in Alberta. Internal report, 2021.
Employment and Social Development Canada. Horizontal evaluation of the Labour Market Development Agreements, Design and delivery of the Labour Market Partnerships program in Alberta. Internal document, 2021.
Employment and Social Development Canada. Labour Market Development Agreements Process for Determination of Similarity. Internal document, 2012.
Employment and Social Development Canada. Technical Report on the Analysis of Employment Benefits and Support Measures Profile, Outcomes and Medium-Term Incremental Impacts from 2010 to 2017. Internal report, 2021.
Employment and Social Development Canada. Third Cycle for the Horizontal Evaluation of the Labour Market Development Agreements: Quantitative Methodology Report. Internal document, 2019.
Lechner, M. Modified Causal Forests for Estimating Heterogeneous Causal Effects. 2019.
Appendix A. List of 7 studies included in the Canada-Alberta synthesis report
Overview of 7 studies included in this report.
Study 1: Examination of medium-term outcomes from 2010 to 2017
Evidence generated: profile of active and former EI claimants; outcomes by claimant type and by subgroup
Methods: before and after results of program participation
Reference period: 2010 to 2012 participants
Observation period: up to 12 years (5 years before participation, 1 to 2 years of participation, and up to 5 years after participation)
Study 2: Estimation of medium-term incremental impacts from 2010 to 2017
Evidence generated: incremental impacts for active and former EI claimants; incremental impacts by subgroup; profile and socio-demographic characteristics of participants
Methods: non-experimental method using propensity score matching in combination with difference-in-differences; statistical profiling
Reference period: 2010 to 2012 participants
Observation period: up to 7 years (1 to 2 years in program, and up to 5 years after participation)
Study 3: Cost-benefit Analysis of Employment Benefits and Support Measures in Alberta
Evidence generated: cost-benefit analysis
Methods: non-experimental method using propensity score matching in combination with difference-in-differences; cost analysis
Reference period: 2010 to 2012 participants
Observation period: 5 years post-program for CEAS; 10 years post-program for FSL/OT
Study 4: Cost-Benefit Analysis of Employment Benefits and Support Measures in Alberta: Incorporating Public Health Care Costs Savings in the Context of the Labour Market Programs Evaluation
Evidence generated: cost-benefit analysis
Methods: estimation of adjusted annualized healthcare costs
Reference period: 2010 to 2012 participants
Observation period: 5 years post-program for CEAS; 10 years post-program for FSL/OT
Study 5: Design and delivery of the Integrated Training program in Alberta
Evidence generated: program design and delivery; challenges and lessons learned
Methods: non-experimental approach (from cycle ii); statistical analysis; document review; 21 semi-structured telephone interviews with 29 key informants
Reference period: 2015 to 2017 participants
Observation period: 2015 to 2019
Study 6: Design and delivery of the Self-Employment program in Alberta
Evidence generated: program design, delivery and success; define outcomes attributed to the program; fill in knowledge gaps; challenges and lessons learned
Methods: document review; statistical analysis of administrative data; Canadian self-employment literature and statistics; 7 semi-structured telephone interviews with 14 key informants; statistical analysis of administrative data; survey of Self-Employment participants in the province
Reference period: 2015 to 2017 participants
Observation period: 2015 to 2020
Study 7: Design and delivery of the Labour Market Partnerships program in Alberta
Evidence generated: program design and delivery; challenges and lessons learned
Methods: document review; questionnaire completed by the province; 8 key informant interviews
Reference period: 2018 to 2020
Observation period: design and delivery at the time of the data collection
Page details
- Date modified: