Evaluation of the Canada-Prince Edward Island Labour Market Development Agreement

On this page

• List of abbreviations
• List of figures
• List of tables
• Executive summary
• Management response
• Introduction
• Canada-Prince Edward Island Labour Market Development Agreement
  • Employment benefits
  • Support measures
  • Eligible participants covered in this study
  • Average share of funding per action plan equivalent
• Methodology
  • Incremental impacts analysis
  • Factors accounted for in the cost-benefit analysis
  • Strengths and limitations of the studies
  • Overview of the studies summarized in this report
• Evaluation findings
  • Profile of participants
  • Incremental impacts for active and former EI claimants
  • Incremental impacts by subgroups of participants
  • Cost-benefit analysis
  • Outcomes for active and former EI claimants
• Supplemental studies
  • Self Employ PEI
  • Work Experience PEI
  • Labour Market Partnerships
  • Research and Innovation
  • Training PEI Apprentices
• Conclusions and recommendations
• References
• Appendix A. List of 8 studies included in this synthesis report

Alternate formats

Executive summary

The Canada-Prince Edward Island Labour Market Development Agreement (LMDA) is a bilateral agreement between Canada and Prince Edward Island for the design and delivery of Employment Benefits and Support Measures (EBSMs).

The objective of EBSMs is to assist individuals to obtain or keep employment through various active employment programs, including training or employment assistance services. Successful delivery of EBSMs is expected to result in participants receiving needed services, a quick return to work, and savings to the Employment Insurance (EI) account.

Programs and services delivered by Prince Edward Island have to correspond to the EBSM categories defined under the EI Act. The following is a short description of the EBSMs examined in the evaluation:

Training PEI helps job seekers to gain the skills and education needed to find and maintain employment. The program provides financial assistance to help participants complete a variety of post-secondary certificate or diploma programs. The current data does not identify what proportion of funds are allocated to each training type.^{Footnote 1}

Employ PEI is an employment program to help job seekers develop new skills and gain work experience through on-the-job training. By providing a temporary wage subsidy to an employer, it helps create long term employment opportunities and assists the employer with the cost of hiring and training a new employee.

Work Experience PEI is a short-term employment program that provides job seekers with an opportunity to work on a project with a local organization. The on-the-job work experience will help the job seeker develop new skills needed for employment in the workforce.

Self Employ PEI is an employment program developed to help job seekers who want to launch their own business. This program provides financial support and business counselling to new entrepreneurs during their first year of operation.

Employment Assistance Services (EAS) aim to assist unemployed individuals to prepare for, obtain and maintain employment. Services can include employment counselling, career planning, personal and professional development, job search training, labour connections and provision of labour market information.

Labour Market Partnerships (LMP) is a program that provides funding to projects that encourage and support employers and/or employer associations and communities to develop and implement strategies to deal with labour force adjustments and meet human resource requirements.

Research and Innovation is program that supports research and innovative projects to identify better ways of helping persons prepare for, return to or keep employment and be productive participants in the labour force.

The LMDA investment

In fiscal year 2020 to 2021, Canada transferred approximately $29.8 million (including nearly $2.7 million in administration funds) to Prince Edward Island.

Table i provides an overview of the share of funding allocated to EBSMs and the average cost per participant. The average cost per participant is calculated based on the 2010 to 2012 data from the EI Monitoring and Assessment Reports. The 2010 to 2012 period corresponds with the cohort of participants selected for incremental impacts and cost-benefit analysis in the LMDA evaluation.

• Sources: EI Monitoring and Assessment Reports for fiscal years^{Footnote 4} 2010 to 2011 and 2011 to 2012.
• Note: total spending may not add up to 100% due to rounding.

Compared to the 2010 to 2012 period, the LMDA budget allocation varied for few programs and services in 2020 to 2021. For example, investments in Work Experience PEI decreased from 9% to 2%, and investments in EAS increased from 17% to 21% of total allocation.

Evaluation objectives

Building on the success of previous LMDA evaluation cycles, the aim of this evaluation is to fill in knowledge gaps about the effectiveness, efficiency as well as design and delivery of EBSMs in Prince Edward Island.

Evaluation methodology

The findings in this report are drawn from 8 separate evaluation studies carried out at the provincial level. These studies examine issues related to program effectiveness, efficiency, and design and delivery. A mix of qualitative and quantitative methods are used, including:

• incremental impact analysis for participants who began an intervention between 2010 and 2012
• outcome analysis
• cost-benefit analysis (including savings to health care)
• key informant interviews with 18 program officials, service providers, sponsors, project holders and key stakeholders
• questionnaires
• a survey of Self Employ PEI participants
• document and literature reviews

The incremental impacts are estimated for 2 types of EI claimants:

• active claimants are participants who started an EBSM intervention while collecting EI benefits
• former claimants are participants who started an EBSM intervention up to 3 years after the end of their EI benefits^{Footnote 5}

Key findings

Across Prince Edward Island, nearly 5,500 EI active and former claimants began participating in LMDA programs and services between 2010 and 2012.

Effectiveness and efficiency of EBSMs

Overall, incremental impacts demonstrate that participation in EBSMs improves labour market attachment and reduces dependence on government income supports compared to similar non-participants. For example, active EI claimants in Training PEI increased their incidence of employment and increased employment earnings. Active EI claimants in Employ PEI had an average annual increase in earnings of $2,722 compared to non-participants over the 4 years post participation. Former claimants in Employ PEI had an annual average change in probability of being employed of 4.8 percentage points compared to non-participants. A subgroup analysis shows that Training PEI improves the labour market attachment and reduces the dependence on income support for all subgroups of participants. EAS-only interventions were found to improve incidence of employment and decrease use of EI benefits for female participants. As well, for most interventions the social benefits of participating in EBSMs exceeds the initial investment costs over time.

Figure i presents the incremental impacts on the incidence of employment for active and former claimants by EBSM. The estimates can be interpreted as a chance in the probability of being employed following participation. For example, participation in Training PEI increases the probability of being employed by 4.9 percentage points for active EI claimants relative to non-participants.

Figure ii presents the annual average increase in employment earnings for active and former claimants over the post-participation period.

As shown in Figure iii, overall active claimants in Training PEI and EAS-only reduce their dependence on government income supports. All other impacts are not statistically significant.

Table ii presents the number of years required for the social benefits to exceed program costs. Social benefits to participation exceed initial investment costs over a period ranging from 1.3 to 8.7 years.

Supplemental studies

A series of supplemental studies addresses information gaps previously identified in LMDA evaluations regarding the design and delivery, challenges and lessons learned for Self Employ PEI, Work Experience PEI, LMP, and Research and Innovation.

Most of these interventions are not suitable for incremental impact analysis. For example, LMP and Research and Innovation do not collect participant information. As a result, a mix of qualitative and quantitative methods are used to examine these EBSMs in detail. Key considerations are included to help guide future program and policy discussions.

Self Employ PEI study

The evaluation found that the Self Employ PEI program aims to assist participants in creating employment for themselves by providing them with a range of services.

Based on a survey, it was found that 2 to 4 years after program participation:

• 22% of self-employment businesses were launched in other services^{Footnote 6}
• nearly 66% of survey respondents who launched a self-employment business were still in operation in winter 2020
• 73% of respondents said that they are financially about the same or better off after the program
• 73% of respondents said that their household net worth is about the same or higher after the program

The survey examined the contribution of the program to the success of self-employment businesses. Over 74% of survey respondents who launched a self-employment business rate the following services and training as very or somewhat important to the business launch, operation and success:

• one-on-one coaching
• financial assistance for living allowance during participation
• assistance with developing a business plan
• financial management training
• financial assistance for costs associated with a business start-up

Work Experience PEI study

Work Experience PEI is a short-term employment program that provides job seekers with an opportunity to work on a project with a local organization. The on-the-job work experience will help the job seeker develop new skills needed for employment in the workforce.

In addition to gaining work experience, key informants expected participants to develop transferable skills and to enhance their career development and job search abilities. Sponsors can benefit from Work Experience PEI through increased capacity by having access to labour, implementing their projects, and developing partnerships with other organisations. At the community level, Work Experience PEI projects are expected to support the local economy.

Key informants highlighted the importance of aligning the motivations of sponsors and participants with the objective of Work Experience PEI to prepare individuals for employment. The removal of material related costs from eligible project costs has ensured that sponsors focus their attention on participants' needs.

Labour Market Partnerships study

The LMP program in Prince Edward Island is a program that provides funding to projects that encourage and support employers and/or employer associations and communities to develop and implement strategies to deal with labour force adjustments and meet human resource requirements. It includes a wide range of funded activities, such as:

• human resource planning, including:
  • development and dissemination of labour market information
  • development of human resources best practices and tools
  • development of a human resources strategy to address current workforce demands and support future industry growth
• labour market adjustment, including:
  • raising awareness about available or future in-demand careers and jobs
  • providing human resources advice to small businesses owners
• skills development, including:
  • development and delivery of human resources workshops
  • piloting an alternative model for training delivery

The Department of Education and Lifelong Learning^{Footnote 7} and key informants confirm that program officials carried out activities to support the formation and maintenance of partnerships (with sector and community stakeholders) as a part of the program design and delivery. The Department and key informants explained that partners' expertise, network and financial contribution are all essential to project implementation and success.

Research and Innovation study

Research and Innovation is a program that supports research and innovative projects to identify better ways of helping persons prepare for, return to or keep employment and be productive participants in the labour force.

From 2014 to 2016, Research and Innovation funded a project to assist secondary school students with high school graduation, post-secondary transitions and entry into the labour force.

Training PEI Apprentices study

The objective of the program is to help apprentices become skilled tradespeople and to increase their labour market attachment. Program participants have generally chosen a career and are already attached to the labour market. The apprenticeship process involves on-the-job learning and technical training in a classroom setting.

The evaluation found that active EI claimants increased their average earnings from $13,672 in the fifth year pre-program to $44,951 in the fifth year after the program start year. Former EI claimants increased their average earnings from $17,372 in the fifth year pre-program to $53,307 in the fifth year after the program start year. After participating in the program, both active and former claimants also decreased their dependence on government income supports.

Recommendations

Since 2012, 15 qualitative and quantitative studies addressed issues and questions related to EBSM design, delivery and effectiveness. The quantitative studies successfully assessed the effectiveness and efficiency of EBSMs by producing incremental impacts and cost-benefit analysis. The qualitative studies help to contextualize the findings from the quantitative studies and to identify specific challenges, lessons learned and best practices associated with the design and delivery of EBSMs. Each study included key considerations for program and policy development or recommendations.

In addition, the recently completed evaluation of the Workforce Development Agreements complements the LMDA qualitative studies. This comprehensive evaluation provided unique insights into challenges and lessons learned to assist persons with disabilities, immigrants and those further removed from the labour market.

Most results from this evaluation stem from the conduct of advance causal analysis whereby impacts found could be attributed to a specific EBSM. These analyses are predicated on having access to high quality administrative records, thereby confirming the importance of the capacity to leverage and integrate relevant administrative data.

From these main findings, 2 key recommendations emerge:

Recommendation #1: Prince Edward Island is encouraged to share and discuss lessons learned, best practices and challenges associated with the design and delivery of programs and services. Discussions are encouraged with ESDC, at the bilateral or multilateral levels, as well as with service delivery networks if necessary.

Recommendation #2: Prince Edward Island is encouraged to pursue efforts to maintain and strengthen data collection provisions in support of reporting, performance measurement and data-driven evaluations at the national and provincial levels.

Management response

Prince Edward Island through the Department of Economic Growth, Tourism and Culture^{Footnote 8} collaborated with 11 provinces/territories to undertake the evaluation of the Labour Market Development Agreement. The Department of Economic Growth, Tourism and Culture accepts the evaluation recommendations and has responded to the following actions in relation to the conclusions of this evaluation:

Recommendation #1

Prince Edward Island is encouraged to share and discuss lessons learned, best practices and challenges associated with the design and delivery of programs and services. Discussions are encouraged with ESDC, at the bilateral or multilateral levels, as well as with service delivery networks if necessary.

Prince Edward Island proactively engages in bilateral and multilateral forums to share and discuss best practices and challenges associated with the design and delivery of labour market development agreements. Prince Edward Island has always had a good relationship as an equal partner with our federal counterparts and informal and formal bilateral discussions provides opportunities to explain Prince Edward Island's position as a small jurisdiction. Historically Prince Edward Island has benefitted from multilateral engagement through the Forum of Labour Market Ministers and forums such as the National Evaluation Steering Committee for both the LMDA and Workforce Development Agreement. Multilateral engagement provides opportunities to learn from other PTs through strategic discussions and ongoing collaboration. Prince Edward Island will look forward to the continuation of a collaborative partnership based on transparency and respect. Prince Edward Island remains committed to federal-provincial collaboration, and multilateral engagement with provincial and territorial counterparts across the country as we work cooperatively to address the wide range of challenges Canadians are experiencing in the labour market.

Recommendation #2

Prince Edward Island is encouraged to pursue efforts to maintain and strengthen data collection provisions in support of reporting, performance measurement and data-driven evaluations at the national and provincial levels.

In 2017 the Performance Measurement Plan was developed multilaterally with ESDC and the provincial and territorial governments as part of the amended LMDA and the new Workforce Development Agreement. The Performance Measurement Plan, detailed performance indicator information, data collection methodologies, frequency of reporting, data sources and data elements for all programs and services as well as participant and employer indicators under the amended LMDA and Workforce Development Agreement. Prince Edward Island invested significant funds to upgrade IT systems to ensure reliable and accurate data collection and reporting to support performance measurement requirements and evaluations at the national and provincial level. Prince Edward Island agrees to continue to maintain data collection as outlined in the Performance Measurement Plan.

If there is a requirement to strengthen data collection as a result of expanded eligibility under the LMDAs or for the requirement to conduct Gender Based Analysis Plus as directed by the federal Treasury Board of Canada Secretariat for evaluations throughout the federal public service, Prince Edward Island is open to work multilaterally with ESDC and provincial and territorial governments to negotiate future labour market transfer agreements. Any proposed changes to data collection must comply with Prince Edward Island's Freedom of Information and Protection of Privacy Act R.S.P.E.I. 1988, Cap. F-15.01. Proposed changes to data collection will be agreed upon and reflected in the Performance Measurement Plan and in the Exchange of Information and Data Sharing Arrangements Annex of the LMDA.

Introduction

Employment and Social Development Canada (ESDC) worked jointly with Prince Edward Island to undertake the third cycle evaluation (2018 to 2023) of the Canada-Prince Edward Island Labour Market Development Agreement (LMDA).

The first LMDA evaluation cycle was carried out from 1998 to 2012. It involved the conduct of separate formative and summative evaluations under the guidance of a bilateral Joint Evaluation Committee.

Building on lessons learned and best practices from the first cycle, the second LMDA evaluation cycle was undertaken between 2012 and 2017. The second cycle was designed and implemented under the guidance of a federal-provincial/territorial LMDA Evaluation Steering Committee. The work was supported by bilateral discussions at the Joint Evaluation Committee.

The third LMDA evaluation cycle builds on the success of the second cycle. It aims to fill in knowledge gaps about the effectiveness, efficiency, and design and delivery of LMDA-funded Employment Benefits and Support Measures (EBSMs). The evaluation cycle was designed and implemented under the guidance of a federal-provincial/territorial LMDA Evaluation Steering Committee composed of ESDC and 12 participating provinces and territories.

For Prince Edward Island, this report presents a summary of third cycle evaluation findings from 8 studies.

Canada-Prince Edward Island Labour Market Development Agreement

The LMDAs are bilateral agreements between Canada and each province and territory for the design and delivery of EBSMs. They were established under Part II of the 1996 Employment Insurance (EI) Act.

In fiscal year^{Footnote 9} 2020 to 2021, Canada transferred nearly $29.8 million to Prince Edward Island.^{Footnote 10} Under the agreement, Prince Edward Island is responsible for the design and delivery of programs and services aimed at assisting individuals to prepare for, obtain, and maintain employment.

LMDA programs and services are classified under 2 categories:

• Employment benefits^{Footnote 11,Footnote 12} fall into 4 sub-categories:
  • Training PEI
  • Employ PEI
  • Self Employ PEI
  • Work Experience PEI
• Support measures fall into 3 subcategories:
  • Employment Assistance Services^{Footnote 13}
  • Labour Market Partnerships
  • Research and Innovation.

Prince Edward Island has the flexibility to adapt EBSMs to their jurisdiction's context as long as they are consistent with Part II of the EI Act.^{Footnote 14}

The objective of EBSMs is to assist individuals to obtain or keep employment through various active employment programs, including training or employment assistance services. Successful delivery of EBSMs is expected to result in participants receiving needed services, a quick return to work, and savings to the EI account.

Programs and services examined in this study include employment benefits and support measures.

Employment benefits

Training PEI helps job seekers to gain the skills and education needed to find and maintain employment. The program provides financial assistance to help participants complete a variety of post-secondary certificate or diploma programs.

Employ PEI is an employment program to help job seekers develop new skills and gain work experience through on-the-job training. By providing a temporary wage subsidy to an employer, it helps create long term employment opportunities and assists the employer with the cost of hiring and training a new employee.

Work Experience PEI is a short-term employment program that provides job seekers with an opportunity to work on a project with a local organization. The on-the-job work experience will help the job seeker develop new skills needed for employment in the workforce.

Self Employ PEI is an employment program developed to help job seekers who want to launch their own business. This program provides financial support and business counselling to new entrepreneurs during their first year of operation.

Support measures

Employment Assistance Services (EAS) is a service that assists unemployed individuals to prepare for, obtain and maintain employment. Services can include employment counselling, career planning, personal and professional development, job search training, labour connections and provision of labour market information.

Labour Market Partnerships (LMP) is a program that supports projects that encourage and support employers and/or employer associations and communities to develop and implement strategies to deal with labour force adjustments and meet human resource requirements.

Research and Innovation is a program that supports research and innovative projects to identify better ways of helping persons prepare for, return to or keep employment and be productive participants in the labour force.

Eligible participants covered in this study

The incremental impacts are estimated for active and former EI claimants:

• active claimants are participants who started an EBSM intervention while collecting EI benefits
• former claimants are participants who started an EBSM intervention up to 3 years after the end of their EI benefits^{Footnote 15}

Average share of funding per Action Plan Equivalent

Table 1 provides an overview of the share of funding allocated to EBSMs and the average cost per Action Plan Equivalent for active and former claimants in Prince Edward Island. It is noted that the average cost per participant is calculated based on the 2010 to 2012 data from the EI Monitoring and Assessment Reports. The 2010 to 2012 period corresponds with the cohort of participants selected for incremental impacts and cost-benefit analysis in the LMDA evaluation.

From the 2010 to 2012 period to the 2020 to 2021 fiscal year,^{Footnote 16} investments in Training PEI decreased by 2 percentage points and those in Work Experience PEI decreased by 7 percentage points. The largest increases in funding are noted for EAS (+4 percentage points) and Employ PEI (+3 percentage points).

• Sources: EI Monitoring and Assessment Reports for fiscal years 2010 to 2012 and 2020 to 2021.
• Note: total spending may not add up to 100% due to rounding.

Methodology

This section presents key aspects of the quantitative analyses carried out as part of the LMDA studies.

All quantitative analyses are based on administrative data from the EI Part I (EI claim data) and Part II (EBSM participation data). The EI Part I and II data are then linked to the T1 and T4 taxation files from the Canada Revenue Agency. Incremental impact and cost-benefit analyses are based on up to 100% of participants in Prince Edward Island who began their EBSM participation in 2010 to 2012.

The 2010 to 2012 timeframe was selected in order to assess the impacts of EBSMs in the years following participation. Impacts were assessed over a period of at least 4 years after program completion up to the 2017 calendar year (most recent available information at the time of this evaluation).

Incremental impact analysis^{Footnote 19}

Program effectiveness is assessed by estimating incremental impacts from EBSM participation on participants' labour market experience. That is, earnings from employment and self-employment, incidence of employment, use of EI, use of social assistance (SA), and dependence on government income supports after participation.

In Prince Edward Island, incremental impacts were estimated for active and former EI claimant participants in Training PEI, Employ PEI and active EI claimant participants in EAS. Incremental impacts for Work Experience PEI were not produced due to small sample size or because the models used to estimate the incremental impacts did not balance. Having balanced models is an essential condition to ensure that participants and non-participants are similar.

The role of the incremental impact analysis is to isolate the effects of participation from other factors. To achieve this, the incremental impact analysis compares the labour market experience of participants before and after their participation with that of similar non-participants. Figure 1 presents an example of incremental impact calculation.

The main estimator used is propensity score kernel matching technique combined with difference-in-differences estimator. Moreover, 3 different state-of-the-art estimation techniques (Inverse Probability Weighting, Nearest Neighbour and Cross-sectional Matching) were conducted separately for each type of EBSM and EI claimant to validate the impact estimates.

As for previous LMDA evaluation studies, the Action Plan Equivalent is the unit of analysis used. Action Plan Equivalents regroup all EBSMs received by an individual within less than 6 months between the end of one EBSM and the start of the next. Action Plan Equivalents are categorized based on the longest EBSM they contain, except for EAS-only Action Plan Equivalents which include only EAS interventions.

The analysis includes Action Plan Equivalents that consist only of LMDA interventions. Action Plan Equivalents that include a combination of LMDA and other labour market programs funded by ESDC, were excluded from the participant sample.

The matching of participants and comparison group members used up to 75 socio-demographic and labour market variables observed over 5 years before participation. Two different comparison groups were used to measure impacts for active and former EI claimants:

For active claimants, incremental impacts were measured relative to a comparison group of active claimants who were eligible to, but did not, participate in EBSMs during the reference period.

For former claimants, the comparison group was created using individuals who participated in EAS only during the reference period.^{Footnote 20} In other words, the experience of former claimants in Training PEI interventions is compared to the experience of former claimants who received EAS only. This is a conservative approach given the fact that participation in EAS can lead to limited effects on labour market outcomes.

Due to this difference in measurement, incremental impacts estimated for active claimant participants should not be directly compared to those of former claimant participants.

Impacts are generated over 4 years for Training PEI and Employ PEI while a fifth year is estimated for participants in EAS.

Factors accounted for in the cost-benefit analysis ^{Footnote 23,Footnote 24}

Building on the results of the incremental impacts, program efficiency is assessed through a cost-benefit analysis. The analysis compares the participants' cost of participating and the government's cost of delivering the program to the benefits associated with the program. Overall, this analysis provides insights on the extent to which the program is efficient for the society (that is, for both participants and the government).

Sources of data and information

The analysis considers all the quantifiable costs and benefits directly related to EBSM delivery and participation that can be measured given the information available. The analysis is comprehensive in that it accounts for the vast majority of possible direct costs and benefits.

However, the analysis does not account for all costs and benefits. For example, there are factors that can lead to an understatement of the benefits (such as positive spillovers to other family members) and other factors that can lead to an overstatement of the benefits (such as effects on skill prices or displacement). ^{Footnote 25}

This study relied on integrated data from the EI Part I and II Databank and Income Tax records from the Canada Revenue Agency. Information about earnings, use of EI, and use of SA was taken from the study of incremental impacts.^{Footnote 26} The program costs were calculated using information available in the EI Monitoring and Assessment Reports.

Relative to the previous cycle of evaluation, the methodology has been extended to incorporate one of the indirect health benefits associated with increased labour market attachment. In particular, the methodology includes an estimate of the change in public health care cost due to the decline in health care utilization resulting from program participation.

Data on average public healthcare costs by income quintiles are taken from the report Lifetime Distributional Effects of Publicly Financed Health Care in Canada (2013) by the Canadian Institute for Health Information.

Program costs are measured using information on LMDA expenditures and new interventions reported in the EI Monitoring and Assessment Report. Other costs and benefits are assessed using integrated administrative data from the EI Part I and II databank and the Canada Revenue Agency.

Incremental impacts measured over the second year of participation and up to 5 post-program years are discounted by 3% to bring them to a common base with the program cost and benefits incurred in the program start year. This 3% rate accounts for the interest the government could have collected if the funds used to pay for the program had been invested. Incremental impacts are estimated using 2010 constant dollars and this accounts for inflation.

Costs and benefits accounted for in the calculations

Program cost: cost incurred by the government for delivering the program (that is, administration and direct program costs calculated from data reported in the EI Monitoring and Assessment Reports).

Marginal social cost of public funds: loss incurred by society when raising additional revenues such as taxes to fund government spending. The value is estimated at 20% of the program cost, sales taxes, income taxes, impacts on EI and impacts on SA paid or collected by the government.

Foregone earnings: estimated net impacts on participants' earnings during the participation period. During labour market program participation, some individuals have lower earnings than what they would have received if they had not participated.

Employment earnings: incremental impacts on participants' earnings during and after participation. In-program earnings represent the foregone earnings for participants.

Fringe benefits: the employer-paid health and life insurance as well as pension contributions. They are estimated at 15% of the incremental impacts on earnings.

Federal and provincial income taxes: incremental impacts on federal, provincial and territorial taxes paid by participants.

Sales taxes: the sales taxes paid by participants estimated as incremental impacts on earnings multiplied by the propensity to consume (97%), the proportion of household spending on taxable goods and services (52%) and the total average federal and provincial sales tax rate (11%).

Social assistance and Employment Insurance benefits collected: incremental impacts on SA and EI benefits use by participants following participation.

Canada Pension Plan contribution and Employment Insurance premiums: these contributions and premiums were identified from the Canada Revenue Agency data and then, the incremental impacts on Canada Pension Plan contributions and EI premiums were estimated.

Public health care costs savings: estimated impact of participation in EBSMs on public health care costs shown as an average change per participant over the post-program period examined.

Strengths and limitations of the studies

One of the key strengths from the studies is that all quantitative analyses are based on administrative data rather than survey responses. Compared to survey data, administrative data are not subject to recall errors or response bias.

The propensity score models used to match participants and non-participants for the incremental impact analyses are judged to be robust. In part this is because they were based on 5 years of pre-participation data. Moreover, these models are based on a vast array of variables including sociodemographic characteristics, location, skill level related to last occupation, and indicators of labour market attachment.

However, the matching process can be further refined for specific subgroups if the following information was available in the future, broadening the scope for greater Gender-based Analysis Plus examination:

• persons with disabilities: the type and severity of the disability, and the capacity/willingness to work full-time
• recent immigrants: the country of origin, the proficiency in English or French, and the relevance of credentials and work experience
• visible minorities: place of birth; individuals who are born outside of Canada face different challenges compared to those born in Canada

Refining the matching process for population subgroups could broaden the scope for greater Gender-based Plus Analysis.

Sensitivity analysis and the use of alternative estimation methods have increased confidence in the incremental impact estimates. However, one limitation with the propensity score matching techniques is that no one can be fully sure the impacts are not influenced by factors not captured in the data.

The cost-benefit analysis accounted for all quantifiable costs and benefits directly attributable to the EBSMs and could be estimated with the available administrative data. It is furthered strengthened by incorporating one of the indirect benefits, which is the change in public health care expenditures associated with program participation. However, the analysis did not account for non-quantifiable factors that can lead to an understatement of the benefits (for example, positive spillovers to other family members) and factors that can lead to an overstatement of the benefits (for example, effects on skill prices or displacement).

In some studies that use qualitative data collection methods, the number of key informants interviewed is relatively small in Prince Edward Island. Responses provided by key informants reflect their own experience and may not be fully representative of Prince Edward Island.

Overview of the studies summarized in this report

The findings in this report are drawn from 8 separate studies conducted at the provincial level. These studies examine issues related to program effectiveness, efficiency, design/delivery and use a mix of qualitative and quantitative methods. Appendix A presents an overview of these studies. The studies are:

• Examination of the medium-term outcomes from 2010 to 2017
• Estimation of the medium-term incremental impacts from 2010 to 2017
• Cost-benefit analysis of Employment Benefits and Support Measures in Prince Edward Island
• Cost-benefit Analysis: Incorporating Public Health Care Costs Savings in the Context of the Labour Market Programs Evaluation in Prince Edward Island
• Design and delivery of the Work Experience Prince Edward Island program
• Design and delivery of Self Employ PEI
• Design and delivery of the Labour Market Partnerships program in Prince Edward Island
• Design and delivery of the Research and Innovation support measure in Prince Edward Island

Evaluation findings

Profile of participants

In Prince Edward Island, nearly 5,500 EI active and former claimants participated in LMDA programs and services between 2010 and 2012.

The profile of participants is presented in Table 2 by gender, age, sociodemographic group, and marital status. Information about their educational attainment, occupation and industry is based on the latest job they held prior to applying for EI benefits. Information about sociodemographic groups is self-reported.

• Note: Values may not equal 100% due to rounding or missing information.

Top 3 occupational groups for active claimants:

• other manual workers = 19%
• semi-skilled manual workers = 15%
• intermediate sales and service personnel = 12%

Top 3 occupational groups for former claimants:^{Footnote 27}

• other manual workers = 21%
• intermediate sales and service personnel; semi-skilled manual workers; other sales and service personnel = 12% each
• skilled crafts and trades workers; clerical personnel = 8 % each

Top 3 industries for active claimants:^{Footnote 28}

• retail trade; and manufacturing = 13% each
• accommodation and food services = 10%
• agriculture, forestry, fishing and hunting = 9%

Top 3 industries for former claimants:

• retail trade; and manufacturing = 13% each
• accommodation and food services = 11%
• construction = 10%

As presented in Table 3, in the year before program participation, former claimants have lower levels of employment and earnings than active claimants. Former claimants also have a higher dependence on SA.

Incremental impacts for active and former EI claimants ^{Footnote 29}

The incremental impact results presented below are generally consistent with those found as part of the second LMDA evaluation cycle.

Incidence of employment

Figure 2 presents the incremental impacts on the incidence of employment^{Footnote 30} for active and former claimants by type of program. The estimates can be interpreted as a change in the probability of being employed following participation.

Active and former EI claimant participants in Training PEI, EAS and Employ PEI increase their incidence of employment relative to similar non-participants.

EAS are relatively modest activities such as counselling, job search assistance and case management, which focus on quicker returns to work for participants. EAS supports that are not provided with longer interventions, are not expected to increase participants' skills or influence their employment levels to a large extent.

Employment earnings

Figure 3 presents the average annual increase in employment earnings for active and former EI claimant participants over the 4 years post-participation. Active EI claimants in Training PEI and Employ PEI increase their employment earnings compared to similar non-participants.

• * The impact is not statistically significant over the entire post-participation period.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Use of EI benefits

As shown in Figure 4, active EI claimant participants in Training PEI, Employ PEI and EAS-only reduce their use of EI benefits in the post-program period compared to similar non-participants.

Use of SA benefits

As shown in Figure 5, former EI claimant participants in Training PEI decrease their use of SA benefits in the post-program period.

Dependence on income support

As shown in Figure 6, active EI claimant participants in Training PEI and EAS reduce their dependence on government income supports compared to similar non-participants.

Active claimant participants in Employ PEI reduce their dependence on government income support in the first- and second-year post-participation.

Incremental impacts by subgroups of participants

Female participants

Nearly 2,750 EI active and former claimant participants in LMDA programs and services, between 2010 and 2012 in Prince Edward Island are female, representing about 50% of participants.

The profile of female participants is presented in Table 4 by age, sociodemographic group, and marital status. Information about their educational attainment, occupation and industry is based on the latest job they held prior to applying for EI benefits. Information about sociodemographic groups is self-reported.

• Note: Values may not equal 100% due to rounding or missing information.

Top 3 occupational groups for active claimants:^{Footnote 31}

• intermediate sales and service personnel = 19%
• clerical personnel = 14%
• other sales and service personnel; and other manual workers = 12% each

Top 3 occupational groups for former claimants:

• intermediate sales and service personnel = 21%
• other sales and service personnel = 16%
• clerical personnel = 12%

Top 3 industries for active claimants:

• retail trade = 17%
• accommodation and food services = 14%
• manufacturing = 10%

Top 3 industries for former claimants:

• retail trade = 16%
• accommodation and food services = 15%
• healthcare and social assistance = 11%

Main findings

Female active EI claimant participants in Training PEI and EAS-only improve their labour market attachment through increases in their incidence of employment. Female active participants in Training PEI also increase their employment earnings and decrease their dependence on income supports (that is, the combined use of EI and SA benefits). Active EI claimant participants in EAS-only decreased their use of EI benefits.

Table 5 presents the detailed incremental impacts. For example, the results reveal that relative to the comparison groups, female active claimant participants in Training PEI have higher annual earnings (+$2,189 per year) and increase their incidence of employment (+4.9 percentage points). They also lower their reliance on government income supports (-3.5 percentage points), due to lesser use of EI benefits (-$557 per year) and SA benefits (-$117 per year). Female active claimant participants in EAS-only increase their incidence of employment (+2.4 percentage points) and decrease their use of EI benefits (-$474).

• Statistical significance level *** 1%; ** 5%; * 10%, other values are not statistically significant.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Male participants

Nearly 2,720 EI active and former claimant participants in LMDA programs and services between 2010 and 2012 in Prince Edward Island are male, representing nearly 50% of participants.

The profile of male participants is presented in Table 6 by age, sociodemographic group, and marital status. Information about their educational attainment, occupation and industry is based on the latest job they held prior to applying for EI benefits. Information about sociodemographic groups is self-reported.

• Note: Values may not equal 100% due to rounding or missing information.

Top 3 occupational groups for active claimants:

• other manual workers = 27%
• semi-skilled manual workers = 24%
• skilled crafts and trades = 13%

Top 3 occupational groups for former claimants:

• other manual workers = 30%
• semi-skilled manual workers = 18%
• skilled crafts and trades = 13%

Top 3 industries for active claimants:

• manufacturing = 16%
• construction = 14%
• agriculture, forestry, fishing and hunting = 12%

Top 3 industries for former claimants:^{Footnote 32}

• construction = 17%
• manufacturing = 15%
• retail trade; and agriculture, forestry, fishing and hunting = 10% each

Main findings

Male active EI claimant participants in Training PEI improve their labour market attachment through increases in their incidence of employment and employment earnings. They also decrease their dependence on government income support, mainly through their lesser use of EI benefits. Active EI claimant participants in EAS-only decreased their use of EI benefits and increased their use of SA benefits.

Table 7 presents the detailed incremental impacts. For example, the results reveal that relative to the comparison groups, male active claimant participants in Training PEI have higher annual earnings (+$2,846 per year) and incidence of employment (+2.4 percentage points). They also lower their dependence on government income supports (-3.4 percentage points), mostly by decreasing their use of EI benefits (-$831). Male active claimant participants in EAS-only decrease their use of EI benefits (-$702 annually) and increase their use of SA benefits (+$197 annually).

• Statistical significance level *** 1%; ** 5%; * 10%, other values are not statistically significant.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Youth participants

Nearly 2,350 EI active and former claimant participants, between 2010 and 2012, were 30 years of age or younger when they began their program, representing nearly 43% of participants.

The profile of youth participants is presented in Table 8 by gender, sociodemographic group, and marital status. Information about their educational attainment, occupation and industry are based on the latest job they held prior to applying for EI benefits. Information about sociodemographic groups is self-reported.

• Note: Values may not equal 100% due to rounding or missing information.
• * < 10 = less than 10 participants.

Top 3 occupational groups for active claimants:

• other manual workers = 20%
• semi-skilled manual workers = 14%
• intermediate sales and service personnel = 13%

Top 3 occupational groups for former claimants:

• other manual workers = 22%
• intermediate sales and service personnel = 14%
• other sales and service personnel = 12%

Top 3 industries for active claimants:

• retail trade = 13%
• accommodation and food services = 11%
• manufacturing = 10%

Top 3 industries for former claimants:

• retail trade = 16%
• accommodation and food services; and manufacturing = 12% each
• construction = 10%

Main findings

Youth active EI claimant participants in Training PEI improve their labour market attachment through increases in their incidence of employment and employment earnings. They also decrease their dependence on government income support mostly by decreasing their use of EI benefits. Active claimant participants in EAS-only decrease their use of EI benefits.

Table 9 presents the detailed incremental impacts. For example, the results reveal that relative to the comparison groups, active claimant participants in Training PEI have higher annual earnings (+$2,601 per year) and incidence of employment (+4.2 percentage points). They also lessen their dependence on government income supports (-2.9 percentage points), mostly due to their lower use of EI benefits (-$455 per year). Active claimant participants in EAS-only decrease their use of EI benefits (-$692 annually).

• Statistical significance level *** 1%; ** 5%; * 10%, other values are not statistically significant.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Cost-benefit analysis

This analysis is based on the EBSM medium-term incremental impacts previously described in this report. Costs and benefits are examined over the participation period of 1 or 2 years and 5 or 10 years after the end of participation.^{Footnote 33}

The cost-benefit analysis addresses the following questions:

1. Are the benefits from EBSMs exceeding the costs within 5 years (for Employ PEI and EAS), 10 years (for Training PEI) after the end of participation?
2. How much is the benefit for the government and society if the government spends $1 in EI part II funding?
3. How many years does it take the benefits to recover the costs?

The following results are presented from the social perspective, that is, the government and individual combined. This allows for a sound assessment of program effectiveness in achieving its objectives of helping unemployed individuals to obtain and maintain employment and to generate EI savings.

Table 10 presents the cost-benefit results for active and former EI claimant participants.

The information below provides examples of the net present value, the benefit-cost ratio, the payback period, the social rate of return and savings to health care costs.

Training PEI^{Footnote 34}

During the 2010 to 2012 period, Training PEI represents almost 54% of EBSM expenditures under the LMDAs in Prince Edward Island. The average duration of a Training PEI Action Plan Equivalent is 57 weeks for active claimants and 61 weeks for former claimants. As shown in Table 10, over the 10 years post-program period the benefit for active claimants is $11,354 higher than the costs, yielding a social return of 91% on investment. This means that if the government spends $1 on Training PEI for active EI claimants, it generates $1.91 of benefit for society. It takes 6.8 years for the benefits to recover the costs of programming. Overall, there are savings to health care costs of $316 per participant. The benefit for former claimants is $3,506 higher than the costs, yielding a social return of 27% on investment. This means that if the government spends $1 on Training PEI for active EI claimants, it generates $1.27 of benefit for society. It takes 8.7 years for the benefits to recover the costs of programming. Overall, there are savings to health care costs of $259 per participant.

Employ PEI

During the 2010 to 2012 period, Employ PEI represents 9% of total EBSM expenditures across Prince Edward Island. The average duration of an Employ PEI Action Plan Equivalent is 33 weeks for active claimants and 29 weeks for former claimants. As shown in Table 10, over the 5 years post-program period the benefit for active claimants is $8,268 higher than the costs, yielding a social return of 84% on investment. This means that if the government spends $1 on Employ PEI for active EI claimants, it generates $1.84 of benefit for society. It takes 2.6 years for the benefits to recover the costs of programming. Overall, there are savings to health care costs of $283 per participant. The benefit for former claimants is $6,908 higher than the costs, yielding a social return of 73% on investment. This means that if the government spends $1 on Employ PEI for active EI claimants, it generates $1.73 of benefit for society. It takes 1.3 years for the benefits to recover the costs of programming. Overall, there are savings to health care costs of $200 per participant.

Employment Assistance Services^{Footnote 35}

EAS includes a variety of services such as computer access for job search services, group sessions to prepare for an interview, career counselling, and action plan development. The administrative data, however, do not allow to identify what proportion of EAS interventions belong to each category or the intensity of services offered to participants.

While EAS is often provided with other EBSMs, this analysis examined only participants who received 1 or more EAS services without participating in other EBSMs. EAS represents about 17% of total EBSM expenditures in Prince Edward Island between 2010 and 2012. The average length of an EAS-only Action Plan Equivalent is 16 weeks compared to between 28 to 57 weeks for active EI claimant participants in other EBSMs.

As shown in Table 10, over the 5 years post-program period the benefits for active claimants in EAS is $779 higher than the costs, yielding a social return on investment of 59%. This means that if the government spends $1 on EAS for active EI claimants, it generates $1.59 of benefit for society. It takes 3.7 years after participation for the benefits to recover the costs. Overall, there are savings to health care costs of $26 per participant.

Outcomes for active and former EI claimants^{Footnote 36}

Incremental impacts for Work Experience PEI are not produced due to small sample sizes or the models used to estimate the incremental impacts not balancing. Having balanced models is an essential condition to ensure that participants and non-participants are similar. However, outcomes can still be examined for participants to describe the average changes that occur from before to after program participation.

The labour market outcomes are based on individuals who began their participation during the 2010 to 2012 period. Statistics focus on 5 years before and 5 years after the program start year.

Work Experience PEI participant outcomes

Approximately 324 active and former EI claimant participants, between 2010 and 2012, participated in the Work Experience PEI program.

Active claimants

As shown in Figure 7, Work Experience PEI participants increase their average earnings from $14,537 in the fifth year pre-program to $19,624 in the fourth year after the end of program participation.

Ninety-one percent of active claimant participants were employed in the program start year. The proportion of employed participants remains at 91% on average during the post-program period. The proportion of participants receiving EI benefits decreases from 99% in the program start year to 49% in the fourth year after the end of program participation. Participants decrease their dependence on income support from 63% in the program start year to 22% in the fourth year after the end of program participation.

Former claimants

As shown in Figure 8, former EI claimant participants in Work Experience PEI increase their average earnings from $13,981 in the fifth year pre-program to $18,865 in the fourth year after the end of program participation.

The proportion of employed participants increases from 77% in the program start year to an average of 83% over the post-program period. The proportion of participants receiving EI benefits decreases from 68% in the program start year to 44% in the fourth year after the end of program participation. Participants decreased their dependence on income support from 46% in the program start year to 21% in the fourth year after the end of program participation.

Supplemental studies

Self Employ PEI^{Footnote 37}

The self-employment program aims to assist participants in creating employment for themselves by providing them with a range of services including: 

• providing feedback on participants' business plan
• counselling, coaching and mentoring
• entrepreneurial training and workshops

Eligible individuals must be unemployed and looking for full-time employment. As well, eligible participants must:

• be legally authorized to work in Canada (Canadian citizen or permanent resident)
• have either a current EI claim or a previous claim for which the benefit period ended within the previous 60 months, or be an unemployed person who meets the EI premiums paid eligibility criteria
• not currently operate their own business

Prince Edward Island has the flexibility to design and deliver the program to meet its labour market needs. In fall 2018, the Self Employ PEI program was delivered internally in Prince Edward Island.

Program officials reported that the amount allocated to Self Employ PEI is set by the province in an annual plan. However, there is flexibility to accommodate meeting labour market needs based on demand for the program.

The application process is structured and aims to ensure that participants are suited for self-employment, have a viable business plan and have the financial resources to launch a business.

Participants' employment outcomes

The following is a summary of labour market outcomes and satisfaction rates from a survey completed in the winter of 2020. The sample of Self Employ PEI respondents participated in the program in 2015 to 2016 and 2016 to 2017.^{Footnote 38} A total of 106 individuals responded, resulting in a 48% response rate.^{Footnote 39}

Type of businesses created and survival rate

Participants launched businesses in a variety of industries, the most common of which was other services.

Nearly 70% of survey respondents launched a self-employment business during program participation. Of those who launched a business:

• 66% were still operating their business 2 to 4 years after participating in the program
• a small number sold their business, but it was still operational
• 30% were unable to maintain the operation of the business

The business survival rate is higher compared to a 2018 Statistics Canada study^{Footnote 40} that found that less than half of unincorporated self-employed individuals continued operations for more than 2 years.

Factors influencing the success or failure of self-employment businesses

Participants who started a business and were still in operation at the time of the survey attributed their business success to:

• having a solid business plan
• the help and support provided under the program
• ongoing discussion with the program officer during the application process and during program participation
• being motivated, having a strong entrepreneurial spirit, a strong marketing plan, good management of cash flow, mentorship and having sufficient financial resources

In Prince Edward Island, key informants identified the following primary reasons that participants fail at starting and maintaining their businesses:

• inaccurate market research and financial projections may create challenges for some participants
• some participants may have unrealistic expectations about what it will take in terms of time management to operate a self-employment business 
• participants may have unrealistic expectations of sales
• insufficient resource
• poor marketing and poor customer service

Earning outcomes and reliance on income support

Survey respondents were not comfortable answering questions that related to their earnings. This situation made it difficult to compare the pre- and post-earnings of Self Employ PEI participants.

As a complement to the earning questions, survey respondents assessed their financial well-being. When considering their entire financial situation:

• 73% of respondents said that they are financially about the same or better off after the program
• 73% of respondents said that their household net worth is about the same or higher after the program

In line with survey findings, service providers stated that immediate increases in earnings are not necessarily an expected outcome of the program. 

Regarding reliance on government income support, participants reduced their use of EI and SA following program participation.

Satisfaction with services received and current employment

82% of respondents who started a self-employment business report that they are more satisfied with their job situation after program participation.

The survey examined the contribution of the program to the success of self-employment businesses. Over 74% of survey respondents who launched a self-employment business rate the following services and training as very or somewhat important to the business launch, operation and success:

• one-on-one coaching
• financial assistance for living allowance during participation
• assistance with developing a business plan
• financial management training
• financial assistance for costs associated with a business start-up

Challenges and lessons learned related to program design and delivery

Key informants identified managing participants' expectations about the time and effort required to participate in the program as the most challenging aspect of program delivery.

Best practices related to program design and delivery included:

• ensuring consistency of the application process, case management and partnership engagement
• good communication between program officials and third-party service providers' staff to identify issues and challenges 
• completing an initial applicant assessment to ensure program suitability
• providing forums for participant support
• supporting participants' access to funding

Key considerations for self-employment program and policy development

Key considerations for program and policy development include:

Self Employ PEI can benefit from an updated objective specifying that it is dedicated to eligible participants who have a viable business plan, the financial or in-kind resources to launch a business, and the required level of dedication.

The data collection process should include only participants who have been deemed suitable for self-employment and accepted into the program. This will require excluding candidates who attended information sessions alone or those deemed not suited for self-employment. The latter participants can be reported under EAS.

Indicators of program success can include: increase in employment and/or self-employment levels; medium-term increase in earnings; business survival rate similar to the local economy and/or the sector; and acquisition of transferable skills.

Work Experience PEI^{Footnote 41}

Work Experience PEI is a short-term employment program that provides job seekers with an opportunity to work on a project with a local organization. The on-the-job work experience will help the job seeker develop new skills needed for employment in the workforce.

Program delivery

Prince Edward Island implements the Work Experience PEI program through the provision of funds to non-profit organizations that implement a project while providing work experience to participants.

The design and delivery of Work Experience PEI allows the province to address a variety of barriers to employment experienced by their citizens (such as lack of work experience).

Program managers reported that the amount allocated to Work Experience PEI is influenced by government priorities, previous funding levels and feedback from stakeholder consultations.

In addition to gaining work experience, key informants expected participants to develop transferable skills and to enhance their career development and job search abilities. Sponsors can benefit from Work Experience PEI through increased capacity by having access to labour, implementing their projects, and developing partnerships with other organisations. At the community level, Work Experience PEI projects are expected to support the local economy.

Challenges and lessons learned

Key informants identify challenges related to:

• the recruitment of participants
• addressing participants' barriers to employment
• factors that affect the administration of Work Experience PEI

Key informants identify the following elements as contributing factors to participants' success:

• the removal of material related costs from eligible project costs has ensured that sponsors focus their attention on participants' needs
• the selection of participants by sponsors is beneficial as it ensures that participants are a good fit for the organization and project
• implementation is effective when SkillsPEI representatives, service providers, and sponsors communicate and collaborate to support participants to overcome their barriers to employment
• hiring of a project coordinator to supervise participants, case manage and communicate with participants supports effective monitoring
• participants in need of work experience and networking opportunities benefit the most from the Work Experience PEI program
• organizations who possess a strong network and focus their attention on participant needs are most likely to succeed as sponsors
• projects found to be best suited under Work Experience PEI include event coordination, arts and culture

Labour Market Partnerships^{Footnote 43}

The LMP program in Prince Edward Island aims to encourage, support and facilitate labour force adjustments and human resource planning activities, which are in the public interest. It includes a wide range of funded activities, such as:

• human resource planning, including:
  • development and dissemination of labour market information
  • development of human resources best practices and tools
  • development of human resources strategy to address current workforce demands and support future industry growth
• labour market adjustment including:
  • raising awareness about available or future in-demand careers and jobs
  • providing human resources advice to small business owners
• skills development including:
  • development and delivery of human resources workshops
  • piloting an alternative model for training delivery

With $1.3 million in 2019 to 2020, the LMP program represents nearly 4% of total expenditure under the Canada-Prince Edward Island LMDA.

Funded organizations

Funded organizations include:

• non-profit organizations
• industry associations
• sector councils
• businesses

Targeted labour market issues

LMP projects targeted current and/or forecasted skills and/or labour shortages. These projects also targeted specific unemployed populations (for example, women, youth, and newcomers).

Generally, funded projects target labour market issues associated with:

• lack of capacity for human resource planning resulting in attraction and retention challenges for employers, communities/regions and sectors
• aging workforce
• projected or actual growth of industry/business
• barriers to employment experienced by a target population
• skills development challenges and training gaps to fill in-demand jobs

All projects reviewed aligned with the LMP program objectives and eligible activities.

Partnerships

The Department of Education and Lifelong^{Footnote 44} Learning and key informants confirmed that program officials conducted activities to support the formation and maintenance of partnerships (with sector and community stakeholders) as a part of the program design and delivery. The Department and key informants explained that partners' expertise, network and financial contribution are all essential to project implementation and success.

The document review of 5 LMP projects confirmed that:

• partnerships were established to support the delivery of all projects reviewed
• partners made a financial or in-kind contribution. The most common forms of in-kind contribution were expertise, as well as staff time to support project delivery
•  project activities delivered with the support of partners included labour market information development and dissemination, career awareness activities, workshop delivery, promotion and implementation of human resource industry specific tool, advertising content development

Performance measurement

As per the Labour Market Transfer Agreements' Performance Measurement Plan, Prince Edward Island collects data on the number of employers that participate in LMP by economic sector, business size and type.

Challenges and lessons learned

The Department of Education and Lifelong Learning identified challenges related to the timing of projects due to the length of time required to obtain approval for projects from various levels of government.

Actions of program officials and project characteristics that are conducive to the success of the program included:

• annual consultations with key stakeholders and ongoing formal and informal outreach activities to ensure that the program is addressing labour market challenges
• developing partnerships and fostering relationships to achieving results that help meet the labour market needs of employers and industry organizations across the province

Key considerations for Labour Market Partnership program and policy development

Considering that the current performance indicators do not reflect the diversity of activities funded under LMP, it is important for ESDC and P/Ts to discuss current LMP funded activities in order to make recommendations on how to best report on results.

Considering the intermittent or limited use of the program in some P/Ts, it is essential to share lessons learned about successful LMP projects. Particularly, for projects targeted to employers (such as workplace or employer-sponsored training), and those assisting communities and economic sectors dealing with labour market adjustment issues (contraction or expansion).

Research and Innovation ^{Footnote 45}

Research and Innovation is a program that supports research and innovative projects to identify better ways of helping persons prepare for, return to or keep employment and be productive participants in the labour force.

Prince Edward Island used the Research and Innovation program intermittently between 2014 and 2017,^{Footnote 46} with Research and Innovation funding varying from less than 1% ($124,000) to 1% ($245,000) of the province's annual LMDA expenditures.

Funded organizations

Funded organizations can include:

• businesses
• federal/Crown corporations
• non-profit organizations
• public health and educational institutions
• provincial government departments in exceptional cases only

Funded Research and Innovation activity

From 2014 to 2016, Research and Innovation funded a project to implement a Student Graduation and Transition Planner to assist students in PEI with high school graduation pathways, post-secondary transitions and entry into the labour force.

Performance measurement

Prince Edward Island completed 2 evaluations on the only project funded under Research and Innovation. One for each year of project funding. Evaluations included:

• a review of project outcomes
• attendance and numbers of participants
• participant feedback regarding services

Project holders are also responsible for the dissemination of all project results and information.

Training PEI Apprentices

The objective of the program is to help apprentices become skilled tradespeople and to increase their labour market attachment. Program participants have generally chosen a career and are already attached to the labour market. The apprenticeship process involves on-the-job learning and technical training in a classroom setting.

Apprentices who have worked enough hours to qualify for EI can apply to receive EI Part I benefits while on training. The program provides financial assistance to EI eligible apprentices to help them offset the costs they incur while they attend technical training. The level of funding is based on the needs of apprentices, the location of the training, and any fees paid by the apprentices.^{Footnote 47}

The profile of participants is presented in Table 11 by gender, age, sociodemographic group, and marital status. Information about their educational attainment, occupation and industry is based on the latest job they held prior to applying for EI benefits. Information about sociodemographic groups is self-reported.

• Note: Values may not equal 100% due to rounding or missing information.

Top occupational groups for active claimants:

• skilled crafts and trades workers = 65%
• other manual workers = 23%

Top occupational groups for former claimants:

• skilled crafts and trades workers = 73%
• other manual workers = 10%

Top 3 industries for active claimants:

• construction = 65 %
• manufacturing = 9%
• retail trade = less than 10 participants

Top 3 industries for former claimants:

• construction = 43 %
• manufacturing = 16%
• retail trade = 13%

Labour market outcomes

The labour market outcomes are based on individuals who began their participation during the 2010 to 2012 period. Statistics focus on 5 years before program participation and 5 years after the program start year.

Active claimants

As shown in Figure 9, program participants increase their average earnings from $13,672 in the fifth year pre-program to $44,951 in the fifth year after the program start year.

The proportion of employed participants declines from 100% in the program start year to an average of 97% over the post-program period. The proportion of participants receiving EI benefits decreases from 99% in the program start year to 41% in the fifth year after the program start year. Participants decrease their dependence on income support from 29% in the program start year to 9% in the fifth year after participation.

Former claimants

As shown in Figure 10, program participants increased their average earnings from $17,372 in the fifth year pre-program to $53,307 in the fifth year after the program start year.

The proportion of employed participants declined from 100% in the program start year to an average of 95% over the post-program period. The proportion of participants receiving EI benefits decreased from 80% in the program start year to 22% in the fifth year after the program start year. Participants decreased their dependence on income support from 13% in the program start year to 3% in the fifth year after participation.

Conclusions and recommendations

The Canada-Prince Edward Island LMDA is the largest annual investment in active labour market programs and services in Prince Edward Island. Based on the findings presented in this report, the EBSMs are meeting the objective of assisting individuals to obtain or keep employment through various active employment programs, including training or employment assistance services.

Summary of findings

Overall, incremental impacts demonstrate that active and former EI claimant participants in most EBSMs improve their labour market attachment and reduce their use of government income supports. A subgroup analyses shows that Training PEI improves the labour market attachment and reduces the dependence on income support of all subgroups of participants. Active EI claimants in Training PEI increased their incidence of employment and increased employment earnings. Active EI claimants in Employ PEI had an average annual increase in earnings of $2,722 compared to non-participants over the 4 years post participation. Former claimants in Employ PEI had an annual average change in probability of being employed of 4.8 percentage points compared to non-participants. As well, the social benefits of participating in EBSMs exceeds the costs of investments for all interventions over time.

A series of supplemental studies address information gaps previously identified in LMDA evaluations for Self Employ PEI, Work Experience PEI, LMP, Research and Innovation, and Training PEI Apprentices. Each study identified lessons learned, best practices and challenges, and issued, when relevant, considerations for policy design and development. Overall, it was found that:

Self Employ PEI helps carefully selected participants to create employment for themselves by providing them with a range of services.

The objective of Work Experience PEI is to provide participants with the opportunity to gain work experience or training that improves their employment prospects. It is difficult to quantify all the positive impacts of Work Experience PEI for individuals, employers and communities.

Prince Edward Island uses the LMP program to encourage, support and facilitate labour force adjustments and human resource planning activities, which are in the public interest. The current performance indicators do not reflect the diversity of funded activities. Therefore, it is important for ESDC and Prince Edward Island to discuss current funded activities to make recommendations on how to best report on results.

Activities funded under Research and Innovation aim to fund previously untried solutions to specific labour market issues or problems and create strategic investment mechanisms to fund experiments that could inform the development of policies, programs or industry practices. There is an added value in documenting and sharing of lessons learned and best practices from demonstration projects.

After participating in Training PEI, apprentices increase their employment earnings and decrease their dependence on government income supports.

Recommendations

Since 2012, 15 qualitative and quantitative studies addressed issues and questions related to EBSM design, delivery and effectiveness. The quantitative studies successfully assessed the effectiveness and efficiency of EBSMs by producing incremental impacts and cost-benefit analysis. The qualitative studies identified specific challenges, lessons learned and best practices associated with the design and delivery of EBSMs. Each study included key considerations for program and policy development or recommendations.

The recently completed evaluation of the Workforce Development Agreements complements the LMDA qualitative studies. This evaluation was also supported by literature reviews and provided unique insights into challenges and lessons learned to assist persons with disabilities, immigrants and those further removed from the labour market.

Most results from this evaluation stem from the conduct of advance causal analysis whereby impacts found could be attributed to a specific EBSM. These analyses are predicated on having access to high quality administrative records, thereby confirming the importance of the capacity to leverage and integrate relevant administrative data.

From these main findings, 2 key recommendations emerge:

Recommendation #1: Prince Edward Island is encouraged to share and discuss lessons learned, best practices and challenges associated with the design and delivery of programs and services. Discussions are encouraged with ESDC, at the bilateral or multilateral levels, as well as with service delivery networks if necessary.

Recommendation #2: Prince Edward Island is encouraged to pursue efforts to maintain and strengthen data collection provisions in support of reporting, performance measurement and data-driven evaluations at the national and provincial levels.

References

Allin, S., Corscadden, L., Gapanenko, K., and Grignon, M. Lifetime Distributional Effects of Publicly Financed Health Care in Canada. CIHI, 2013.

Douwere G. and Huju Liu. The Entry into and Exit out of Self-employment and Business Ownership in Canada. Statistics Canada, 2018.

Employment and Social Development Canada. Cost-Benefit Analysis: Incorporating Public Health Care Costs Savings in the Context of the Labour Market Programs Evaluation. Internal document, 2022. Available upon request.

Employment and Social Development Canada. Cycle II of the Evaluation of the Labour Market Development Agreements: Cost-Benefit Analysis of Employment Benefits and Support Measures. Internal document, 2015. Available upon request.

Employment and Social Development Canada. Design and delivery of the Work Experience PEI program in Prince Edward Island. Internal report, 2019.

Employment and Social Development Canada. Employment Insurance Monitoring and Assessment Reports. 2011 to 2022.

Employment and Social Development Canada. Evaluation of the Labour Market Development Agreements, Design and delivery of Self Employ PEI. Internal report, 2021.

Employment and Social Development Canada. Horizontal evaluation of the Labour Market Development Agreements, Design and delivery of the Labour Market Partnerships program in Prince Edward Island. Internal document, 2021.

Employment and Social Development Canada. Labour Market Development Agreements Process for Determination of Similarity. Internal document, 2012.

Employment and Social Development Canada. Technical Report on the Analysis of Employment Benefits and Support Measures (EBSMs) Profile, Outcomes and Medium-Term Incremental Impacts from 2010 to 2017. Internal report, 2021. Available upon request.

Employment and Social Development Canada. Third Cycle for the Horizontal Evaluation of the Labour Market Development Agreements: Quantitative Methodology Report. Internal document, 2019. Available upon request.

Appendix A. List of 8 studies included in this synthesis report

Overview of studies included in this synthesis report.

Study 1: Examination of medium-term outcomes from 2010 to 2017
Evidence generated: profile of active and former EI claimants; outcomes by claimant type and by subgroup
Methods: before and after results of program participation
Reference period: 2010 to 2012 participants
Observation period: up to 12 years (5 years before participation, 1 to 2 years of participation, and up to 5 years after participation)

Study 2: Estimation of medium-term incremental impacts from 2010 to 2017
Evidence generated: incremental impacts for active and former EI claimants; incremental impacts by subgroup; profile and socio-demographic characteristics of participants
Methods: non-experimental method using propensity score matching in combination with difference-in-differences; statistical profiling
Reference period: 2010 to 2012 participants
Observation period: up to 7 years (1 to 2 years in program, and up to 5 years after participation)

Study 3: Cost-benefit Analysis of Employment Benefits and Support Measures in Prince Edward Island
Evidence generated: cost-benefit analysis
Methods: non-experimental method using propensity score matching in combination with difference-in-differences; cost-benefit analysis
Reference period: 2010 to 2012 participants
Observation period: 5 years post-program for EAS; 10 years post-program for Training PEI

Study 4: Cost-Benefit Analysis: Incorporating Public Health Care Costs Savings in the Context of the Labour Market Programs Evaluation in Prince Edward Island
Evidence generated: cost-benefit analysis
Methods: estimation of adjusted annualized healthcare costs
Reference period: 2010 to 2012 participants
Observation period: 5 years post-program for EAS; 10 years post-program for Training PEI

Study 5: Design and delivery of the Work Experience PEI program
Evidence generated: program design and delivery; challenges and lessons learned
Methods: summary of participant outcomes (from cycle II); statistical analysis; document review; semi-structured telephone interviews with 8 key informants
Reference period: 2015 to 2017 participants
Observation period: 2015 to 2019

Study 6: Design and delivery of Self Employ PEI
Evidence generated: program design, delivery and success; define outcomes attributed to the program; fill in knowledge gaps; challenges and lessons learned
Methods: document review; 1 semi-structured telephone interview with 4 key informants; statistical analysis of administrative data; survey of Self Employ PEI participants
Reference period: 2015 to 2017 participants
Observation period: 2015 to 2020

Study 7: Design and delivery of the Labour Market Partnerships program in Prince Edward Island
Evidence generated: program design and delivery; challenges and lessons learned
Methods: document review; questionnaire completed by the province; 6 semi-structured interviews with 6 key informants
Reference period: 2018 to 2020
Observation period: design and delivery at the time of the data collection

Study 8: Design and delivery of the Research and Innovation support measure in Prince Edward Island
Evidence generated: program design and delivery; challenges and lessons learned
Methods: document review; questionnaire completed by key informants
Reference period: 2017 to 2020
Observation period: design and delivery at the time of the data collection