Departmental Evaluation Plan 2022 to 2027

List of acronyms

CPP
Canada Pension Plan
EI
Employment Insurance
ESDC
Employment and Social Development Canada
FAA
Financial Administration Act
GBA Plus
Gender-Based Analysis Plus
OAS
Old Age Security
PMEC
Performance Measurement and Evaluation Committee
TBS
Treasury Board of Canada Secretariat

Alternate formats

Large print, braille, MP3 (audio), e-text and DAISY formats are available on demand by ordering online or calling 1 800 O-Canada (1-800-622-6232). If you use a teletypewriter (TTY), call 1-800-926-9105.

Deputy Head Departmental Evaluation Plan confirmation note

I approve the Departmental Evaluation Plan of Employment and Social Development Canada for 2022 to 2027. I submit it to the Treasury Board of Canada Secretariat as required by the Policy on Results.

I confirm that this 5-year rolling Departmental Evaluation Plan:

  • plans for evaluation of all ongoing programs of grants and contributions with 5-year average actual expenditures of $5 million or greater per year at least once every 5 years, in fulfillment of the requirements of subsection 42.1 of the Financial Administration Act
  • meets the requirements of the Mandatory Procedures for Evaluation, and
  • supports the requirements of the expenditure management system including, as applicable, Memoranda to Cabinet, Treasury Board Submissions, and resource alignment reviews

I will ensure that this Plan is updated annually. I will provide information about its implementation to the Treasury Board of Canada Secretariat, as required.

Jean-François Tremblay
Deputy Minister of Employment and Social Development

Executive summary

As per the Treasury Board of Canada Secretariat Policy on Results, Employment and Social Development Canada (ESDC) must develop and publish an annual 5-year departmental evaluation plan. Informed by internal consultations and building on last year’s accomplishments, this plan targets the evaluation coverage of ESDC’s program spending for 2022 to 2027, with a focus on departmental risks, needs and priorities.

In 2021 to 2022, the Evaluation Directorate at ESDC continued to play an important role in the results agenda by fulfilling its mandate to evaluate the relevance and performance of departmental programs, policies, initiatives and services. The Directorate influenced policy and program design by conducting rigorous analysis, generating high quality evidence, disseminating knowledge, engaging with partners and providing trusted advice. It also continued to develop innovative approaches, including those that support the Government of Canada’s commitment to Gender-based Analysis Plus in both quantitative and qualitative evaluation design.

Key accomplishments in 2021 to 2022

Supporting evidence-based decisions

  • Completed 10 evaluation reports and 40 technical reports to support program officials and the Employment Insurance Commissioners
  • Collaborated with provinces and territories on a bilateral and multilateral basis on labour market program evaluations to produce an additional 49 technical reports
  • Incorporated innovative approaches and state-of-the art techniques to address important policy-relevant questions
  • Provided sound advice as well as reliable and timely evaluation evidence to enable sound decision-making for ESDC’s labour market programs

Disseminating and engaging

  • Delivered 15 virtual presentations at 10 different national and international conferences or events and published 2 technical reports
  • Collaborated with the Organization for Economic Co-operation and Development on a project that will highlight how certain countries, like Canada, are leveraging administrative data to support impact evaluations
  • Networked with the academic community to explore new avenues of knowledge dissemination and collaboration

Acting as a trusted advisor

  • Advised program officials and Head of Performance Measurement on elements associated with the assessment and revision of 15 Performance Information Profiles
  • Provided advice to improve the use of the Gender-based Analysis Plus lens
  • Supported the Employment Insurance Commissioners for Employers and Workers in examining labour market issues and provided assistance with the work of the Monitoring Report and Advisory Committee

Advancing innovative approaches and gender-based analysis plus

  • Pursued innovative practices to better estimate the impact of programs on specific sub-groups and disseminate results to inform decisions

Challenges and opportunities for the evaluation function

In 2021 to 2022, the Evaluation function continued to operate in a context shaped by the COVID-19 pandemic. Among key challenges was the integration of new post-pandemic recovery measures as part of ongoing evaluations, as well as the availability and quality of data for certain programs.

On the other hand, the context of the pandemic also represented an opportunity for the function to carefully review the way it conducts evaluations to optimize their usefulness and value. Furthermore, ongoing departmental efforts in terms of collaboration with Statistics Canada to integrate administrative and survey data will eventually support the conduct of more valuable and relevant policy analysis, research and evaluation activities from a Gender-based Analysis Plus perspective.

Going forward

The Directorate plans to complete 15 evaluations in 2022 to 2023 and 9 evaluations in 2023 to 2024. In the years ahead, the Evaluation Directorate will continue to:

  • collaborate with policy and program officials to provide timely evaluation evidence to inform decision makers
  • incorporate innovative methods into its operations and the services it provides to the Department and other stakeholders, the measurement of the impacts of ESDC’s programs as well as for the dissemination of results
  • integrate and explore ways to enhance Gender-based Analysis Plus data analysis and collection methods when conducting quantitative and qualitative evaluation activities, to support the Department’s efforts to address systemic inequities and disparities
  • engage experts in the academic community on how best to conduct evaluations to ensure that ESDC’s evaluators adopt best practices from other jurisdictions, departments and disciplines
  • collaborate with federal counterparts to strengthen the evaluation function across government

Introduction

Purpose of the Plan

The purpose of this Plan is to assist the Deputy Head by ensuring the availability of factual, neutral, and timely information on the Department’s programs and services. This information supports evidence-based decision-making and helps keep Canadians informed on departmental results. The Plan also helps ensure transparency in the decision making process for the evaluation function, highlighting priorities and coverage decisions.

This 5-year rolling Plan, required under the Treasury Board Policy on Results, is informed by an annual planning exercise that identifies the timing of the individual evaluations based on departmental needs, risks, and priorities.

To that end, the Evaluation Directorate is responsible for developing and implementing the Departmental Evaluation Plan. This includes:

  • reviewing various sources of information to identify all mandatory evaluations and assess coverage of the programs
  • consulting with program officials, key internal committees and Treasury Board of Canada Secretariat
  • assessing risk when determining the timing and order of future evaluations, prioritizing mandatory evaluations and evaluations that are most likely to inform major policy and program decisions
  • validating prioritization and securing final approval from the Deputy Head
  • monitoring and adapting to the changing departmental context and environment

Annex 1 provides more information about the Evaluation Planning Process.

Departmental context

ESDC’s mandate is to build a stronger and more inclusive Canada, to support Canadians in:

  • helping them live productive and rewarding lives
  • improving their quality of life

To that end, the Department delivers a range of programs and services that affect Canadians throughout their lives and across the country. The Department’s portfolio includes: (1) ESDC, (2) the Labour Program, and (3) Service Canada:

  1. ESDC delivers a range of programs and services that assist Canadians. For instance, it provides seniors with basic income security, supports unemployed workers while helping them return to employment, helps students to finance their post-secondary education and to transition to the labour market, and assists parents who are raising young children. The Department also assists persons with disabilities and their families.
  2. The Labour Program fosters safe, healthy, fair and inclusive work environments and cooperative workplace relations in the federal jurisdiction.
  3. Service Canada serves as a single point of access for some of the Government’s largest and most well known programs and services. This includes Employment Insurance, Old Age Security, Canada Pension Plan, the Social Insurance Number, and the Passport Program.

ESDC expenditures on programs and services totalled $252.3 billion in fiscal year 2020 to 2021. More than 95% of the departmental spending directly benefitted Canadians through Employment Insurance, the Canada Pension Plan, Old Age Security, Canada Emergency Response Benefits and other statutory transfer payment programsFootnote 1.

In fiscal year 2021 to 2022, ESDC continued providing support to Canadians facing hardship as a result of the global COVID-19 outbreak. In particular, the Department delivered a range of temporary COVID-19 recovery benefits to Canadians (for example, the Canada Recovery Benefit, the Canada Recovery Sickness Benefit, the Canada Recovery Caregiving Benefit, and the Canadian Workers Lockdown Benefit). The Department also implemented measures to support early learning and childcare, youth and helped Canadians to gain the foundational skills needed to succeed in today’s economy.

Evaluation directorate mandate

The Evaluation Directorate is mandated to evaluate the relevance and performance of departmental programs, policies, initiatives and services. The Directorate influences policy and program design through rigorous analysis and evidence presented in evaluation reports, as well as via technical reports, syntheses or supplemental studies. In that context, the Directorate works with program officials and senior management on the evaluation process through an inclusive approach to governance (Annex 2).

The Evaluation Directorate supports program officials at all stages of the program and policy life cycle:

  • at the initial stage, officials use evaluations to help inform the development of new programs and policies
  • during implementation, officials use evaluation recommendations to guide adjustments or program renewal
  • as a program or policy matures or winds down, officials can use evaluations to inform lessons learned for future programs or policies

The Evaluation Directorate occasionally advises program officials that are developing responses to the recommendations found in final reports. Lastly, the Directorate monitors progress on the implementation of action plans. This approach promotes a system of continuous improvement throughout the program and policy cycle.

Accomplishments in 2021 to 2022

While evaluation activities continued to be shaped by the COVID-19 pandemic, the evaluation team successfully received Performance Measurement and Evaluation Committee (PMEC) approval for evaluation reports, offered different evaluation products and services to meet information needs in the Department and contributed to the improvement of programs and services.

Supporting evidence-based decisions

Evaluation Directorate influenced and supported the Department evidence-based decision making by:

  • completing 10 evaluation reports and 40 technical reports to support program officials and the Employment Insurance Monitoring and Assessment Report 2019 to 2020
  • collaborating with provinces and territories on a bilateral and multilateral basis on the evaluation of the Labour Market Development Agreements and the Workforce Development Agreements to produce an additional 49 technical reports
  • advising on 37 Treasury Board Submissions and 6 Memoranda to Cabinet to inform policies and programs
  • incorporating innovative approaches and state of the art techniques in impact evaluation and cost-benefit analysis methodologies to address policy relevant questions
  • providing reliable and timely evaluation evidence to support labour market policy and program decisions, such as evidence regarding employment outcomes of Workforce Development Agreement participants before and during the COVID-19 pandemic
  • collaborating with program authorities and other groups within the Department, including the Chief Data Office, to optimize the availability and collection of data that can be used to support evaluation
  • obtaining thorough peer reviews for all evaluation results and evaluation methodologies used by the Directorate Footnote 2

Disseminating and engaging

In 2021 to 2022, the Evaluation Directorate continued to disseminate knowledge and engage with partners inside and outside of the federal government. These activities included sharing knowledge through publications, presentations and engagement activities. To that end, the Directorate delivered 15 virtual presentations at 10 different national and international conferences and events, such as:

  • 7 presentations at the Canadian Economic Association conference, focusing on the methodology used and key results from several evaluations including: the incremental impacts for the Labour Market Development Agreements, an analysis of Employment Insurance benefits and job match quality, and the integration of Temporary Foreign Worker Program data into the Canadian Employer-Employee Dynamics Database
  • 4 presentations at the Canadian Evaluation Society conference, covering various topics on how to design the evaluation of a pilot, harness findings from qualitative methods to inform program effectiveness, leverage web analytics as a data collection strategy, and promote the utilization of evaluation results through various dissemination strategies

The Evaluation Directorate also:

  • published 2 technical reports related to labour market programs for Persons with Disabilities and Indigenous peoples
  • collaborated with other federal departments or provincial/territorial governments on 6 evaluations, including those led by ESDC or other federal departments
  • collaborated with the Organisation for Economic Co-operation and Development on a project that will highlight how certain countries, like Canada, are leveraging administrative data to support impact evaluations. The report is expected to be released in 2022
  • strengthened and broadened its network with the academic community to explore new avenues of knowledge dissemination and collaboration
  • engaged in Federal/Provincial/Territorial discussions on results of various studies and best practices to design and deliver programs (for example, Labour Market Partnerships, Research and Innovation)
  • collaborated with ESDC’s Chief Data Office to share more data insights across the federal government
  • worked horizontally with Treasury Board of Canada Secretariat and other federal Heads of Evaluation

Acting as a trusted advisor

Over the course of 2021 to 2022, the Evaluation Directorate continued to provide trusted advice to program officials and senior management. The Directorate did this by:

  • providing advice on elements associated with the assessment and revision of 15 Performance Information Profiles
  • sharing technical reports and evaluations with policy developers across the Department to inform policy analysis and program improvements
  • providing advice and guidance to improve the use of a GBA Plus data collection and analysis lens to support the assessment of inequalities and determine whether diverse groups are well-served by our programs
  • participating in consultation activities on program design and delivery resulting from evaluation studies, in response to provincial and territorial requests
  • collaborating in the development of the draft Policy Framework for service to disadvantaged and under-served populations

The Evaluation Directorate also continued to support the Employment Insurance Commissioners for Employers and Workers in examining labour market issues and supporting the work of the Monitoring Report and Advisory Committee. In particular, the Directorate collaborated with the Employment Insurance Policy Directorate to:

  • provide input and analyze data for the annual Employment Insurance Monitoring and Assessment Report
  • produce supplemental studies to inform the Report (the Plan includes supplemental studies and internal analysis – the list is available in the annual Monitoring and Assessment Report)
  • prepare an annual work plan with members of the Monitoring Report and Advisory Committee
  • coordinate the call for study proposals and prioritize work for the 2021 to 2022 work plan

Given the scale and unique objectives of each Employment Insurance benefit, the Employment Insurance program is also subject to a number of evaluations.

Each year, the Evaluation Directorate administers a questionnaire to program officials following evaluations. In 2021 to 2022, the majority of respondents agreed that:

  • the evaluation contained valid, evidence-based findings and conclusions
  • the recommendations were realistic and actionable
  • the reports were clearly written
  • the evaluators were professional at all times in terms of competence, objectivity and knowledge

Evaluators are working to address areas where the Directorate rated lower such as providing officials with sufficient time to prepare documentation required for the evaluation and demonstrating a good understanding of the program’s operating environment.

Advancing innovative approaches and gender-based analysis plus (GBA Plus)

In 2021 to 2022, the Directorate pursued innovative practices to better estimate the impact of programs on specific sub-groups and disseminate results to inform decisions. Many of these practices strengthen the conduct of GBA Plus data collection and analysis.

The incorporation of GBA Plus analysis into evaluations can help identify barriers, faced by under-served and disadvantaged populations, to access programs and services, as well as data gaps.

For example, the evaluation of the Workforce Development Agreements provided significant evidence from the literature and from service providers about programs and services as well as additional supports that can assist specific sub-groups of participants with their integration into the labour market, including Persons with Disabilities, Older Workers, Youth, Indigenous people, Social Assistance Recipients and Newcomers.

The Directorate’s work in this area also included:

  • conducting virtual expert panels for the evaluation of the Enabling Accessibility Fund using a utilization and equity-focused approach. The Directorate held 3 expert panels, each of which focused on a theme relating to accessibility for a specific age group including youth, adults, and seniors
  • using innovative causal machine learning algorithms (such as the Modified Causal Forest Method) to learn how participants with various demographic backgrounds benefit differently from labour market programs and what works for whom
  • promoting the development of a qualitative methodology framework for GBA Plus across program areas
  • extending cost-benefit model to better account for the benefits associated with stronger labour market attachment, associated with certain employment benefits and support measures, on public health care spending

Furthermore, departmental efforts, led by the Chief Data Office, in close collaboration with Statistics Canada paved the way to the integration of administrative data (for example, Canada Emergency Response Benefits) and complementary survey data from Statistics Canada. The integration of administrative and survey data is an important development in terms of enabling the conduct of more valuable and relevant policy analysis, research and evaluation activities from a GBA Plus perspective.

Taken together, these recent developments will also better inform and contextualize the contribution of the Department’s programs and services to advance progress on the Sustainable Development Goals.

Evaluation coverage in 2021 to 2022

The Evaluation Directorate uses a risk-based approach when deciding the timing and scope of future evaluations. This approach prioritizes mandatory evaluations and evaluations that are most likely to inform major policy and program decisions.

In 2021 to 2022, the Evaluation Directorate completed the 10 planned evaluations, including those required under the Financial Administration Act as well as the Policy on ResultsFootnote 3.

In collaboration with program officials, the Directorate developed 9 evaluability assessments and 3 evaluation strategies in 2021 to 2022. These documents outline options for evaluating a given program or service and contain information on the scope, methodology and timing of an evaluation project.

The PMEC is responsible for reviewing and approving these documents.

Table 1: Deputy Head approved evaluation reports for 2021 to 2022

Table 1: Deputy Head approved evaluation reports for 2021 to 2022
Planned Evaluation Reports Completed (Deputy Head Approval)
Temporary Foreign Worker Program Completed: June 2021
Federal Mediation and Conciliation Service Phase 2 Completed: June 2021
Student Work Placement Program Completed: January 2022
Canada Education Savings Program Completed: January 2022
Canada Pension Plan Disability – Reassessment Element Completed: February 2022
Wage Earner Protection Program Completed: March 2022
Union Training and Innovation Program Completed: March 2022
Employment Insurance Seasonal Claimant Pilot Project (No. 21) Completed: March 2022
Workforce Development Agreements Completed: March 2022
Enabling Fund for Official Language Minorities Completed: March 2022

Consultations

The Evaluation Directorate conducted its annual consultation process for this Plan in November 2021 to identify key evaluation priorities, evidentiary needs, and timing of future evaluations. Officials from programs and with specialized expertise (for example, Head of Performance Measurement, Department’s research divisions, the Chief Data Office, and Internal Audit Services) participated in the discussions.

The 2021 consultations emphasized the need to:

  • take into consideration the capacity constraints of programs and reduce the burden on them
  • ensure the complementarity of oversight activities (for example, audits)
  • carefully review the timing and scope of proposed evaluations to optimize their usefulness and value
  • identify lessons learned from recently introduced measures to inform future policy/program design
  • leverage linkages between evaluation and research activities to inform evaluation projects, particularly in the area of service delivery

Using information obtained through the consultations, the Directorate has adjusted the plan put forward in 2020 to 2021 in light of new policy developments and priorities. As part of these consultations, the Directorate indicated that it will continue to:

  • develop additional approaches for measuring program impacts on disadvantaged groups
  • clarify the scope of evaluations to better meet program needs or adjust to program changes
  • liaise with program areas and the Head of Performance Measurement to stay up-to-date on changes in Performance Information Profiles to ensure policy relevance and appropriate data collection for evaluations

In addition, the consultation sessions facilitated collaboration among different groups within the Department who are involved with data in order to increase the availability and integration of data within the department. The evaluation planning process also included consultations with Treasury Board of Canada Secretariat’s officials undertaken in February 2022.

Risk assessment

Once mandatory evaluations have been scheduled, the Directorate considers additional discretionary evaluations using an analysis of risk, needs and priorities. Specifically, the Evaluation Directorate examines:

  • program risks (for example, size of the population affected or targeted by the program, knowledge gaps with respect to the program)
  • departmental risks identified in the Corporate Risk Profile
  • internal capacity constraints, such as the availability of staff

This multi-factor and multi-level approach ensures that evaluators consider the perspectives of key partners and stakeholders and internal risks. The Directorate considers key corporate risks for the Department, including the Risk-Based Audit Plan. In situations where audit and evaluation schedules overlap for the same program, the Evaluation Directorate collaborates with Internal Audit Services to ensure the complementarity of internal audit/evaluation activities. This provides a means of efficiently examining performance and outcomes.

Challenges and opportunities

The Plan is also based on an analysis of the environment in which the Evaluation Directorate is operating. This includes the quality and availability of program data and the impact of the operating context (for example, COVID-19).

Quality of data and performance measurement to support evaluations

Directorate evaluations depend on timely, quality data to produce useful evidence. Evaluators use performance measurement information and other data to determine the relevance of programs and services, and whether they deliver value for money. Evaluators also need better access to administrative data to be able to improve GBA Plus analysis for certain programs.

The ESDC Data Strategy, including the advancement of a joint ESDC-Statistics Canada Data Strategy, supports the implementation of the Departmental Evaluation Plan. The Evaluation Directorate works closely with ESDC’s Chief Data Office to ensure alignment with the strategy and collaboration on opportunities to maximize the ability to leverage data and analytics for evaluation purposes.

The Evaluation Directorate supports the Department Head of Performance Measurement in the development of performance measurement and data collection practices across the Department. In particular, evaluators advise PMEC, the Head of Performance Measurement and program officials on the quality of performance information in the Performance Information Profiles and its utility in supporting evaluations. The Profiles outline the performance measurement approach for each of ESDC’s program.

There is a need in the Department to improve access to external data sources and integrated data files, in keeping with the principles of Open Data. When data are available in a timely manner, they can be harnessed more effectively. This will ultimately enable improved policy and service delivery.

The Evaluation Directorate works closely with internal and external partners to enhance our Department’s data capacity through improved access to and integration of survey and administrative data. Recent examples include the integration of the Temporary Foreign Worker Program as well as the Apprenticeship database with Statistics Canada’s databases, and the development of the Labour Market Program Data Platform in collaboration with the Chief Data Office.

In addition, evaluators carefully address data issues when developing plans for program evaluations. These evaluability assessments or evaluation strategies include a review of a program’s performance information and the quality of data supporting the performance indicators. When data gaps that could hamper an evaluation are identified, evaluators work with officials to develop approaches to mitigate those challenges.

COVID-19 pandemic-related challenges

The Evaluation Directorate experienced specific challenges relating to the pandemic, including the temporary unavailability of data due to facility closures, adaptation to virtual data collection methods, the integration of COVID-19 measures into ongoing evaluations, and challenges relating to the evaluation of short-term measures. These challenges also represented an opportunity to continue moving towards a more agile evaluation function.

Human resources capacity

In 2021 to 2022, the Evaluation Directorate had 70.78 full time equivalents. To meet short-term capacity demands, the Directorate uses flexible staffing tools such as hiring students, casual employees and consultants to fill specific capacity gaps.

Resource allocation

In 2022 to 2023, Evaluation’s budget remains largely unchanged from fiscal year 2021 to 2022. The forecasted amount for salary expenditures for fiscal year 2022 to 2023 is $6.20M and forecasted operations and maintenance is $1.50M. The Directorate will continue to focus on providing advice and planning evaluations of new and modified programs and services in addition to undertaking previously planned evaluations. It will manage its resources prudently and remain flexible in order to ensure it is capable of handling potential changes relating to the pandemic over the near term.

Figure 1. Annual directorate spending in millions ($)

Figure 1. Annual directorate spending in millions ($)

Descriptive text for figure 1:
Figure 1. Annual directorate spending in millions ($)
Year Salaries O&M Total
2010 to 2011 $ 5.65 $ 3.47 $ 9.12
2011 to 2012 $ 5.16 $ 4.06 $ 9.22
2012 to 2013 $ 5.17 $ 2.87 $ 8.04
2013 to 2014 $ 4.72 $ 2.55 $ 7.27
2014 to 2015 $ 5.51 $ 2.01 $ 7.52
2015 to 2016 $ 5.23 $ 1.02 $ 6.25
2016 to 2017 $ 4.58 $ 1.00 $ 5.58
2017 to 2018 $ 5.32 $ 1.13 $ 6.45
2018 to 2019 $ 5.13 $ 0.91 $ 6.04
2019 to 2020 $ 5.70 $ 1.30 $ 7.00
2020 to 2021 $ 6.46 $ 1.50 $ 7.96
2021 to 2022 $ 6.56 $ 1.26 $ 7.82
2022 to 2023 $ 6.20 $ 1.50 $ 7.70

Going forward

The Directorate plans to complete 15 evaluations in 2022 to 2023 and 9 evaluations in 2023 to 2024.

In the years ahead, the Evaluation Directorate will continue to:

  • collaborate with policy and program officials to provide timely evaluation evidence to inform decision makers
  • incorporate innovative methods into its operations and the services it provides to the Department and other stakeholders, the measurement of the impacts of ESDC programs as well as for the dissemination of results
  • integrate and explore ways to enhance GBA Plus data analysis and collection methods when conducting quantitative and qualitative evaluation activities, to support the Department’s efforts to address systemic inequities and disparities
  • engage experts in the academic community on how best to conduct evaluations to ensure that ESDC’s evaluators adopt best practices from other jurisdictions, departments and disciplines
  • collaborate with federal counterparts to strengthen the evaluation function across government

This document provides an evaluation coverage table in the Annex 3. The table demonstrates the Directorate will meet all coverage requirements mandated by the Treasury Board of Canada Secretariat, the Policy on Results, and the Financial Administration Act. This is in addition to completing discretionary evaluations designated as a priority by the Department. The Evaluation Directorate will use flexibilities provided in the Policy on Results to prioritize evaluations of programs.

Programs of grants and contributions that do not have a 5-year average actual expenditure of $5 million or greater per year are not required to be evaluated every 5 years. These programs include:

  • Canadian Benefit for Parents of Young Victims of Crime
  • Sustainable Development Goals
  • Skilled Trades and Apprenticeship Grants (Red Seal Program)
  • Accessible Canada Initiative
  • Strategic Engagement and Research Program

Annex 1 – Evaluation planning process

Overview of ESDC Evaluation Planning Cycle

The ESDC evaluation planning cycle is an inclusive approach to planning and governance that is compliant with Treasury Board of Canada Secretariat (TBS) requirements.

The ESDC Evaluation Directorate constantly collects and analyzes facts informing the Departmental Evaluation Plan while collaborating and sharing knowledge with key internal/external partners to reflect the continuous planning cycle. This cycle composes of 5 areas, which are outlined below.

1. Gather information to understand the evaluation environment

Factors include:

  • Departmental mandate, programs spending, and priorities
  • Mandate Letters
  • Most recent Budget
  • Departmental Results Framework
  • Departmental Risk Profile
  • Financial Administration Act and TBS submission requirements
  • Available evaluation resources
  • Challenges and opportunities (for example, data availability/quality, Audit Services Plan)

2. Consult/engage to understand partner needs and concerns

Key partners include:

  • Department’s senior management
  • Program management staff
  • Central Agencies
  • ESDC Internal Audit Services
  • Chief Data Officer
  • Head of Performance Measurement
  • Department's Research Divisions
  • Other departments/agencies
  • Working/oversight committees

3. Analyze to balance the needs, resources and mandatory requirements

Prioritization of evaluation projects that keep in mind:

  • Mandatory requirements according to the Financial Administration Act and the Policy on Results
  • made in Submissions to the TBS, Memoranda to Cabinet and other legislative requirements
  • Departmental risks, priorities, needs and concerns
  • Program-specific risks
  • Audit Services projects (and other oversight functions)
  • Balance of efforts and available resources

4. Get approval* to validate prioritization

Key partners include:

  • Departmental senior management
  • ESDC Internal Audit Services
  • Chief Data Officer
  • Head of Performance Measurement
  • Department’s research functions
  • Performance Measurement and Evaluation Committee

*The Deputy Head provides final approval

5. Communicate, implement and monitor to ensure transparency

Necessary actions to communicate and adapt to the changing environment include:

  • Share the plan with TBS
  • Make the plan publicly available
  • Monitor progress and adjust priorities, scope, and timing, if necessary
  • Administer a questionnaire to program officials following evaluations to measure the performance of the Evaluation Directorate
  • Report on completed evaluations in the subsequent plan

Annex 2 – Evaluation governance

Performance Measurement and Evaluation Committee

Composed of senior-level ESDC partners, including the Head of Evaluation.

  • Approves the Departmental Evaluation Plan
  • Reviews evaluation reports, including management responses
  • Approves Evaluability Assessments and Evaluation Strategies
  • Discusses follow-up on action plans

Evaluation Advisory Committee

Composed of stakeholders from across ESDC and/or other departments and agencies such as TBS Program Sector and Finance Canada (Director General level and below).

  • Identifies options for evaluation scope
  • Discusses preliminary findings
  • Provides input for final report

Evaluation working group

Composed of partners from across ESDC (Director level and below).

  • Focuses on technical aspects
  • Performs data collection and analysis
  • Provides input for preliminary report

Annex 3 – Planned annual coverage table

Program Last evaluation of the Program Planned Evaluation in the next 5 years Planned fiscal year of approval Reason for Evaluation
Employment Insurance Program Evaluation of the Employment Insurance Seasonal Claimant Pilot Project (2022) Evaluation of EI Parental and Maternity Benefits 2022 to 2023 First Quarter Commitment in TB Submission
Early Learning and Childcare Not Applicable. New Program. National Progress Report for the Early Learning and Child Care Program: Final Report on the Implementation of the Multilateral Framework for the 2017 to 2020 Period 2022 to 2023 First Quarter Commitment in TB Submission
Labour Market Development Agreements Third Cycle Evaluation of the Labour Market Development Agreements (2018 to 2023) Evaluation of Labour Market Development Agreements 2022 to 2023 Second Quarter Commitment in TB Submission
Literacy and Essential Skills (formerly the Skills for Success program) Literacy and Essential Skills (2017) Evaluation of the Office of Literacy and Essential Skills 2022 to 2023 Second quarter FAA Requirement
Enabling Accessibility Fund Enabling Accessibility Fund (2018) Evaluation of the Enabling Accessibility Fund 2022 to 2023 Third Quarter FAA requirement
Supports for Student Learning Programs Pathways to Education 2018 Evaluation Evaluation of the Supports for Student Learning Programs 2022 to 2023 Third Quarter FAA requirement
Labour Market Development Agreements & Opportunities Fund for Persons with Disabilities Not Applicable. GBA Plus Evaluation not previously conducted. Gender-Based Analysis Plus Evaluation Study of the Opportunities Fund and the Labour Market Development Agreements 2022 to 2023 Third Quarter Departmental Needs/Risks
Reaching HomeFootnote 4 Homelessness Partnering Strategy (2018) Evaluation of the Reaching Home Strategy 2022 to 2023 Fourth Quarter FAA requirement
Skilled Trades Awareness and Readiness Not Applicable. New Program. Skilled Trades Awareness and Readiness Evaluation 2022 to 2023 Fourth Quarter FAA requirement
Future Skills Program Not Applicable. New Program Evaluation of Future Skills Program 2022 to 2023 Fourth Quarter FAA requirement
Canada Education Savings Program Canada Education Savings Program: Summative Evaluation (2015) Canada Education Savings Program Impact Evaluation 2022 to 2023 Fourth quarter Deputy Minister Request in response to an OAG report
Sectoral Workforce Solutions Program (formerly the Sectoral Initiatives Program) Evaluation of Sectoral Initiatives Program (2018) Evaluation of Sectoral Initiatives Program 2022 to 2023 Fourth quarter FAA requirement
Skills and Partnership Fund Aboriginal Skills and Employment Training Strategy and the Skills and Partnership Fund (2020) Evaluation of the Skills and Partnership Fund 2022 to 2023 Fourth Quarter FAA Requirement
Canada Student Financial Assistance Program and Canada Apprentice Loans Not applicable Evaluation of the Canada Student Financial Assistance Program - Forgiveness for Family Doctors and Nurses 2022 to 2023 Fourth quarter Departmental Priority
Canada Pension Plan Service Improvement Strategy Not Applicable. New Program / expansion of a Pilot Evaluation of Canada Pension Plan – Service Improvement Strategy 2022 to 2023 Fourth Quarter Commitment in Treasury Board Submission
Canada Disability Savings Program Evaluation of Canada Disability Savings Program (2018) Evaluation of the Canada Disability Savings Program 2023 to 2024 Departmental Priority
Workplace Equity Evaluation of the Employment Equity Program (2019) Evaluation of the Workplace Equity Program 2023 to 2024 Departmental Priority
Canada Service Corps Not Applicable. New Program Canada Service Corps Evaluation 2023 to 2024 FAA requirement
Canada Student Financial Assistance Program and Canada Apprentice Loans Evaluation of the Canada Student Loans Program (2021) Canada Student Financial Assistance Program: Loan Repayment 2023 to 2024 Commitment in TB Submission
Employment Insurance Program Evaluation of the Work-Sharing Program (2016) Evaluation of the Work-Sharing Program 2023 to 2024 Departmental Needs/Risks
Opportunities Fund for Persons with Disabilities Opportunities Fund for Persons with Disabilities – Phase 2 Evaluation (2020) Evaluation of the Opportunities Fund for Persons with Disabilities 2023 to 2024 FAA Requirement
Old Age Security Evaluation of the Old Age Security program, phase 2 (2020) Old Age Security Service Improvement Strategy Review 2023 to 2024 Commitment in TB Sub
Indigenous Early Learning and Child Care Not Applicable. New Program. Horizontal Indigenous Early Learning and Child Care Evaluation 2023 to 2024 FAA Requirement
Employment Insurance Program Evaluation of the Employment Insurance Seasonal Claimant Pilot Project (2022) Evaluation of Employment Insurance Fishing Benefits 2023 to 2024 Departmental Needs/Risks
Social Development Partnerships Program Evaluation of Social Development Partnerships Program (2019) Evaluation of the Social Development Partnerships Program 2024 to 2025 FAA requirement
International Labour Affairs Multilateral Labour Affairs (2018) Evaluation of International Labour Affairs 2024 to 2025 Departmental Needs/Risks
Occupational Health and Safety Occupational Health and Safety (2019) Occupational Health and Safety 2024 to 2025 Departmental Priority
Labour Standards Labour Standards (2019) Labour Standards 2024 to 2025 Departmental Priority
Federal Workers Compensation Service Evaluation of the Federal Workers' Compensation Service (2018) Federal Workers’ Compensation Service 2024 to 2025 Departmental Priority
Employment Insurance Program Evaluation of the Employment Insurance Seasonal Claimant Pilot Project (2022) Evaluation of Employment Insurance Caregiving Benefits 2024 to 2025 Departmental Needs/Risks
Apprenticeship Grants Evaluation of the Apprenticeship Grants (2019) Evaluation of the Apprenticeship Grants 2024 to 2025 FAA requirement
Indigenous Skills and Employment Training Program Aboriginal Skills and Employment Training Strategy and Skills and Partnership Fund (2020) Evaluation of Indigenous Skills and Employment Training Program 2024 to 2025 FAA requirement
Youth Employment and Skills Strategy Summative Evaluation of the Horizontal Youth Employment Strategy, (2020) Horizontal Evaluation of Youth Employment Skills Strategy 2024 to 2025 FAA requirement
Student Work Placement Program Evaluation of the Student Work Placement Program – Phase 1 (2021) Evaluation of Student Work Placement Program – Phase 2 2024 to 2025 FAA requirement
Canada Student Financial Assistance Program and Canada Apprentice Loans Evaluation of the Canada Apprenticeship Loan (2020) Evaluation of the Canada Apprentice Loan 2025 to 2026 Commitment in TB Submission
Social Innovation and Social Finance Strategy Not Applicable. New Program / expansion of a Pilot Evaluation of the Social Finance Fund 2025 to 2026 Commitment in TB Submission
Old Age Security Guaranteed Income Supplement Evaluation (2020) Evaluation of the Guaranteed Income Supplement 2025 to 2026 Departmental Needs/Risks
New Horizons for Seniors Program Evaluation of the New Horizons for Seniors Program (2020) Evaluation of the New Horizons for Seniors Program (2020) 2025 to 2026 Required by the FAA
Employment Insurance Program – Special Benefits Evaluation of the Employment Insurance Seasonal Claimant Pilot Project (2022) Evaluation of Employment Insurance Extended Parental Benefits 2025 to 2026 Departmental Needs/Risks
Foreign Credential Recognition Program Evaluation of the Foreign Credential Recognition Program (2020) Evaluation of the Foreign Credential Recognition Program 2025 to 2026 FAA requirement
Passport Program Evaluation of the Passport Program (2020) Evaluation of the Passport Program 2025 to 2026 Departmental Priority
Workforce Developments Agreements Evaluation of the Workforce Development Agreements (2022) Evaluation of the Workforce Development Agreements (2027) 2026 to 2027 Commitment in TB submission
Enabling Fund for Official Language Minority Communities Enabling Fund for Official Language Minority Communities (2022) Enabling Fund for Official Language Minority Communities (2027) 2026 to 2027 FAA requirement
Federal Mediation and Conciliation Service Evaluation of the Federal Mediation and Conciliation Service – Phase 2 Federal Mediation and Conciliation Service 2 2026 to 2027 Departmental Priority

Page details

Date modified: