Departmental Evaluation Plan: Fiscal years 2022 to 2023 until 2026 to 2027

List of tables

• Table 1: Evaluation positions as of April 2022
• Table 2: Evaluation projects conducted in the 2021 to 2022 fiscal year
• Table 3: Number of new discretionary and mandatory evaluations, by fiscal year
• Table 4: Evaluations in fiscal years 2022 to 2023 and 2023 to 2024

Deputy Head confirmation note

I approve the Departmental Evaluation Plan for fiscal years 2022 to 2023 until 2026 to 2027, which I submit to the Treasury Board of Canada Secretariat as required by the Policy on Results (2016).

I confirm that this 5-year rolling Departmental Evaluation Plan^{Footnote 1}:

• meets the requirements of the Mandatory Procedures for Evaluation
• supports the requirements of the expenditure management system including, as applicable, Memoranda to Cabinet, Treasury Board submissions, and resource alignment reviews

I will ensure that this plan is updated annually, and I will provide information about its implementation to the Treasury Board of Canada Secretariat, as required.

Original signed by:

Paul Thompson
Deputy Minister
Public Services and Procurement Canada

1. Introduction

This document presents the Departmental Evaluation Plan (the Plan) for Public Services and Procurement Canada (PSPC) for fiscal years starting in 2022 to 2023 and ending in 2026 to 2027, developed in compliance with the requirements of the Treasury Board (TB) Policy on Results (2016) and the Financial Administration Act.

1.1 Purpose of the Plan

The Plan’s purpose is to ensure that the department meets its evaluation-related accountability obligations as well as senior management needs. The Plan outlines PSPC’s operational context and the planning, assessment, and prioritization process used to identify areas for an evaluation, so far as to provide information on departmental spending, coverage and rationales for any exclusions over the 5-year timeframe.

Introduced by the TB Secretariat, the Policy on Results (2016) requires every large department, like PSPC, to have a 5-year evaluation plan based on the principle that all spending should be evaluated periodically and evaluations strategically focus on the risks, needs and priorities of the Department and Government of Canada (GoC). That said, the departmental 5-year evaluation plan is updated yearly, then is approved by the deputy minister and shared with the TB Secretariat.

1.2 Development of the Plan

The Plan ensures senior management has access to credible, neutral and timely information to support evidence-based decision-making on policy, expenditure management and program improvement.

Ensuring evaluation coverage requirements are being met involves^{Footnote 2}:

• reporting on the status of what was completed in the 2021 to 2022 fiscal year
• ensuring the Plan meets the requirements of the Mandatory Procedures for Evaluation
• supports the requirements of the expenditure management system including, as applicable, Memoranda to Cabinet, TB submissions, and resource alignment reviews

With the integration of both the Audit and Evaluation functions, the Plan sought to continue building on a strategic approach to harmonize the planning exercises, to the extent possible. The objective was not only to promote better synergy between the 2 functions, but also to reduce the burden of this planning exercise on the stakeholders. To achieve this, the following activities in the Plan were carried out in collaboration with Audit:

• planning and engagement of joint consultations with the Department's senior management
• alignment of the evaluation and audit schedules with respect to governance and development of common presentations to some committees

2. Public Service and Procurement Canada Evaluation Sector

This section describes the environment and parameters within which the Evaluation Sector (ES) operates, plans and manages evaluations. It also covers the ES capacity and resources.

2.1 Organizational structure and governance of the Evaluation Sector

The mission of the ES is to conduct high-quality and timely evaluations to guide and support the decision-making process, improvements, innovation and accountability.

The Chief Audit, Evaluation and Risk Executive is the designated Head of Evaluation and is supported by the Director of Evaluation, as well as the Evaluation team.

2.2 Overview of the Evaluation Sector

The 2 following sub-sections describe the ES’s goals and the types of services the ES offers as it relates to the priorities of the department, GoC, and in accordance with the Policy on Results.

2.2.1 Goal

The unifying goal supported by the ES team’s work is to provide sound, professional, in-house evaluation services to support the decision-making process, operations improvement, and accountability measures within the department. The ES will continue to:

• provide timely delivery of high-quality evaluations and information, within budget and in conformance with professional standards, to support decision-making and meet legal and TB requirements
• enrich the internal evaluation team capacity by investing in people and fostering an environment of innovative thinking
• ensure timely public release of evaluation reports
• follow up on the implementation of Management Action Plans (MAP) in response to evaluation recommendations to ensure improvements have been implemented

2.2.2 Types of evaluation services

The ES applies the flexibility provided by the Policy on Results and offers different types of evaluations to meet the information needs of senior management. Both mandatory and discretionary evaluations are most often conducted at PSPC.

Mandatory evaluations: Evaluations of programs or initiatives that must be implemented at specific times in accordance with commitments made to Treasury Board. In addition, horizontal evaluations led by other departments in which PSPC is involved as well as assessments conducted in compliance with policy requirements, such as the neutral assessment of the evaluation function. Also, evaluations of all ongoing grants and contributions programs with 5-year average actual expenditures of $5 million or greater per year are mandatory. However this requirement does not apply to PSPC since it has no grants and contributions programs.

Discretionary evaluations: Evaluation of programs, internal services, policies and priorities for which the evaluation schedule is based on an analysis of risks and senior management needs and priorities.

The evaluations take into account the lifecycle of a program, initiative or policy (design, implementation and others) and use social science research methods that enable the neutral and systematic gathering and analysis of evidence.

In addition to conducting evaluations, the ES offers a variety of advisory services relating to performance measurement, research, data analysis and special targeted studies intended to support the department in an effective and timely manner.

The ES also has resources dedicated to professional practices to conduct activities that strengthen the ES. Apart from planning, reporting and liaison activities with federal institutions, the objective of the professional practices component is to develop tools to promote innovation and efficiency within the evaluation team.

2.3 Resources of the Evaluation Sector

The 2 following sub-sections describe the ES team composition and its funding arrangements.

2.3.1 The evaluation team

In fiscal year 2021 to 2022, the ES has been in transition with a focus on rebuilding the evaluation capacity. As of April 2022, the evaluation team comprised of 9 full-time equivalent (FTE) positions, in addition to 2 coop students and 1 casual. The rebuilding of the team was achieved by onboarding employees through promotions, deployments, secondments, assignments and collective competitions to staff vacant EC-04, EC-05, EC-06 and EC-07 positions. In total, 4 students and 2 casual employees were also hired to support the ES in the delivery of the Evaluation plan for the 2021 to 2022 fiscal year.

2.3.2 Financial resources

Over the last several years, the ES has had an ongoing funding pressure as it is only partially funded (approximately 7 positions of 13 FTEs and 2 students required) representing an annual budget of approximately $948,000 ($765,000 in salaries, excluding the employee benefit plan and $183,000 in operations and maintenance).

3. Key achievements of the Evaluation Sector in 2021 to 2022

This section outlines the key results and accomplishments the ES achieved against the goals established in corresponding reports, plans and priorities.

3.1 Evaluation projects conducted in the 2021 to 2022 fiscal year

The evaluation team worked on a total of 5 evaluation projects for the 2021 to 2022 fiscal year. Of these evaluation projects, 3 were led in house, and 2 were horizontally led by Environment Climate Change Canada (ECCC) and Canadian Heritage (PCH), respectively. While the ES, like others, experienced operational transformations, the team continued to meet its key obligations in support of departmental priorities.

3.2 Reconciliation framework

This year, the ES continued its commitment to incorporate Indigenous perspectives into the practice of evaluations, as appropriate. Striving to transform the current design of evaluation practices at PSPC, the ES increasingly emphasizes models of co-development and co-creation with Indigenous partners in all of its evaluation projects.

As such, the ES is developing a framework to support evaluators in incorporating Indigenous perspectives into the design of evaluations. The purpose is to guide PSPC evaluators in incorporating Indigenous evaluation knowledge, world views and/or Indigenous capacity development at key points in the evaluation lifecycle. Ultimately, this framework will support evaluators to strengthen their capacity, enhance and coordinate Indigenous policy and evaluation networks, and collaborate with subject matter experts to advance evaluation tools, guides and strategies that integrate Indigenous knowledge and worldviews. The framework is targeted for completion and approval in fiscal year 2022 to 2023.

3.3 Evaluation Learning Series

The Evaluation Learning Series developed by the ES occurs at least monthly on a wide range of subjects to build the knowledge base of all members of the evaluation team. Topics include evaluation-related subjects where guest speakers from the Evaluation functions in other government departments are invited to share their experience, or topics in which subject matter experts within PSPC are invited to present. The series started in December 2021. There have been 9 learning series events.

4. Annual evaluation planning exercise

This section outlines the evaluations anticipated for fiscal years starting in 2022 to 2023 and ending in 2026 to 2027. Steps and factors that went into identifying all ES engagements are also described below.

4.1 Approach and methodology overview

For this year’s planning process, the ES used multiple lines of input to help determine the timing, scope, and type of evaluation engagements over the upcoming 5 year period. To ensure the coverage decisions are precisely planned, key steps such as identifying the evaluation universe, consultations, and prioritization of evaluation projects were conducted.

A draft list of evaluation projects covering the next 2 fiscal years (2022 to 2024) was presented to the Evaluation Advisory Committee (EAC) for senior management review and feedback in February 2022 and afterwards presented to the Departmental Audit Committee and the Executive Committee for information in March 2022. The complete Plan was presented to the EAC in June 2022 for endorsement. Subsequently, the list of evaluation projects planned for the next 5 fiscal years (Annex 1: Planned Evaluation Projects for 2022 to 2027) was sent secretarially to the Performance Measurement, Evaluation and Results Committee for endorsement and then recommended for Deputy Minister approval.

The Plan has been developed to support flexibility in timing, issue and scope of evaluations to support senior management information needs and departmental commitments. As such, this plan should be considered evergreen in nature and will continue to be revised annually.

4.2 Planned evaluation projects

A total of 25 evaluation projects are planned to be completed between fiscal year 2022 to 2023 and fiscal year 2026 to 2027^{Footnote 3}.

Amongst the evaluation projects, 20 projects will begin within the 5 year time horizon. Of these, 7 are mandatory (3 of which are horizontal) and 13 are discretionary. Table 4 demonstrates this breakdown of evaluations starting in each fiscal year.

In addition to the 20 new evaluation projects identified in table 4, the ES is completing 5 evaluation projects which were started in fiscal year 2021 to 2022. Amongst these, 3 are mandatory evaluation projects, including 2 horizontal evaluations.

Table 4 displays the simplified schedule of evaluation projects covering the 2022 to 2023 fiscal year. Whereas, Annex 1: Planned Evaluation Projects for 2022 to 2027 contains the detailed schedule of all evaluation projects for the next 5 fiscal years from 2022 to 2023 until 2026 to 2027. In addition, any exclusions and their rationales from this Plan are provided in Annex 2: Program Excluded from the 2021 to 2025 Plan.

Annex 1: Planned evaluation projects for 2022 to 2027

Annex 2: Programs excluded from the 2021 to 2025 plan

The following programs are excluded from the plan:

Payments and collection of revenues (2.75% of departmental planned program spending in 2021 to 2022)

The purpose of this program is to ensure the integrity of the Treasury for Canadians as well as to develop and maintain the Receiver General’s treasury systems. The Receiver General program was evaluated in 2015 to 2016. Consequently, the priority for evaluating this program is low, unless there is a major change in the program’s design.

Government-wide accounting and reports (0.4% of departmental planned program spending in 2021 to 2022)

This program helps to ensure maintenance of the accounts of Canada and the Chart of Accounts at the federal administration level as well as the production of the Public Accounts of Canada, including the audited consolidated annual financial statements and the Monthly Statements of Financial Operations for the Government of Canada. The Receiver General program was evaluated in 2015 to 2016. Consequently, the priority for evaluating this program is low, unless there is a major change in the program’s design.

Communication services (0.4% of departmental planned program spending in 2021 to 2022)

This program sector supports the services providing information to Canadians and government institutions on Government of Canada programs and publications. The activities of this program have not changed much over the years and remain relatively stable. However, the evaluation requirements will be reviewed at the appropriate time, given the communication modernization project underway.

Asset disposal (0.1% of departmental planned program spending in 2021 to 2022)

The asset disposal program includes GCSurplus (GCSurplus.ca, GCMil, GCTransfer and GCRecycle) and the management of seized goods. With the last evaluation dating back to 2006 to 2007, the priority for evaluating this program is low, unless there is a major change in the program’s design. Modernization efforts are currently being proposed. We will monitor the implementation of the modernization initiative and will determine the need for an evaluation in future annual updates of the Departmental Evaluation Plan.

Canadian General Standards Board (0.05% of departmental planned program spending in 2021 to 2022)

The Canadian General Standards Board is a centre of expertise within the Government of Canada that develops and updates national standards and provides certifications for personnel, products and organizations. Since the last official evaluation dates back to 2008, the program conducted a validated self-evaluation in 2016 to 2017. The priority for evaluating this program is low, unless there is a major change in the program’s design.