Departmental Evaluation Plan 2024-25 to 2028-29

Evaluation Services Directorate
June 25, 2024

On this page

• List of acronyms
• List of tables
• List of figures
• 1. Objective and Overview
  • 1.1. Message from the Deputy Minister
• 2. The Department of Canadian Heritage
  • 2.1. Canadian Heritage Evaluation Governance
  • 2.2. Canadian Heritage Evaluation Services Directorate
• 3. Evaluation Plan Coverage 2024-25 to 2028-29
  • 3.1. Four Evaluation Priorities 2024-25
• 4. Overview of the PCH 5-year Evaluation Plan
• Annex 1: Evaluation achievements in 2023-24
• Annex 2: Modifications from the 2023-24 evaluation plan
• Annex 3: Annual Evaluation Planning Exercise
• Annex 4: Neutral Assessment of the evaluation function
• Annex 5: Schedule of evaluations per sector for 2023-24 to 2027-28

List of acronyms

FAA

Financial Administration Act

PCH

Canadian Heritage

List of tables

• Table 1: core responsibilities and expenditure forecasts, 2024-25
• Table 2: evaluation Services Directorate budget, 2024-25
• Table 3: schedule of evaluations 2024-25
• Table 4: Cultural Affairs
• Table 5: Community and Identity
• Table 6: Multiculturalism and Anti-Racism
• Table 7: Official Languages, Heritage and Regions
• Table 8: Sports, Major Events and Commemorations
• Table 9: Strategic Policy, Planning and Corporate Affairs
• Table 10: Internal Services
• Table 11: Horizontal initiatives from other Government Departments

List of figures

• Figure 1: evaluation in the Government of Canada

1. Objective and Overview

The Department of Canadian Heritage Departmental Evaluation Plan describes the evaluation priorities and the schedule of projects for the five-year period from 2024-25 to 2028-29.

The plan includes:

• Message from the Deputy Minister
• Context on the Department of Canadian Heritage and its evaluation function
• Summary of evaluation coverage and priorities
• Calendar of projects planned for the five-year period
• Annexes: supplemental information

1.1. Message from the Deputy Minister

I approve the Departmental Evaluation Plan of the Department of Canadian Heritage for the fiscal years 2024-25 to 2028-29 for submission to the Treasury Board of Canada Secretariat as required by the Policy on Results (2016). I confirm that this five-year rolling Departmental Evaluation Plan:

• allows Canadian Heritage to evaluate, at least every five years, all ongoing grants and contributions programs with five-year average actual expenditures of $5 million or greater per year, in accordance with the requirements of section 42.1 of the Financial Administration Act (FAA);
• meets the requirements of the Mandatory Procedures for Evaluation; and
• supports the requirements of the expenditure management system, including, as applicable, Memoranda to Cabinet, Treasury Board Submissions and resource alignment reviews.

The Department has a broad mandate and plays an important role in the lives of Canadians. Canadian Heritage supports cultural industries, the arts, heritage and celebration, sports, official languages, Indigenous languages, diversity and inclusion, as well as youth.

Evaluation results are supporting evidence-based decision-making at Canadian Heritage. A strong evaluation function contributes to the improvement of the Department’s many programs and initiatives, enabling Canadians to create, share and participate in a diverse and inclusive society.

This current five-year evaluation plan acknowledges pressures on the evaluation function to fully address all demands. It takes a risk-based approach to prioritize projects to respect the Policy on Results and the Financial Administration Act while being useful and timely for senior management decision-making. It will continue to improve how evaluations support government priorities on Gender-based Analysis Plus, Inclusion, Diversity, Equity and Accessibility, as well as reconciliation with Indigenous peoples and implementation of the modernization of the Official Languages Act.

I will ensure that this plan is updated annually, and I will provide information about its implementation to Treasury Board Secretariat, as required.

Original signed by:

Isabelle Mondou
Deputy Minister
Canadian Heritage

2. The Department of Canadian Heritage

The Department of Canadian Heritage plays a vital role in the cultural, civic and economic life of Canadians. It focuses on strengthening and promoting “Canadian identity and values, cultural development, and heritage.”

The Department of Canadian Heritage Act provides jurisdiction over: human rights; multiculturalism; arts; cultural industries and heritage; sports; official languages; state ceremonial and Canadian symbols; broadcasting; conservation, export and import of cultural property; and national libraries, archives and museums.

Canadian Heritage’s Departmental Results Framework, established in November 2017, sets out five core responsibilities that reflect the Department’s mandate.

Source: Canadian Heritage – 2024-25 Departmental Plan. The figures include the following expenditures: salaries, operations and maintenance, capital, statutory expenditures, Grants and Contributions and the cost of employee benefit plans.

Figure 1: Evaluation in the Government of Canada – text version

Standards/Requirements:

• Treasury Board Policy on Results (2016) and related directives
• Financial Administration Act: Section 42.1
• Guidance for Ethical Evaluation practice – Canadian Society Evaluation

Evaluation, the systematic and neutral collection and analysis of evidence to judge merit, worth or value

• Program Implementation
  • Inform program improvement
  • Advise on performance information
• Program Reporting
  • Support results reporting
  • Contribute to corporate reporting
• Program planning
  • Advise on MCs and TB subs
  • Input into corporate planning

Results:

• Required Treasury Board of Canada Secretariat accountability and reporting
• Analysis and advice to support senior management decision making and risk management
• Departmental and Program learning

2.1. Canadian Heritage Evaluation Governance

The Assistant Deputy Minister of the Strategic Policy, Planning and Corporate Affairs sector assumes the role of Head of Evaluation and reports directly to the Deputy Minister.

The Canadian Heritage Executive Committee serves as the Performance Measurement and Evaluation Committee as per the Policy on Results.

The Head of Evaluation and the Director General of the Planning, Evaluation and Research Branch are supported by the work of the Evaluation Services Directorate which is led by the Director of Evaluation.

2.2. Canadian Heritage Evaluation Services Directorate

The Evaluation Services Directorate:

• Is an internal evaluation function with experienced and qualified evaluators.
• Provides quality and timely evaluations and advice to support accountability, decision-making, expenditure management, continuous program improvement and policy development.
• Uses external consultants to supplement its capacity as needed.

A summary of Evaluation Services Directorate achievements in 2023-24 is presented in Annex 1.

Source: PERB operational budget, 2024-25
Note: Not all positions are currently filled; in 2023-24 the ESD had approximately 19 positions filled.

3. Evaluation Plan Coverage 2024-25 to 2028-29

Following the annual evaluation planning exercise,^{Footnote 1} the plan provides the following evaluation coverage:

• 100% of ongoing Grants and Contribution programs with 5-year average actual expenditures of $5 million or greater annually
  • 92% of the Department’s total expenditures for 2024-25
• Commitments outlined in Treasury Board Submissions.
• Certain projects requested by senior management.

This 5-year evaluation plan:

• includes 6 projects that will not respect the Financial Administration Act for 5-year timing: Celebration and Commemoration, TV5, Canada Cultural Investment Fund, Canada Cultural Spaces Fund, Canada Book Fund and Hosting Program; and
• reflects some changes in projects and timing compared to the 5-year plan 2023-24 to 2027-28 (see Annex 2).

3.1. Four Evaluation Priorities 2024-25

1. Respect the evaluation requirements of the Treasury Board Policy on Results

• Timely completion of Financial Administration Act required evaluations as possible
• Annual evaluation planning exercise for the development of a risk-based five-year plan
• Public release of evaluation reports within 120 days of Deputy Minister approval
• Semi-annual analysis of progress on the implementation of Management Response Action Plan items
• Integration of a Gender Based Analysis Plus lens
• Consider and implement any changes from the review of the Policy on Results, currently underway

2. Complete strategic evaluation initiatives

• Horizontal analysis of evaluation conclusions and recommendations, 2019-2024
• Roll-up of evaluation findings related to COVID-19, 2021-2025
• Review and update evaluation tools and guidelines to respect and advance key priorities of inclusion, diversity, equity and accessibility, as well as reconciliation with Indigenous peoples
• Review and confirmation of new risk-based evaluation planning approach

3. Address key evaluation function challenges and opportunities

Implement actions identified in the Management Response and Action Plan for the Neutral Assessment of the PCH Evaluation Function (see Annex 4) to:

• Continue to implement models and approaches that support timely completion of evaluations
• Support strong performance measurement across programs and systems for stronger results reporting
• Strengthen the neutrality and independence of the evaluation function

4. Maintain and strengthen evaluation capacity to fulfill objectives

• Maintain and strengthen internal evaluation
  • Attract and retain evaluation professionals within a competitive environment
  • Promote an environment of learning
  • Complete human resource and succession planning
  • Respect employment equity, diversity and inclusion
• Support a healthy workplace

4. Overview of the PCH 5-year Evaluation Plan^{Footnote 2}

Year 1: 2024-25

• 10 ongoing evaluations
• 5 new to launch
• 9 for approval

Year 2: 2025-26

• 6 ongoing evaluations
• 6 new to launch
• 4 for approval

Year 3: 2026-27

• 8 ongoing evaluations
• 5 new to launch
• 7 for approval

Year 4: 2027-28

• 8 ongoing evaluations
• 6 new to launch
• 4 for approval

Year 5: 2028-29

• 8 ongoing evaluations
• 6 new to launch
• 7 for approval

Total of 33 programs to be evaluated, 30 of the 33 programs require evaluations, 4 evaluations will group programs.

Annex 1: Evaluation achievements in 2023-24

• Four evaluations approved in 2023-24. Another 10 evaluations in various stages of progress.
• Follow-up on progress in implementing approved evaluation management action plans, as per the Policy on Results.
  • 2023-24 calibrated mid-year exercise reported on 13 of 21 evaluations; of 23 outstanding recommendations, 6 were completed, 9 were delayed and 6 in progress.
• Strong satisfaction ratings from program clients – 98%
  • Post-evaluation project surveys with program clients focus on: utility, relationships, processes and reports, and project management
• Decreased contracting and improved internal capacity and oversight.
  • Stabilized organizational structure and ensured training and development for evaluators.
  • 72% reduction in operations and materials spending (mostly contracting) to $162,063 in 2023-24 from a previous 5-year spending average of $580,741.
• Supported the completion of the mandatory Neutral Assessment of the Evaluation Function as per the Policy on Results.

Annex 2: Modifications from the 2023-24 evaluation plan

Annex 3: Annual Evaluation Planning Exercise

Annex 3: Annual evaluation planning exercise – text version

Overview of the Planning Exercise: in compliance with the Treasury Board’s requirements, the Evaluation Services Directorate undertakes an annual planning exercise to determine which evaluation projects will be conducted during the next five-year evaluation cycle.

1. Evaluation Universe: the evaluation universe is the starting point for planning. The 2024-25 program directory, which indicates the Department’s direct program spending, and the Departmental Results Framework, were used as a basis for determining the evaluation universe to give us an overview of all the mandatory and potential “evaluable” units.
2. Planning Process:
   1. consultation of relevant documents (Policy on Results Interim Guidance, Departmental Plan, Departmental Risk Profile, Performance Information Profiles) used as a reference to account for the environmental context and determine the potential changes and additions.
   2. risk evaluation for planning purposes to update program information.
   3. preparation of the Annual Reference Level Update for the coming year.
   4. analysis of mandatory evaluations under the Financial Administration Act and commitments stemming from Treasury Board submissions, compared with discretionary evaluations in accordance with the flexibility offered by the Directive on Results.
3. Consultations: the Evaluation Services Directorate has conducted consultations with the Treasury Board Secretariat, the Office of the Chief Audit Executive, Program’s senior management and other federal departments and agencies. The objective was to set departmental priorities with regard to evaluation and take into consideration the specific needs and concerns expressed by the partners.
4. Prioritization of evaluation projects: risk evaluation for purposes of planning evaluation projects and consultations, keeping in mind the review of documents to be used to update the five-year evaluation schedule. They are also used to ensure the balanced distribution of resources and efforts, as well as to meet the requirements of the Treasury Board Secretariat.
5. Approval:
   1. the draft plan is submitted to the Director of the Evaluation Services Directorate, the Director General and the Head of Evaluation.
   2. the plan is submitted to the Executive Committee, which can recommend it for approval by the Deputy Minister.
   3. the plan is forwarded to the Treasury Board Secretariat and posted on the canada.ca site.

Annex 4: Neutral Assessment of the evaluation function

The Neutral Assessment of the Evaluation Function, required to be conducted every 5 years to respect the Policy on Results and completed in March 2024 by an external consultant, contains three recommendations directed to Canadian Heritage that will be subject to a management response and action plan:

1. Further develop focused and flexible approaches to the planning and implementation of evaluations, based on risks, to allow meeting management needs and priorities while also respecting the Financial Administration Act requirements.
2. Better support the utility, neutrality and independence of the evaluation function, by: (a) briefing the Head Of Evaluation more systematically on evaluation issues; and (b) enhancing opportunities for the evaluation function to strategically engage ExCom (Performance Measurement Executive Committee) and other senior management members in evaluation issues.
3. Strengthen and streamline program-level performance measurement to simplify the planning and accelerate the conduct of program evaluations by improving:
   1. the understanding of gaps and solutions;
   2. the coordination among departmental stakeholders; and
   3. the advice to ExCom.

Annex 5: Schedule of evaluations per sector for 2023-24 to 2027-28^{Footnote 8}

© His Majesty the King in Right of Canada, as represented by the Minister of Canadian Heritage, 2024
Catalogue No.: CH1-39E-PDF
ISSN: 2561-018X