Policy on Results: What is Evaluation?

On this page

Evaluations seek to systematically and neutrally gather and analyze evidence to assess whether, why and how a program, initiative or policy works, with the aim of informing decision making, improvement, innovation, and accountability. They and the evaluation functions in departments play an important role in the results agenda by providing the reservoir of evidence and expertise necessary to understand the results programs and initiatives have and how they can be improved.

This note answers:

Purpose

Evaluations typically consider one or more issues including whether what is being evaluated is relevant and responsive to a demonstrable need; the results it is producing and whether it is doing so effectively; its efficiency; and how it can be improved. Evaluations can target programs, initiatives, policies, priorities, projects, or other subjects, as needed. They:

  • Help program managers and executives improve the achievement of results in their program or department
  • Inform Memoranda to Cabinet, Treasury Board Submissions, and expenditure management generally
  • Support communication to Parliament and Canadians of the results departments are achieving
  • Facilitate innovation and learning within organizations

Who

Large Departments must:

  • Have an evaluation function, led by a Head of Evaluation and overseen by a Performance Measurement and Evaluation Committee (PMEC) chaired by the Deputy Head
  • Have a five-year evaluation plan based on the principle that all spending should be evaluated periodically and then target evaluations strategically to focus on risks, needs and priorities, providing a rationale for what will not be evaluated, where applicableFootnote 1

Small Departments:

  • Must have a Head of Evaluation and a PMEC
  • Need not have a formal evaluation plan, but must annually conduct an evaluation planning exercise to identify the department’s five-year evaluation needs by considering information needs to support management decision making, other planned departmental oversight activities, and joint evaluations with other departments

Evaluation functions must be internal to the department but outside of program areas. The credibility and utility of evaluations is derived from respectful collaborative relationships in which Heads of Evaluation and evaluators demonstrate neutrality in behavior, process and decisions, and actively apply evaluation standards, and individual values and ethics. Evaluation reports are approved by the Deputy Head. Centrally-led evaluations can be conducted as necessary.

How

Evaluations typically use social science methods such as document reviews, interviews, surveys, case studies, focus groups, statistics and randomized control trials. They can look at a wide variety of sources to gather information, ranging from program beneficiaries to social media to predictive analytics based on big data. They can be conducted before a program is implemented, to inform its design; during a program, to help it adjust course; or after a program is complete or has been running for some time, to assess its impact.

Common types of evaluations include:

  • Impact evaluations, which focus on assessing the results of the program and understanding whether the program caused the results
  • Economic evaluations, which consider whether the program was cost-effective, or whether the costs justified the benefits
  • Innovation evaluations, which assess pilots where departments are experimenting with new approaches and want to determine efficacy and the feasibility of taking pilots to scale
  • Delivery/Process/Formative evaluations, which consider how to improve the implementation or delivery of the program
  • Theory of Change evaluations, which explore and test the underlying assumptions of why and how a program works
  • Design/Needs evaluations, which aim to develop or assess program design, or redesign an existing program

Evaluation and Performance Measurement

Performance measurement and evaluation are part of the same continuum and help support each other. Heads of both functions report to the Performance Measurement and Evaluation Committee, and both functions help inform program managers, senior management, central agencies, Parliament and Canadians.

Evaluation Performance Measurement
  • Typically periodic
  • Done by neutral evaluators who work outside of the programs
  • Focuses on explaining whether, why and how a program or policy works, as well as what results can be attributed to the program itself
  • Often useful to identify and compare solutions to problems identified with program-led performance measurement
  • By analogy, looking ‘under the hood’ of a car to explain or improve its performance
  • Ongoing
  • Done by program managers who work in consultation with the Head of Performance Measurement and the Head of Evaluation
  • Uses performance indicators and progress compared to targets to assess performance
  • Often useful to identify problems
  • By analogy, a car’s dashboard, informing about speed, engine temperature and fuel levels
Report a problem or mistake on this page
Please select all that apply:

Thank you for your help!

You will not receive a reply. For enquiries, contact us.

Date modified: