Policy on Results: What is Evaluation?
On this page
Evaluations seek to systematically and neutrally gather and analyze evidence to assess whether, why and how a program, initiative or policy works, with the aim of informing decision making, improvement, innovation, and accountability. They and the evaluation functions in departments play an important role in the results agenda by providing the reservoir of evidence and expertise necessary to understand the results programs and initiatives have and how they can be improved.
This note answers:
- What the purpose of evaluation is
- Who evaluations are done by
- How evaluations are done
- How evaluation relates to performance measurement
Evaluations typically consider one or more issues including whether what is being evaluated is relevant and responsive to a demonstrable need; the results it is producing and whether it is doing so effectively; its efficiency; and how it can be improved. Evaluations can target programs, initiatives, policies, priorities, projects, or other subjects, as needed. They:
- Help program managers and executives improve the achievement of results in their program or department
- Inform Memoranda to Cabinet, Treasury Board Submissions, and expenditure management generally
- Support communication to Parliament and Canadians of the results departments are achieving
- Facilitate innovation and learning within organizations
Large Departments must:
- Have an evaluation function, led by a Head of Evaluation and overseen by a Performance Measurement and Evaluation Committee (PMEC) chaired by the Deputy Head
- Have a five-year evaluation plan based on the principle that all spending should be evaluated periodically and then target evaluations strategically to focus on risks, needs and priorities, providing a rationale for what will not be evaluated, where applicableFootnote 1
- Must have a Head of Evaluation and a PMEC
- Need not have a formal evaluation plan, but must annually conduct an evaluation planning exercise to identify the department’s five-year evaluation needs by considering information needs to support management decision making, other planned departmental oversight activities, and joint evaluations with other departments
Evaluation functions must be internal to the department but outside of program areas. The credibility and utility of evaluations is derived from respectful collaborative relationships in which Heads of Evaluation and evaluators demonstrate neutrality in behavior, process and decisions, and actively apply evaluation standards, and individual values and ethics. Evaluation reports are approved by the Deputy Head. Centrally-led evaluations can be conducted as necessary.
Evaluations typically use social science methods such as document reviews, interviews, surveys, case studies, focus groups, statistics and randomized control trials. They can look at a wide variety of sources to gather information, ranging from program beneficiaries to social media to predictive analytics based on big data. They can be conducted before a program is implemented, to inform its design; during a program, to help it adjust course; or after a program is complete or has been running for some time, to assess its impact.
Common types of evaluations include:
- Impact evaluations, which focus on assessing the results of the program and understanding whether the program caused the results
- Economic evaluations, which consider whether the program was cost-effective, or whether the costs justified the benefits
- Innovation evaluations, which assess pilots where departments are experimenting with new approaches and want to determine efficacy and the feasibility of taking pilots to scale
- Delivery/Process/Formative evaluations, which consider how to improve the implementation or delivery of the program
- Theory of Change evaluations, which explore and test the underlying assumptions of why and how a program works
- Design/Needs evaluations, which aim to develop or assess program design, or redesign an existing program
Evaluation and Performance Measurement
Performance measurement and evaluation are part of the same continuum and help support each other. Heads of both functions report to the Performance Measurement and Evaluation Committee, and both functions help inform program managers, senior management, central agencies, Parliament and Canadians.
Report a problem or mistake on this page
- Date modified: