Shared Services Canada Five-Year Departmental Evaluation Plan  2023-24 to 2027-28

This publication is also available online on Shared Services Canada’s website.

Permission to Reproduce

Except as otherwise specifically noted, the information in this publication may be reproduced, in part or in whole and by any means, without charge or further permission from Shared Services Canada, provided that due diligence is exercised to ensure the accuracy of the information reproduced is maintained; that the complete title of the publication is produced; that Shared Services Canada is identified as the source institution; and that the reproduction is not represented as an official version of the information reproduced, nor as having been made in affiliation with, or with the endorsement of the Government of Canada.

Commercial reproduction and distribution is prohibited except with written permission from Shared Services Canada. For more information, please contact Shared Services Canada at ssc.information.spc@canada.ca.

© His Majesty the King in Right of Canada, as represented by the Minister responsible for Shared Services Canada, 2023.

Shared Services Canada’s Five-Year Departmental Evaluation Plan 2023-24 to 2027-28

Cat. No. P115-13E-PDF

ISSN 2817-8025

Publié aussi en français sous le titre :

Plan d’évaluation quinquennal du ministère 2023-2024 à 2027-2028

No. de catalogue P115-13F-PDF

ISSN 2817-8033

Deputy Head Confirmation Note for All Departments

I approve the Departmental Evaluation Plan of Shared Services Canada for the fiscal years 2023-24 to 2027-28, which I submit to the Treasury Board of Canada Secretariat as required by the Policy on Results.

I confirm that this five-year rolling Departmental Evaluation Plan:

• Plans for evaluation of all ongoing programs of grants and contributions with five-year average actual expenditures of $5 million or greater per year at least once every five years, in fulfillment of the requirements of subsection 42.1 of the Financial Administration Act,
• Meets the requirements of the Mandatory Procedures for Evaluation, and
• Supports the requirements of the expenditure management system including, as applicable, Memoranda to Cabinet, Treasury Board submissions, and resource alignment reviews.

I will ensure that this plan is updated annually, and I will provide information about its implementation to the Treasury Board of Canada Secretariat, as required.

1.0 Context and Background

1.1 Purpose and Government Priorities

Evaluation is the systematic and neutral collection and analysis of evidence to judge merit, worth or value.

This document presents Shared Services Canada’s Departmental Evaluation Plan. The Plan is an information and management tool that reflects the results of the department’s annual evaluation planning exercise. The Plan identifies evaluations for the next five years and the resourcing of the evaluation function.

The scope and content of this plan are consistent with the requirements of the Treasury Board Policy on Results.

1.2 Government of Canada Context

Evaluation is a priority of the Government of Canada. The President of the Treasury Board has been charged with instilling a culture of evaluation, measurement and evidence-based decisions across departments for program and policy design and delivery.

The December 2021 mandate letters directed all departments to draw on lessons learned from the pandemic to further adapt and develop more agile and effective ways to serve Canadians. The upcoming SSC evaluations will help the department meet this commitment.

SSC evaluations will also support the department in any upcoming Strategic Policy Reviews. Specifically, the evaluations will provide information to verify the relevance and effectiveness of SSC programs. The analysis of efficiency will identify opportunities to improve processes and reallocate resources to adapt government programs and operations to the post-pandemic reality.

1.3 Requirements of the Policy on Results

The Policy on Results requires that deputy heads annually approve a five-year evaluation plan and provide this plan to the Secretariat of the Treasury Board of Canada (TBS). The Plan should include:

• A detailed schedule of planned evaluations.
• The extent of evaluation coverage of the Program Inventory, by amount of organizational spending and by program of the Program Inventory.

• The extent of organizational spending and programs that will not be evaluated in the planning period, and justification (i.e. what has been excluded from the five-year schedule and why). 
• Advice to the deputy heads on the adequacy of resources for evaluation.

1.4 Gender-Based Analysis Plus (GBA+)

GBA+ is an analytical process that provides a rigorous method for the assessment of systemic inequalities, and to assess how diverse groups of women, men, and gender diverse people may experience policies, programs and initiatives.

Evaluation offers an opportunity to examine considerations related to gender and other intersecting identity factors. Shared Services Canada is committed to improving the integration of GBA+ and accessibility considerations into its evaluation processes.

1.5 Governance

The Policy on Results calls for deputy heads to establish and chair a Performance Measurement and Evaluation Committee (PMEC) of senior officials. At SSC, the PMEC responsibilities are carried out by the Executive Oversight Board (EOB). The EOB is SSC’s most senior executive governance body. It sets strategic direction and departmental priorities, oversees SSC’s performance measurement and evaluation functions, and aligns horizontal and enterprise strategies with operational activities. 

As PMEC, the EOB aims to strengthen the planning, coordination and use of performance measurement and evaluation within the department, and to ensure appropriate resourcing of these functions.

The Policy on Results requires the PMEC to:

• Review and advise the deputy head on the establishment, maintenance and implementation of the Departmental Results Framework (DRF), Program Inventory (PI), and Performance Information Profiles (PIPs).
• Review and advise the deputy head on availability, quality, utility and use of performance information including evaluation.
• Review and advise the deputy head on departmental evaluation planning, resources and activities, including recommending approval of the departmental evaluation plan.

• Review evaluation reports and summaries, including management responses and action plans, and recommend their approval to the deputy head.
• Review and advise the deputy head on the neutral assessment of the evaluation function.
• Monitor follow-up on evaluation recommendations and action plans and advise the deputy head of any issues.

1.6 The Evaluation Function at SSC

The Office of Audit and Evaluation (OAE) is a key provider of reasonable assurance and advice on departmental operations to the President SSC.

Reporting directly to the President, the Chief Audit and Evaluation Executive (CAEE) makes recommendations on the relevance and performance of departmental programs, policies, and initiatives through the OAE evaluation function. The CAEE is also the designated departmental Head of Evaluation and represents the departmental link to TBS for evaluation.

The Office of Audit and Evaluation, under the CAEE, provides the following services:

The Evaluation Function at SSC - Long description

The Chief Audit and Evaluation Executive reports directly to the President and is the Head of Internal Audit, the Head of Evaluation, and the Senior Officer for Internal Disclosure. The Office of Audit and Evaluation contains Internal Audit Operations, Evaluation, and Professional Practices. Professional Practices contains the Internal and External Liaison, coordination for the Departmental Audit Committee, Quality Assurance, and Management Action Plan Follow-up.

SSC will continue to grow a capability to meet evolving needs. A fully resourced OAE evaluation function would be resourced for 16 full-time equivalent (FTE) positions and an overall annual budget of $1.91M. 

The focus in 2023-24 will be to continue planned staffing. At full capacity, the evaluation function will deliver three to four reports annually.

Resources for Fiscal Year 2023-24

Salary	Operations and Maintenance	Total
$ 1.62 M	$ 0.29 M	$ 1.91 M

Positions (as of May 2023)

	EX-01	AS-01	EC-07	EC-06	EC-05	EC-04	total
Establishment	1	1	2	4	4	4	16
Filled	1	0	2	3	4	1	11

2.0 Fiscal Year 2022-23 Accomplishments

2.1 Completed Evaluation

In 2022-23, OAE successfully completed the evaluation of SSC’s COVID-19 Response. The Evaluation of SSC’s COVID-19 Response assessed SSC’s responsiveness, effectiveness, efficiency, and sustainability of the actions taken by SSC in meeting the urgent demands of the COVID-19 pandemic. The evaluation generated recommendations on improving prioritization, performance measurement, efficiency, and partner communications.

2.2 Other Accomplishments

In 2022-23, The evaluation team:

• Provided advice for the Police Infrastructure Operations (PIO) Review; 
• Provided assessment and support for the revision process of the Performance Information Profiles (PIPs);
• Established guidelines for developing Management Response and Action Plans (MRAP) for evaluations;
• Supported the development of the Audit and Evaluation Resource Centre (AERC) which helps SSC branches effectively respond to audit and evaluation requirements.
• Developed the Cloud Evaluation Strategy which has been reviewed by the Performance Measurement and Evaluation Committee and approved by the President. 

3.0 Evaluations in Progress

OAE is presently working on four evaluations.

The Mobile Devices and Fixed Lines evaluation is examining the responsiveness, alignment with the enterprise approach, effectiveness and efficiency of the components of the Telecommunications Program, which is primarily responsible for providing and modernizing voice services in the GC workplace.

By assessing the Cloud program's alignment with major orientation documents such as SSC 3.0 and the Cloud Adoption Strategy, the Cloud evaluation will provide forward-looking and strategic evidence that would support SSC in:

• Planning or revising its services for the future of the Cloud Program;

• Clarifying the value-added of an Enterprise approach.

In the first quarter of 2023-24, OAE is launching the Modernization of Hosting Solutions evaluation. The Networks evaluation will launch in the third quarter.

Name of Evaluation in Progress	Target Completion Date
Mobile Devices and Fixed Lines	July 2023
Cloud	April 2024
Modernization of Hosting Solutions (Data Centres)	June 2024

4.0 Evaluation Planning

4.1 Key Planning Considerations

The selection and sequencing of the evaluations was driven by:

• The need to address all policy requirements
• The need to review significant spending that has never been evaluated
• Programs with key initiatives supporting SSC 3.0 strategic priorities 
• The maturity of Programs
• Prioritization of evaluations to inform decision-making on stewardship and progress towards an enterprise approach
• Customer Satisfaction Feedback Initiative (CSFI) scores
• Prioritization of evaluations to inform decision-making on funding renewals
• Prioritization of evaluations that focus on externally-facing outcomes supporting partner departments and clients
• Requests by branches to conduct advisory engagements

4.2 Consultation and Coordination

The Departmental Evaluation Plan is based on analysis and consultations conducted during the Winter of 2023. This included:

• Document review: The review of various sources of information to identify any mandatory evaluation requirements and assess coverage of the Program Inventory. Key documents included the last approved 5-Year Plan, Performance Information Profiles, Service Reviews, planned memoranda to Cabinet and Treasury Board submissions, etc.
• Consultations with Branches: All twelve SSC branches were consulted to review the proposed evaluations, and to confirm their usefulness and value.
• Coordination with Internal Audit. The plan is coordinated with the Risk-Based Audit Plan (RBAP) for Internal Audit engagements to reduce the burden placed on Branches due to oversight activities.
• Consideration of External Audits. The plan has taken into consideration external audits conducted by the Office of the Auditor General and the Office of the Comptroller General. 
• Consultation with the Treasury Board of Canada Secretariat. In accordance with the Policy on Results, the Head of Evaluation has consulted with TBS.

4.3 SSC Program Inventory and Components

SSC Program Inventory and Components - Long description

There are seven programs in SSC’s program inventory. Each program contains one or more components.

The Workplace Technologies program contains five components, including email services, digital communications and collaboration, workplace technology software provisioning, workplace technology hardware provisioning, and workplace technology services.

The Networks program contains four components, including LAN, WAN, satellite, and internet.

The Enterprise Services Design and Delivery program contains four components, including architecture, science program, service management, and partner engagement.

The Cloud program contains one component, which is cloud services.

The Data Centre Information Technology Operations program contains eight components, including enterprise operations, Linux/Unix, Windows and virtualization, integrated high performance computing, mainframe and bulk print management, platform services, storage backup and file services, and facilities management.

The Telecommunications program contains four components, including conferencing services, contact centre infrastructure services, toll free voice, and mobile devices and fixed line phones.

The Security program contains three components, including security management and governance, secret infrastructure, and infrastructure security.

5.0 Evaluation Coverage Overthe Next Five Years

5.1 Planned Coverage

The Policy on Results introduced the requirement for a Program Inventory, which contains the entire list of programs in the Department. The Program Inventory helps demonstrate how the Department delivers on its mandate, supports the authorization of expenditures through Cabinet, and supports performance reporting.

The Program Inventory represents the entire evaluation portfolio. The Policy on Results requires all programs and planned spending in the Program Inventory (except for internal services) to be evaluated periodically.

The Evaluation team plans to deliver 12 evaluations over the next five years, representing all seven programs in the SSC Program Inventory. This represents 100% of policy coverage requirements.

5.2 Advisory Engagement

The evaluation team will also conduct two advisory engagements over the next five years. The scope of these advisory engagements will be determined through consultations with SSC branches and senior management.

5.3 Programs Excluded from the Planned Coverage Calculation

At the program level, there are no excluded operational programs.

Some SSC programs are significant in size and will require more than one evaluation. As a result, the Plan reflects a calibrated approach based on value for decision-making. This means that some of the planned spending associated with specific components within programs may be evaluated in future years.

* Planned program spending is based on the SSC Departmental Plan 2023-24.

Appendix 1 – Five-Year Evaluation Delivery Schedule

Five-Year Evaluation Delivery Schedule - Long description

July 2023

Mobile Devices and Fixed Lines (Telecommunications)

April 2024

Cloud Services (Cloud) HSB

June 2024

Modernization of Hosting Solutions (Data Centre IT Operations) HSB

September 2024

Networks (Networks) NSSB

December 2024

Advisory Engagement #1 (TBD)

September 2025

Conferencing Services (Telecommunications)

December 2025

IT Service Management (Enterprise Services Design and Delivery)

December 2025

Security (Security) NSSB-CTOB

May 2026

Advisory Engagement #2 (TBD)

June 2026

Science Program (Enterprise Services Design and Delivery)

June 2027

Workplace Technology Provisioning and Services (Workplace Technology)

June 2027

Contact Centre Infrastructure Services (Telecommunications) DSB

December 2027

Digital Communications and Collaboration (Workplace Technology) DSB

January 2028

Partner Engagement (Enterprise Services Design and Delivery) CSDM

Appendix 2 – Five-Year Evaluation Schedule

Year1	Evaluation Name	Link to Program Inventory	Branch	Strategy2 to EOB (PMEC)	President Approval
2023-24	Mobile Devices and Fixed Lines	Telecommunications	DSB	-	Jul 23
2024-25	Cloud Services	Cloud	HSB	Feb 23	Apr 24
	Modernization of Hosting Solutions	Data Centre IT Operations	HSB	Jun 23	Jun 24
	Networks	Networks	NSSB	Oct 23	Sep 24
	Advisory Engagement #1	TBD	TBD	Jun 24	Dec 24
2025-26	Conferencing Services	Telecommunications	DSB	Aug 24	Sep 25
	IT Service Management	Enterprise Services Design and Delivery	OMB	Nov 24	Dec 25
	Security	Security	NSSB / CTOB	Nov 24	Dec 25
2026-27	Advisory Engagement #2	TBD	TBD	Nov 25	May 26
	Science Program	Enterprise Services Design and Delivery	CTOB	May 25	Jun 26
2027-28	Workplace Technology Provisioning and Services	Workplace Technologies	DSB	May 26	Jun 27
	Contact Centre Infrastructure Services	Telecommunications	DSB	May 26	Jun 27
	Digital Communications and Collaboration	Workplace Technologies	DSB	Nov 26	Dec 27
	Partner Engagement	Enterprise Services Design and Delivery	CSDM	Dec 26	Jan 28

Appendix 3 – Examples of Program Evaluation Questions

SSC evaluations will incorporate recent international trends in evaluation and best practices. They can also examine alignment with government priorities and Enterprise 3.0.

• Relevance is the extent to which a program addresses and is responsive to a demonstrable need. For SSC, the focus will be on the extent to which the objectives and design of a program respond to the needs of partner departments and continues to do so as circumstances change.
  1. To what extent has the program responded to changing client needs and priorities?
  2. Is the program aligned with emerging technology trends and best practices?
• Alignment is a component of relevance and it assesses the linkages between objectives and government priorities.
  3. To what extent has the program streamlined and moved towards an enterprise approach?
  4. Is there an effective process in place to guide SSC prioritization of work with clients?
• Effectiveness examines the impacts of a program and the extent to which it is achieving its expected outcomes. The focus is on immediate and intermediate outcomes, i.e. the “so what”?
  5. Has the program achieved the desired outcomes (e.g. reducing risk, achieving outcomes for clients, and other benefits from modernization)?
  6. Are there any unexpected outcomes (desired or not)?
• Efficiency is the assessment of resource utilization in relation to the production of outputs and outcomes.
  7. What are the opportunities to improve efficiency?
  8. What are the innovative practices that have been implemented by SSC to modernize programs?

Some engagements may include support from Internal Audit for lines of evidence that focus on risk:

• Is there a good alignment and understanding of the enterprise risks in the SSC program?
• What is the progress achieved by SSC in reducing risks associated with the program?

Appendix 4 – List of Acronyms

Acronym	Description
CTOB	Chief Technology Officer Branch
HSB	Hosting Services Branch
DSB	Digital Services Branch
CSDM	Client Service Delivery and Management
CAEE	Chief Audit and Evaluation Executive
EOB	Executive Oversight Board
FY	Fiscal Year
GBA+	Gender-Based Analysis plus
IT	Information Technology
NSSB	Networks and Security Services Branch
PIO	Policing Infrastructure Operations
PMEC	Performance Measurement and Evaluation Committee
OAE	Office of Audit and Evaluation
OMB	Operations Management Branch
SSC	Shared Services Canada
TBD	To be determined