1. Introduction

1.1 Overview

Public Service Commission mandate

1. The mandate of the Public Service Commission is to promote and safeguard merit-based appointments and, in collaboration with other stakeholders, protect the non-partisan nature of the public service. Through collaboration with departments and agencies, it is dedicated to building a public service that is based on excellence and is representative of Canada’s diversity. It safeguards non-partisanship and promotes and protects merit and the use of both official languages in staffing and recruitment. It also supports departments and agencies in recruiting talented people from coast to coast to coast through the use of innovative and modern services, tools and practices.

The Public Service Resourcing System

2. The Public Service Recruitment System (PSRS) facilitates the recruitment process for federal public service advertised positions, and is the principal tool by which the Public Service Commission manages staffing and related activities under the Public Service Employment Act. The publicly accessible front end, GC Jobs, is where job opportunities are posted and visible to job seekers. The back end of the system supports job postings, job searches, applying for a job, applicant screening and referral, and notifications such as the notification of appointment. Public Service Commission officials use data that is captured in PSRS to fulfil reporting obligations to Parliament.

3. System development work began in 2001 under stream 1 of the Public Service Staffing Modernization Project. The project was established to support the implementation of National Area of Selection, the coming into force of the Public Service Modernization Act, and changes to the Public Service Employment Act that allowed for greater delegation of authority for staffing from the Public Service Commission to deputy heads. PSRS was introduced in 2003 as an external staffing system prototype that was piloted in 2 regions. In 2005, the system was deployed to other regions as an interim government-wide solution and then to other organizations in 2006.

4. The Public Service Commission is allocated funds to maintain and operate PSRS through an annual reference level update transfer from departments and agencies that operate under the Public Service Employment Act. The funding envelope is fenced and cannot be used for any other purpose outside of the operation, maintenance and improvement of PSRS. The total funding allocation for 2019–20 is $6,768,258.

The evolution of the Public Service Resourcing System

5. As a result of amendments to legislation, policies, directives and business processes that underpin the administration and use of the system, changes have been implemented as required. Exhibit 1 highlights important changes that were made to the system between 2009 and 2018.

Exhibit 1 – Important changes made to the Public Service Resourcing System by year, 2009–18
Year Important changes made to Public Service Resourcing System by year
2009
  • Introduction of the "case file" management feature for executive processes that required a custom workflow management process, reports, and advertisement management functions.
  • Introduction of the unsupervised internet test feature that required significant modifications to the "Post a job" & "Apply for a job" use cases, as well as interaction with a new application developed by a third-party development team (online internet test).
2010
  • Introduction of a new registration process for the job seeker accounts — validated email addresses, password hashing method replacement and various other tweaks.
  • Introduction of a new data export schema built in collaboration with Publiservice so that referral data from PSRS & Publiservice could be easily amalgamated.
  • Introduction of additional unsupervised internet test feature options.
  • Overhaul of Federal Student Work Experience Program area of selection criteria.
2011
  • Introduction of the "bundle" feature, used to group multiple staffing processes and perform communication and assessments in a streamlined manner.
  • Introduction of the recent work log feature allowing administrators to quickly navigate between their different staffing processes within the application.
  • Introduction of the administrator email/note feature, allowing communiqués to be sent to other administrators via the application (email and in-app messages supported).
2012
  • Introduction of the "Trustee" ownership model for staffing processes.
  • Introduction of the Statement of Merit Criteria structure for building an advertisement.
  • Introduction of the "News box" feature for publishing notifications to applicants.
2013
  • Introduction of the Public Service Entrance Exam unsupervised internet tests and supervised tests.
  • Removal of all the scoring functions, replaced with an external service (developed by a third-party Public Service Commission scoring web service team).
  • Introduction of the unsupervised internet test and supervised test "by invitation" feature.
2014
  • Introduction of internal staffing, staffing notifications and an email alert feature.
  • Introduction of Veteran's Hiring Act features to enable the legislation — including data exchange with the Priority Information Management System, Department of National Defence and the Canadian Armed Forces.
2015
  • Conversion of the PSRS backend from ISO-8859-1 to UTF-8 character sets.
  • Introduction of features necessary to support the Nunavut Land Claim Agreement legislation (data capture, candidate ranking, export functions).
  • Application conversion to enable the use of the Email Transformation Initiative.
2016
  • Replacement of all Federal Student Work Experience Program functions in PSRS with a new application, Student Recruitment Service.
2017
  • Modifications to enable duty-to-accommodate policy changes.

6. Within this context, the data provided to the assessment team for the period from 2009 to 2018 demonstrates that the number of change requests created by the Information Technology Services Directorate (ITSD) during software development has been declining since 2011. In addition, the average time to deploy change requests has been relatively stable since 2015 (see Exhibit 2).

Exhibit 2 – Change requests per year and average number of days to deploy
Year Change requests implemented Average deployment time per request (calendar days)
2009 304 32
2010 258 26
2011 347 25
2012 308 19
2013 221 15.25
2014 378 16
2015 119 11.6
2016 78 13.7
2017 41 11.7
2017 27 10.25

7. The only deviation from the trend was in 2014 and the beginning of 2015, when the Public Service Commission was preparing to implement the provisions established by the Veterans Hiring Act.

8. According to the Public Service Commission’s 2015–16 Annual Report, in order to implement the Veterans Hiring Act, it “improved the job application system by merging the website used to advertise internal job postings with the external jobs.gc.ca site (previously only used for job opportunities open to the public). This provided all applicants, including veterans and current Canadian Armed Forces members, with direct electronic access to all internal and external job opportunities from any location. Veterans are able to use their jobs.gc.ca account to highlight their status to hiring managers and invoke the veterans preference or mobility provision.”

Public Service Resourcing System sustainability factors

9. First implemented using state-of-the art Java technology, the system has undergone multiple modifications and upgrades to incorporate legislative and policy changes, as well as emerging business requirements. As well, technology and development methodology have evolved, and as a result the system now has considerable technical debt. According to ITSD, technical debt refers to any and all stakeholder dissatisfaction with an application from a non-functional perspective. Such dissatisfaction may stem from the high operational cost of an application, to the cost and slow pace of delivery of business changes to the application, through to applications failing to keep up with security standards. It should be noted that technical debt exists to some extent in all software systems. When a system is operated with technical debt concerns, changes can still be made, however the resources required may be disproportionate to the benefits in certain cases. This said, it must be noted that not all changes require the same level of effort and have the same impact on the system.

10. In terms of current system use statistics, on an average business day, PSRS sends out approximately 83 000 email alerts and facilitates the posting of 51 job advertisements. In 2017–18, 7 000 job advertisements were posted, producing over half a million job applications. While there is no impending urgent requirement for the immediate replacement of PSRS in fiscal year 2019–20, many factors suggest that the system is nearing the end of its useful life. The most recent draft options analysis (still in development as of June 4, 2019) document being prepared for Phase 0 of the GC Jobs Transformation project stated, “Despite the effort put in the system’s maintenance since its deployment and the remarkable adaptability of PSRS to changing business requirements, the end of the system is within sight.”

11. Since the system was launched in 2003, alternative solutions have been discussed. Up until 2015, the approaches taken were to enhance rather than replace the system. In 2016–17, the Internal Audit and Evaluation Directorate provided a consolidated risk assessment of maintaining PSRS and the Priority Information Management System. While this consulting engagement did not test controls nor issue recommendations, results were reported to Public Service Commission senior management for consideration and action as required. More recently, the September 2018 concept case for the GC Jobs Transformation project that was submitted to the Treasury Board of Canada Secretariat stated, “A recent analysis of PSRS confirmed that the system can no longer be improved to meet user needs and expectations.”

1.2 GC Jobs Transformation

12. The Public Service Commission has made transforming the recruitment platform a priority. Work has started and consultations have occurred with 3 key user types — job seekers, hiring managers and human resources professionals. The transformation of recruitment across the federal public service is planned to take place iteratively, through a phased approach. Public Service Commission officials will test and pilot various solutions to make sure the solution meets user needs before it is deployed to departments and agencies. Roll-out of the solution would follow consultations with the Treasury Board Secretariat.

Phase 0: Pre-project (August 2018 – March 2020)

13. The pre-project discovery phase in 2018–19 entailed collecting information from job seekers, hiring managers, HR professionals and stakeholders about their needs, and gathering information from industry about what currently exists in the private sector in terms of recruitment solutions.

14. In 2019–20 an options analysis will be completed to determine the best way forward, whether it is to buy, build or a combination of both. In addition, proof of concept testing with be conducted with selected departments and agencies to assess the feasibility of commercially available solutions.

Phase 1: Pilot deployment (2020–21)

15. In this next phase of the project, live pilots will be conducted with selected departments and agencies to gather user feedback on the solution, validate assumptions and inform further configuration efforts.

Phase 2: Implementation (2021–22 and beyond)

16. This is the final phase of the project in which GC-wide implementation begins, including final configuration work pursuant to findings from pilot evaluations. The project will then transition to product management, soliciting feedback on the new recruitment solution to inform enhancements and continuously improve the user experience.

1.3 Objective

17. The engagement objective was to assess PSRS change management governance and related processes, and review information on the effort to maintain the system in light of sustainability until the GC Jobs Transformation solution is in place.

1.4 Scope

18. The scope encompasses the current PSRS operating environment, including the processes that govern proposed changes to the system and their implementation, as well as data on changes that have occurred over the past decade.

19. The scope does not include work that would allow the assessment team to provide estimates on the life expectancy of the system. Since it has been determined that PSRS will be replaced, the focus was on the change management processes in place to maintain the current system, in light of sustainability concerns, until the GC Jobs Transformation solution can be implemented.

1.5 Methodology

20. The assessment was conducted from April to June 2019. The approach consisted of a combination of interviews, examination of relevant documentation, and testing of processes. Evidence was gathered through the techniques listed below.

21. Interviews and discussions. Several interviews were conducted with key Public Service Commission officials. These included members of the Information Technology Services Directorate (ITSD), Business Development and Systems Directorate (BDSD), Policy and Communications Sector, and Human Resources Management Directorate. Individuals interviewed included officers, team leaders, managers, directors, process owners and members of the senior management team.

22. Review of documentation and analysis. Relevant documentation such as policies, procedures, process documentation, guidance materials, briefing decks, records of decisions and lessons learned were analyzed to gain a thorough understanding of the current operation of the system. Information was directly obtained from key informants, and statistics were obtained from Public Service Commission officials, including data provided by Service Canada. An analysis was performed of changes made to PSRS based on the statistics obtained from ITSD and BDSD that are from tools such as GitLabFootnote 1 and BugzillaFootnote 2.

23. Review testing. Sample testing was performed to review the change management processes in place for each type of release.

24. The opportunities for improvement identified were validated with management prior to finalization of the report.

Statement of conformance

25. In our professional judgement, sufficient and appropriate review procedures were performed and appropriate evidence was gathered to support the accuracy of the conclusions contained in this report. The conclusions were based on observations and analyses at the time of our review. The conclusions regarding the current operation of PSRS are only applicable for the period under review. The evidence was gathered in accordance with the Treasury Board Policy on Internal Audit and the Institute of Internal Auditors International Standards for the Professional of Internal Auditing.

2. Observations

2.1 System administration and management

26. This section focuses on ongoing system administration and management. The objective was to review how issues are identified, managed and resolved. The processes in place to perform ongoing monitoring of the system were also reviewed.

GC Jobs and Public Service Resourcing System administration

27. PSRS Business Operations provide a multi-level user support service to its 2 main clients — job seekers and PSRS administrators.

Job seekers (includes internal and external applicants)

  • Tier 1 – Service Canada Call Center
  • Tier 2 – PSRS Helpdesk (BDSD/Bus Ops)
  • Tier 3 – Program Analysts (BDSD/Bus Ops)

Public Service Resourcing System administrators

  • Tier 1 – Departmental super users
  • Tier 2 – PSRS Helpdesk (BDSD/Bus Ops)
  • Tier 3 – Program Analysts (BDSD/Bus Ops)

28. The 3-tiered model provides direct support to user and applicant enquiries with specific processes in place to manage them. This approach ensures user needs are addressed in a systematic manner. Enquiries are either addressed or escalated to the next tier for further analysis, action and response. The processes involve interactions with multiple stakeholders including Service Canada, BDSD and ITSD.

Tier 1

29. A memorandum of understanding is in place between the Public Service Commission and Service Canada’s Digital Service Directorate to administer Tier 1 services. The memorandum has a value of $224,300 and is in effect from April 1, 2019, to March 31, 2020. Roles and responsibilities, service delivery standards and a staffing plan to support this service are included in the memorandum. The Quality Assurance Monitoring Program for phone and email inquiries, business continuity considerations and privacy of information precautionary measures are outlined as well.

30. Tier 1 enquiries are broken down as follows.

  • Job seeker enquiries are managed by Service Canada. Basic enquires include resetting passwords, applicant account creation, how to apply and generic system-related questions.
  • PSRS administrator enquiries are managed by organizational super users. Basic enquires include account creation, system training and responses to user questions.

31. Enquiries are either resolved directly or escalated as required.

Tier 2

32. Tier 2 enquiries are more complex or technical in nature. Service Canada / organizational super users escalate these to the PSRS support team for response and resolution.

Tier 3

33. Tier 3 enquiries must be escalated due to the complexity, sensitivity and/or technical nature of the issues. This is the final line of support. Program analysts will either resolve the enquiry, or request IT support from the recruitment application support team within ITSD to resolve a system related issue. Based on the scope or sensitivity of the issue, senior management will also be consulted.

34. Exhibit 3 provides an overview of tier 2 and tier 3 enquiries in fiscal year 2018–19.

Exhibit 3 – Volume of Public Service Resourcing System related calls to the PSRS Help Desk, 2018–19
Month Tier 2 Tier 3
April-2018 531 16
May-2018 570 10
June-2018 588 41
July-2018 574 22
August-2018 515 11
September-2018 565 16
October-2018 587 24
November-2018 584 15
December-2018 548 8
January-2019 664 1
February-2019 499 9
March-2019 580 13
Source : Service Canada 2018–19 report provided to BDSD

35. Enquiries are recorded and tracked for volume measurement and identified by top task. These allow for further analysis in order to consider system enhancements and/or improve user support. If a system change or bug is identified, it is entered in Bugzilla and then goes through the change request process for consideration.

36. Exhibit 4 provides insights on the top tier 3 tasks related to PSRS in 2018–19.

Exhibit 4 – Top tier 3 tasks created in 2018–19
Top tier 3 tasks Number
Other administrator tasks 120
Create a job poster 20
Give trustee access 17
Search for applicants 13
Manage selection process 13
Apply for a job 9
Refer applicants 4
Create an applicant account 1
Create an administrator account 1
Sign up for e-mail alerts 1
Create notification 1
Source : Service Canada 2018–19 report provided to BDSD

Ongoing monitoring of the Public Service Resourcing System

37. The ITSD recruitment application support team monitors the system in 2 ways — through daily monitoring activities and by performing occasional real-time monitoring.

38. Daily monitoring. The team monitors PSRS to ensure it is operating normally. Automated processes and system checks are performed at various times of day to ensure that web pages are functioning normally. Team members review application logs, server logs and automated alerts. These logs contain detailed information about any user requests or scheduled jobs that have not been completed successfully. When concerns are identified, corrective actions are taken. These actions may include developing data correction scripts that are tested and then released to the system. These activities are recorded in GitLab, which provides an audit trail of the actions.

39. Real-time monitoring. When real-time issues or incidents (such as unexpected slowdowns occur), the recruitment application support team liaises with the ITSD database analyst and middleware group colleagues, and at times with external stakeholders, such as Shared Services Canada officials. These types of communications are tracked in a corporate workload management tool called RemedyFootnote 3. The system is actively monitored to identify the cause and potential solutions to address the issue.

Managing changes in the Public Service Resourcing System


Exhibit 5 – An overview of the change request process Footnote 4

Exhibit 5 – An overview of the change request process

Change request process- overview

Hear

Analyse

Prioritize

Implement

Evaluate

Request received

Triage request

Analyse request

Design solution

Implement solution

Request closed

Evaluate change request process

Process improvement implemented

Hear and analyze

40. As with any system, there are minor and major changes that need to be made to PSRS. The categorization depends on whether requested changes are cosmetic, typos, formatting, or defects that users come across as they operate the system. Changes may also be required to enhance the operation of the system or to address new legislation, policies or business process improvements.

41. Change proposals are initiated through change request forms. Users submit this form to BDSD officials, who review and analyze the information to determine whether all the necessary details are captured. This is a part of the triage process. If more information is requested from users, notes are entered and tracked in Bugzilla.

Prioritize

42. BDSD team members analyze requests and develop an initial prioritization based on established criteria (see Exhibit 6). The requests are then reviewed in the monthly meeting of the Change Request Round Table (or sometimes ad-hoc) for prioritization. ITSD and BDSD officials discuss the requests and review them in light of system risks and opportunities.

Exhibit 6 – Severity levels used in analyzing proposed changes to the Public Service Resourcing System

Severity

Description

Process

Critical

  • Legislative compliance requirement.
  • High risk of inaction (potential complaint regarding official languages, accommodations, human rights)
  • Escalate to Change Control Board (weekly BDSD management meeting) for approval
  • Schedule for emergency release

High

  • Major issue/high user impact with no feasible workaround
  • Monthly round table to communicate/confirm severity
  • Prioritize amongst other change requests for seasonal (quarterly) release
  • Obtain Change Control Board management approval for seasonal release

Medium

  • Major issue/high user impact that has a workaround
  • Monthly round table to communicate/confirm severity
  • Defer to GC Jobs Transformation

Low

  • Minor issue/medium to lower impact
  • Monthly round table to communicate/confirm severity
  • Defer to GC Jobs Transformation

43. While the Change Request Round Table terms of reference have not been formally updated to reflect the new change request process in place since the Fall 2018, its revised mandate is to:

  • consult, inform and share information
  • identify opportunities and risks
  • validate and identify issues for resolution
  • recommend the list of prioritized change request for management approval

The chair is the Change Request Manager, who reports to the Director, Project Management Office. The membership is made up of:

  • the Project Management Office manager and a subject-matter expert if needed
  • a director or manager from Central Programs and Regional Offices for Student Application Renewal Project and Post-secondary recruitment, and a subject-matter expert, if needed
  • an IT representative (manager), including quality assurance team lead if needed
  • a business operation manager
  • a change request project manager

44. At Change Request Round Table meetings, change requests are challenged and prioritized. Key factors considered include user impact (the number of users affected by the change), the risk of not implementing the change to operations, the complexity of the change and the level of effort to make the change. When a request is rejected, the BDSD team member communicates with the change request requestor to inform them of the decision and the rationale.

Implement and evaluate

45. After change requests are prioritized at Change Request Round Table, the list is presented to the Change Control Board (which is part of the BDSD Management Meeting) for approval to proceed. Following this, the ITSD development team implements the required coding changes. In certain cases, change requests may not be prioritized for immediate action, for instance, where an issue can be resolved by using a workaround solution, where the impact of the change is deemed to be low, or where the level of effort is deemed to be high.

46. The ITSD quality assurance team then tests the code provided by the development team from a functional and an accessibility point of view. The process is iterative so that bugs and issues with the code are fixed as needed. The quality assurance test results are documented in Bugzilla or an email is sent to the development team to notify them that the code is working as intended. The quality assurance team confirms the test results with the development team, who then inform BDSD of the results.

47. User acceptance testing is conducted, and if there are any concerns, the development team fixes the code, which is re-tested by the quality assurance team. After testing, the development team informs the ITSD’s Change Advisory Board that the changes are ready for implementation in PSRS, and the new code is deployed into the code base.

48. The BDSD team that administers the system reviews processes and procedures on a regular basis to ensure that the processes are operating as efficiently as possible. The team also assesses how issues and change requests have been administered with a view to adjusting certain aspects of the process as required.

49. Exhibit 7 outlines the 3-phased approach to managing approved changes.

Exhibit 7 – Management of approved changes to Public Service Resourcing System

Exhibit 7 – Management of approved changes to Public Service Resourcing

Management of approved changes to Public Service Resourcing System

Development Team

Quality Assurance Team

User Acceptance Testing (UAT)

Writes the coding and scripts to make changes to PSRS per the change request

Completes the necessary functional testing and accessibility testing in PSRS.

Document the quality testing results to confirm the coding changes are operating as intended.

Typically consist of the end users of the business stakeholders

They test the changes made by the development team, after the coding and scripts have passed the quality assurance team’s tests.

The UAT tests are documented where the users approve the changes.

Case study – application of the Veterans Hiring Act lessons learned

50. The Public Service Commission officials responsible for administering the system are focused on continuous improvement and ensuring that the processes and system meet current needs as the GC Jobs Transformation project is underway. As mentioned earlier, the implementation of the Veterans Hiring Act required major changes to the system. At the end of the project that was managed to make the required system changes, a lessons learned document was produced. Based on our process walkthroughs, the BDSD team demonstrated how they amended several processes to manage and implement changes to PSRS as a result of the Veterans Hiring Act lessons learned document. A case study is provided in Exhibit 8.

Testing of the change request process

51. The Information Systems Audit and Control Association’s Control Objectives for Information and Related Technologies 5 is the overarching business and management framework for governance and management of enterprise IT. It highlights that organizations should follow a robust change management process to maintain a reliable IT system. This allows for only authorized changes that are thoroughly tested to be implemented. Following established management of IT change processes enables consistent operations of a system, iterative change tracking and recording, prioritization of resources to allow for changes that are aligned with business objectives and outcomes, and alignment with security and compliance requirements as applicable.

52. Changes to PSRS are released at various points in time. Approved changes can be released either as:

  • a scheduled seasonal release, which happens twice a year
  • a weekly scheduled release, which typically happens on Thursdays
  • an emergency release, which can happen at any time

53. The assessment team conducted sample testing on 3 different types of PSRS releases that occurred in fiscal year 2018–19. These were:

  • Release 17 (Emergency Release), maintenance 4 deployment on May 4, 2018 – Bug 6249 – Infotel extract files missing information since PSRS Winter release 2018 (Release 17)
Exhibit 8 – Veterans Hiring Act lessons learned on how Public Service Resourcing System changes are managed

The review team studied the project closeout and transition report from the Veterans Hiring Act project. Although a detailed follow-up on actions to address lessons learned was not a part of this review, it is worth noting some of the progress that was made in addressing how changes to PSRS are managed.

  • The overall IT change management process has been revised over the past 3 years to integrate project management practices as part of PSRS change processes. These include:
    • elements of project initiation (where the change request is made)
    • project planning (where the changes are triaged, analyzed and prioritized)
    • project execution (where the changes are developed and tested)
    • project monitoring and control (where the changes are released)
    • project closure
  • The processes for triaging PSRS change requests and gathering information help clarify the business need, as well as how critical the request is. They also help identify if there is a work-around that could decrease the demands on change process resources. As the Public Service Commission moves forward both with PSRS and GC Jobs Transformation, it will be important to develop established criteria to prioritize changes and to involve governance committees to provide oversight through discussions involving ITSD and BDSD in an open transparent manner.
  • The process for managing IT changes requires the quality assurance testing to be completed before starting the user acceptance testing, which reduces duplication of effort since any errors in the code and scripts can be corrected by ITSD before the business stakeholders are involved in testing activities.
  • Release 17 (Routine Weekly Release), maintenance 10 from October 25, 2018, was selected – Bug 6278 – New notification advertisement template following policy review
  • PSRS Release 18 (Seasonal Release), Maintenance window dated March 28, 2019 – Bug 6356 – Lifecycle for alternate recruitment ads and Issue 60 – Content modification to inventory paragraph in advertisement template

54. The sample testing was performed to review the change management processes for each type of release. In general, system releases were well managed, and the BDSD team worked with ITSD and key stakeholders throughout the process. The assessment team noted 2 opportunities for improvement that were important to highlight given the significance of having well-established, documented processes in place to manage system changes.

Documentation

55. Documentation of approvals. Based on our review of the 3 releases and overall process walkthrough, the assessment found that the documentation of decisions could be improved. Specifically, the notes that are currently maintained in Bugzilla do not provide sufficient information on approval decisions. Maintaining documentation of approvals is important as the Public Service Commission moves towards implementing the GC Jobs Transformation solution. Currently, various decisions are documented in a number of systems — such as Bugzilla, GitLab, Remedy — and through emails and records of management meetings. Essentially, having a central repository where decisions are maintained, the rationale for rejecting some requests and the related communication feedback loop to the requestor / business is important from corporate knowledge and governance perspectives.

56. Documenting business rules that govern functional and accessibility testing. The assessment found that for instances where the quality assurance team was not involved in functional and accessibility testing prior to the deployment of a proposed change to PSRS, the rationale and mitigating steps were unclear. While quality assurance team involvement is not always required in the change process, documenting the reasons why they were not included is important to ensure that changes and the involvement of stakeholders are handled consistently.

57. The assessment also found that user acceptance testing results were not always documented in Bugzilla. For this reason, in a certain number of cases, there is no documented evidence to support whether tests were successfully completed prior to releasing changes into production. Given the importance of having strong corporate knowledge of how the system operates, having proper documentation related to specialized (custom and tailored) systems such as PSRS is of critical importance.

58. Management consideration 1. The Vice-President, Services and Business Development Sector, should consider establishing a structure to maintain complete documentation of PSRS change decisions made by relevant governance committees. In addition, user acceptance testing and quality assurance results should be consistently entered into the appropriate monitoring tool.

Internal governance

59. Strengthening internal governance. While ITSD have a key role in supporting the on-going operation of PSRS through system monitoring and managing change requests, including tier 3 issues, there is no overarching document that outlines the key roles, responsibilities, accountabilities and performance expectations for BDSD, ITSD and key internal stakeholders. The assessment team was informed that in 2014 a service level agreement type document had been drafted outlining the key roles and responsibilities of various stakeholders, and that it was out of date and no longer in use. The assessment team was unable to determine whether the draft service level agreement had ever been signed.

60. Given IT involvement in system maintenance and operations, a document that provided for the transfer of budget authority in the amount of $1.5 million for PSRS IT resources was approved and came into effect on April 1, 2017. This represents approximately 30% of the overall PSRS allocation provided in the annual reference level update process. The document specifies that ITSD will ensure that all relevant Computer Systems resources are allocated to PSRS activities, and identifies a requirement to report progress on a quarterly basis to BDSD and its Vice-President. In addition, funding is coded to ensure appropriate tracking of expenditures and forecasts of this portion of the PSRS fenced funding; and access to financial information will be made available to BDSD for reporting to the PSRS Interdepartmental Steering Committee. Finally, under this agreement, BDSD continues to be accountable and responsible for the overall planning of PSRS priorities and forecasted annual expenditures; it also prepares the overall spending forecast for the fiscal year, and includes ITSD planned PSRS expenditures, before the beginning of each fiscal year.

61. Recently, a new business process was established for the way in which Central Programs and Regional Offices (CPRO), Services and Business Development Sector informed their BDSD colleagues of system issues and change requests. In the past CPRO staff from across the country submitted issues and change requests directly to BDSD for assessment and processing. A new control was put in place whereby all CPRO staff that identify PSRS issues or change requests should submit them to the Director, Central Recruitment Programs. These issues / requests would be assessed by CPRO and prioritized before being submitted to BDSD for consideration and processing.

62. The assessment team heard from key business stakeholders within the Public Service Commission that their roles and responsibilities could be clearer with respect to their participation in the change request review and implementation process, and that decisions could be better communicated to them. In some instances it was raised that there are impacts that the triage process has on them. For example, if a change request can be addressed by a work around, with consideration to its feasibility, then it is not a priority for BDSD. However, this impacts the work that business owners have to perform to manage the issue.

63. Management consideration 2. The Vice-President, Services and Business Development Sector should consider putting in place a formal service level agreement with the Vice-President, Corporate Affairs Sector, that clarifies and expands on the roles, responsibilities and reporting responsibilities outlined in the 2017 briefing note that re-distributed the funds to support PSRS.

64. It should be noted that the 2 management considerations are proposed in light of the current GC Jobs Transformation project, and are aimed at strengthening internal processes that will have to be in place regardless of the solution that is implemented.

Updating Public Service Resourcing System as a result of legislative, policy or direction changes

65. As the Public Service Commission will continue to use PSRS over the short to medium term as per the GC Jobs Transformation Project timelines, the assessment team obtained information from BDSD and ITSD concerning what would be involved in making a change to PSRS to address current policy directions with respect to gender identification.

66. A May 16, 2019, Executive Management Committee decision made the collection of gender information in PSRS optional. Prior to this decision, the ITSD development team and quality assurance team estimated the level of effort that would be required for a number of options to make a change to PSRS gender fields. They concluded that the level of effort for the approach that was selected would be minimal.

67. Furthermore, the ITSD recruitment application development team developed an estimated level of effort based on the potential number of operations that would have to occur to implement this decision. The factors considered include the amount of code that would need to be written, modified or removed, the volume of data elements that would need to be amended, and the level of complexity of the tasks.

68. A project concept document was developed to assess whether there was sufficient ITSD capacity to implement the changes to PSRS in fiscal year 2019–20. Preliminary level of effort estimates were developed and approved in April 2019. The estimated ITSD cost of implementing the change is $58,948.33 and the estimated business cost is $100,290.47, for a total estimated cost of $159,238.80.

69. The assessment team noted that the concept document decision did not contain an approval from BDSD. However, in discussions with BDSD officials, it was noted that a member of their team was on the working group discussing the proposed changes and costs, and that this change will go through the regular change request process described above. When the change request goes through the established governance, it will be reviewed and cost estimates will be further refined before official approval will be provided by BDSD representatives.

70. This example highlights the fact that even though the system change to implement the policy decision to make gender information optional for users can be viewed as an essential change to address a policy direction, the amount of effort required is not as high as for other more complex system changes that require significant development, coding and quality assurance work.

3. Conclusion and key considerations

71. In conclusion, the Public Service Commission has made maintaining PSRS a priority while work is underway on the GC Jobs Transformation Project. This includes the need for BDSD to work in collaboration with Service Canada, ITSD officials, policy and business process owners, and system users to ensure that the system continues to serve the needs of human resources officials, hiring managers and job seekers.

72. A very important governance element is the way in which BDSD works with ITSD to support and maintain the system. The teams have been focused on continuous improvement and finding ways to efficiently operate the system, while at the same time putting in place measures to triage change requests and implement higher priority changes based on established criteria. Also, the Public Service Commission has demonstrated a strong capacity to be innovative and make major changes to this legacy system when required. This was demonstrated by the work that was performed to implement the provisions established by the Veterans Hiring Act on July 1, 2015.

73. There are a number of change management process elements in place that are working as intended. As the system ages, processes are continuously assessed to efficiently manage changes in an operating environment that is focused on sustainability until a new solution can be implemented. These include the identification of 4 levels of criticality, focusing on higher-level priority changes, and adopting lessons learned from projects to ensure that resources and levels of effort are managed appropriately.

74. There were 2 opportunities for improvement identified in this report presented as management considerations. The first focuses on strengthening the documentation of PSRS change request decisions and the consistent documentation of user acceptance testing and quality assurance results in the appropriate tools. The second focuses on documenting key roles, responsibilities and performance expectations with respect to the shared approach used to administer and operate PSRS.

75. Management consideration 1. The Vice-President, Services and Business Development Sector should consider establishing a structure to maintain complete documentation of PSRS change decisions made by relevant governance committees. In addition, user acceptance testing and quality assurance results should be consistently entered into the appropriate monitoring tool.

76. Management action plan. The Services and Business Development Sector will review current change processes, documentation and records to ensure to capture decision points clearly. It will work with the Corporate Affairs Sector to ensure results are captured consistently in an appropriate tool. This may require onboarding of a new tool such as JIRA. The actions will be completed by September 30, 2019.

77. Management consideration 2. The Vice-President, Services and Business Development Sector, should consider putting in place a formal service level agreement with the Vice-President of the Corporate Affairs Sector that clarifies and expands on the roles, responsibilities and reporting responsibilities outlined in the 2017 briefing note that re-distributed the funds to support PSRS.

78. Management action plan. The Services and Business Development Sector will work in collaboration with ITSD to review the 2017 document and develop a service-level agreement that expands upon and clarifies roles, responsibilities and reporting responsibilities on the ongoing maintenance and operation of PSRS. The actions will be completed by December 31, 2019.

79. The management considerations are proposed in light of the current GC Jobs Transformation project and are aimed at strengthening internal processes that will have to be in place regardless of the solution selected.

Page details

Date modified: