Evaluation of the Human Resources Services Modernization Initiative

Executive summary

The evaluation of the Human Resources Services Modernization (HRSM) initiative was conducted between August 2015 and March 2016 by the Treasury Board of Canada Secretariat (the Secretariat). The evaluation assessed the relevance and performance, including efficiency, of the four components of the initiative from the 2011 to 2012 fiscal year to the 2013 to 2014 fiscal year.

Conclusions

  • The evaluation found that some departments,Footnote 1 particularly small ones, did not have the necessary financial or human resources to fully implement the requirements of the HRSM initiative in a timely manner.
  • The evaluation found that the HRSM initiative was not monitored after implementation. Post‑implementation monitoring would have allowed for course corrections and reinforcement of the importance of the initiative among departments to avert the use of “shadow” HR systems, where old processes continued to operate in parallel with the new processes.
  • The evaluation found that the components of the initiative’s governance operated in silos and that the interaction among the HRSM components was poorly understood. This resulted in insufficient clarity of the nature and extent of the involvement by participants in the HRSM initiative and uneven buy-in.
  • The evaluation found that the Secretariat should have been involved earlier in its role as business owner and that the short time frame for implementation was a challenge for departments. Once the Secretariat’s role was clarified, the Secretariat’s support was appropriate and appreciated, particularly in the case of the Common Human Resource Business Process component, whose supportive and guiding role during implementation was praised by departments. Some departments would have been open to more direction from the Secretariat.

Recommendations

The recommendations are addressed to the management of the Secretariat’s Back Office Transformation Initiative, the area responsible for the ongoing work of PeopleSoft v9.1 Joint Build and Human Resources Information System. These elements have since evolved to My GCHR and the Interoperability Standard.

  • To effectively implement transformation initiatives, the Secretariat should allocate financial and human resources not only to design and implementation, but also to ongoing support and monitoring in order to assess the extent to which the new processes and systems are meeting the needs of departments and operating as intended. The Secretariat should also clarify which department has the responsibility for support and monitoring.
  • The Secretariat should communicate expectations of departments (for example, implementation timelines, the degree and nature of involvement by participants, and the expected outcomes of the initiative) early in the life cycle of a transformation initiative.
  • The Secretariat should ensure that government-wide initiatives have governance and oversight that engage all functional areas at all levels of the governance chain to minimize silos.

Introduction

This document is the final report on the evaluation of the Human Resources Services Modernization (HRSM) initiative. This evaluation is part of the Treasury Board of Canada Secretariat’s (the Secretariat’s) approved five-year evaluation plan for the 2015 to 2016 fiscal year and was conducted by the Internal Audit and Evaluation Bureau with the assistance of Goss Gilroy Inc.

Background to the initiative

Launched in the 2011 to 2012 fiscal year, the HRSM initiative was to modernize human resources (HR) services in departmentsFootnote 2 to support one process, one system and one data set for the Government of Canada (GC) and to reduce the number of HR systems, leading to efficiencies for transactional HR activities.

The HRSM initiative was founded on the following guiding principles:

  • improve efficiency and quality of HR service delivery and reduce the HR footprint and costs
  • focus on horizontal collaborative efforts between HR and Chief Information Officer information systems and technology, and other internal services to ensure alignment of HR business requirements with HR systems
  • identify priorities for HR systems on a service‑wide basis
  • converge HR services planning and development under an integrated approach

A total of $22.4 million, which includes partner contributions and existing funds for the Office of the Chief Human Resources Officer (OCHRO), was spent over three years (from the 2010 to 2011 fiscal year to the 2013 to 2014 fiscal year) to support streamlined and standardized HR processes and to reduce the costs of maintaining outdated information technology systems. The initiative was coordinated centrally by OCHRO ($849,583) under the HRSM Secretariat function.

The HRSM initiative consists of the following components:

  1. Implementing a Common Human Resources Business Process (CHRBP) to foster an enterprise‑wide approach to HR management practices, led by the Secretariat and OCHRO ($3.7 million).
  2. Undertaking the Human Resources Business Solutions Pilot (HRBSP) to build the Management System Standard in PeopleSoft version v8.9, led by the Secretariat and the Chief Information Officer Branch (CIOB) ($8.3 million).

    After the HRBSP, the PeopleSoft v9.1 Joint Build project was undertaken to establish a standard Government of Canada (GC) configuration of PeopleSoft v9.1. This standard was completed by a team consisting of the Secretariat‑OCHRO, the Secretariat-CIOB, Employment and Social Development Canada, and Public Services and Procurement Canada, and governed by the Joint Build Assistant Deputy Minister Steering Committee, co-chaired by OCHRO and Employment and Social Development Canada ($5.1 million).

  3. Replacing the Human Resources Information System (HRIS) with a single configuration of PeopleSoft for small departments, led by Public Services and Procurement Canada ($3.8 million).
  4. Introducing the enterprise-wide Human Resources Data Interoperability Standard (HRDIS) to design the framework for a common approach among HR systems and to permit a seamless exchange of data, led by the Secretariat and CIOB ($584,557).

Governance

A central governance structure was established for the HRSM initiative in the 2011 to 2012 fiscal year. The structure was to drive an enterprise-wide HR strategic vision and HR service delivery model.Footnote 3 The four components were designed to work in an integrated manner under an overarching governance structure coordinated by the HRSM Secretariat at OCHRO. The HRSM Assistant Deputy Minister Advisory Board oversaw the implementation of the initiative and set the direction for future investments in HR management processes and systems. This Advisory Board was co-chaired by the Assistant Deputy Minister, Governance, Planning and Policy Sector, and by the Chief Information Officer.

Evolution

As the initiatives were implemented, two important changes were made.

First, the PeopleSoft v9.1 Joint Build project was merged with the HRIS so that rather than having one configuration for large and medium-sized departments and another for small departments, all departments would implement the modules of the PeopleSoft v9.1 Joint Build.

Second, because of a delay in the start date for the HRDIS, the funding earmarked for the project’s second year (which would have been obtained through project approval) was removed from the fiscal framework and was no longer available. The HRDIS project was terminated in March 2013, following completion of the interoperability framework and architecture design, and the standard was not developed.

It is important to note that the output of the HRBSP and the PeopleSoft v9.1 Joint Build was to be the creation of the single, standard HR system, known as My GCHR (PeopleSoft v9.1), for the Government of Canada. The implementation of the My GCHR project falls under the umbrella of the Back Office Transformation Initiative, led by the Secretariat and approved by the Public Service Management Advisory Committee in December 2014. The implementation of My GCHR by departments from the 2014 to 2015 fiscal year to the 2018 to 2019 fiscal year is outside the scope of this evaluation.

Expected outcomes

The expected outcomes of the HRSM initiative, as shown in its logic model (see Appendix B), are as follows:

Immediate

  • implementation of the common HR business processes
  • initial departmental adoption of GC standard
  • better alignment of HR systems investments to GC‑wide business priorities
  • GC interoperability standard ready for development
  • contribution to one set of data

Intermediate

  • consistent, more efficient and effective HR service delivery across the GC
  • lower GC-wide total cost of HR management system ownership
  • improved data exchange, information interoperability capability and capacity across the GC back office

Long-term

  • transformation of HR management to enable achievement of the GC’s strategic goals and more robust capacity to manage HR

The logic model focuses on the accountabilities and responsibilities of the Secretariat and does not include the range of accountabilities and responsibilities for all government departments.

Evaluation context

The evaluation of the HRSM in the 2015 to 2016 fiscal year was required in order to report to the Public Service Management Advisory Committee on the progress made to achieve the early outcomes of the initiative.

The evaluation examined the first phase of the HRSM initiative, which included the groundwork to establish the initiative, and took into account the scope, maturity and timing of each of the initiative’s components. The evaluation measured only the extent to which the immediate and intermediate outcomes were achieved. The extent to which the long-term outcome was achieved was not measured, since the 2015 to 2016 fiscal year evaluation is a mid-term assessment of progress toward outcomes. The evaluation examined the funds allocated from the 2011 to 2012 fiscal year to the 2013 to 2014 fiscal year.

Box A: core evaluation issues

Relevance

  • Issue 1: continued need for program
  • Issue 2: alignment with government priorities
  • Issue 3: alignment with federal roles and responsibilities

Performance (effectiveness, efficiency and economy)

  • Issue 4: achievement of expected outcomes
  • Issue 5: demonstration of efficiency and economy

By component, the scope included the following:

  • CHRBP: The timeline for this component was the 2012 to 2013 fiscal year to the 2013 to 2014 fiscal year. However, the evaluation examined impacts of this component through to March 31, 2015.
  • HRBSP: The timeline for the initial deliverables was the 2012 to 2013 fiscal year to the 2013 to 2014 fiscal year. However, the evaluation used information up to March 31, 2015, to examine progress beyond the core timeline.
  • PeopleSoft v9.1 Joint Build and HRIS: The timeline for the initial deliverables was the 2012 to 2013 fiscal year to the 2013 to 2014 fiscal year. However, the evaluation used information up to March 31, 2015, to examine progress beyond the core timeline.
  • Data interoperability framework: Work was conducted only in the 2012 to 2013 fiscal year. The extent to which the framework was being used up to March 31, 2015, was examined.

The evaluation excluded the investments and activities related to the implementation of My GCHR and Back Office Transformation but gathered information from the initial pilot of departments to measure progress on the intermediate outcomes related to the four components of the HRSM initiative. Thus, although My GCHR was outside the scope of this evaluation, the initial activities, outputs and outcomes of the first group of departments to implement the initiative (known as “Wave 0”) were considered to allow the evaluation to draw conclusions about HRBSP and HRIS activities.

The evaluation approach is consistent with the Treasury Board Policy on Results, the Mandatory Procedures for Evaluation and the Standard on Evaluation. The evaluation looked at the five core issues identified in the Policy on Evaluation (Box A), but calibrated the scope to put more weight on the assessment of immediate and intermediate outcomes, given that the initiative is still under implementation.

Methodology

The evaluation used the following lines of evidence:

  • administrative data: reviews of administrative data and documents were undertaken
  • key informant interviews: in total, 51 interviews were conducted: 10 addressed the initiative overall; 29 addressed the CHRBP component; 4 addressed the HRBSP component; 7 addressed the PeopleSoft v9.1 Joint Build and HRIS component; and 1 addressed the data interoperability component
  • case studies: two case studies were conducted for each component, with the exception of data interoperability

See Appendix A for a detailed explanation of the methodology, including how qualitative evidence is presented in this report.

Limitation of the evaluation

Box B: process modelling hierarchy

Level 1: value chain

  • a connected series of HR processes
  • describes the highest-level processing that makes up an organization (for example, human resources)

Level 2: process

  • cross-functional, cross-border series of interrelated sub-processes
  • transform information from one form to another to achieve a business outcome
  • represents the decomposition of a value chain (for example, talent management)

Level 3: sub-process

  • a further definition of the processes identified in the sub-process
  • each box at this level represents a sub-process that can be broken down into tasks (for example, identify staffing/recruitment needs)

Level 4: task

  • a discreet unit of work performed by an individual
  • lowest level of work that can be portrayed in a process

Level 5: step

  • the detail of how the task is performed is outlined at this level
  • the information can be used to support the development of training material

Source: OCHRO, CHRBP

One limitation occurred in the document review. Evaluators encountered challenges in the amount of information within the sources to address some of the performance indicators. Administrative data were available only for the CHRBP component. In addition, despite edits to the CHRBP performance indicators in the 2014 to 2015 fiscal year to correct for data gaps, other data gaps limited the evaluator’s ability to compare departments. As well, detailed CHRBP data were collected only for the 20 largest departments rather than for all 84 participating departments, as directed in project approvals. Consequently, detailed analysis was limited to these 20 departments.

Nonetheless, these 20 departments represented the greatest use of resources (approximately 80%), and data for these were complete and comprehensive. Thus, evaluators were able extract reliable information from these sources and mitigate this limitation.

Relevance

Conclusion: The modernization of HR services remains relevant because there is a continuing need to enhance HR efficiencies and service delivery for Canadians. As the predecessor of My GCHR, PeopleSoft v9.1 in the HRSM initiative is consistent with the roles, responsibilities and priorities of the Government of Canada and the Secretariat.

Continuing need

The lines of evidence confirm a continuing need for HR modernization within the Government of Canada. My GCHR has taken over where the PeopleSoft v9.1 Joint Build and HRIS replacement left off. The impetus for this continuing need is that until all departments adopt a common system, HR information will continue to be too fragmented and expensive to maintain, and may not meet the needs of decision makers for timely, accurate and accessible information based on a single set of data. The diversity of platforms makes it difficult to gather enterprise-wide data for Canadians and to achieve efficiencies within the public sector.

There is also a continuing need to ensure that HR processes are consistent across the Government of Canada in order to achieve a single GC process and resultant efficiencies. This would in turn facilitate the implementation of other GC initiatives. The implementation of a common system will help increase the consistency of processes at a more detailed level of operations (for example, Level 5 of the process modelling hierarchy in Box B).

Finally, there is a continuing need to ensure that enterprise-wide systems are interoperable to improve the sharing of data and information across the Government of Canada and to facilitate the gathering of enterprise‑wide data for Canadians.

Alignment with priorities, roles and responsibilities

Interviews and documents confirm that the HRSM initiative is aligned with GC priorities, namely, developing GC-wide solutions that modernize the way the federal government delivers services to Canadians while generating operational savings. Both the Speech from the Throne and the Budget speech from 2016 underscore the importance of transforming the government’s back office systems in support of a better government for Canadians. Similarly, Blueprint 2020 calls for a whole-of-government approach to enhance service delivery and value for money.

The HRSM initiative is also aligned with the Secretariat’s strategic outcome related to good governance and sound stewardship to enable efficient and effective service to Canadians. The Secretariat’s Report on Plans and Priorities for the 2015 to 2016 fiscal year stated that the modernization of the government’s HR management is an ongoing priority and that the Secretariat “will continue to advance this work in the year ahead.” Interviewees felt strongly that the federal government is the most appropriate entity to oversee and implement a transformation initiative of this nature. This opinion‑based evidence, combined with the documentary evidence pertaining to the Secretariat’s mandate to support a more effective and efficient government-wide approach to people management, suggests that the HRSM initiative is aligned with federal roles and responsibilities.

Initiative performance

Effectiveness

Conclusion: The HRSM initiative was implemented in participating departments by March 31, 2015. Various factors hampered the achievement of some immediate and intermediate outcomes. For example, despite the plan to implement the CHRBP standard to Level 4 of the process modelling hierarchy, some key informants indicated that implementation at Level 5 could have introduced a greater degree of consistency in processes. Implementing to Level 5 would require a system to be in place.

As well, a few departments indicated that they were encouraged to address only gaps and manageable opportunities, thus resulting in some inconsistencies in business processes across departments. A few departments admitted that they were still using pre‑CHRBP processes, further reinforcing the finding that HR business processes are inconsistent at Level 4. Furthermore, the limited progress in the interoperability standard for HR prevented the achievement of the development of a single data set. Finally, the evaluation found that the effective delivery of HR services across the Government of Canada has improved, but was not fully achieved since some functionality at Level 4 of PeopleSoft v9.1 was not available after implementation.

Achievement of immediate outcomes

Implementation of CHRBP

The review of administrative data and documents confirmed that the CHRBP was implemented in the participating departments. The first step in the implementation of the CHRBP was for these departments to identify the gaps and opportunities associated with implementing the new processes. Participating departments identified 179 gaps and 538 opportunities across the seven CHRBP processes. By March 31, 2014, all departments had fully addressed these gaps and opportunities to Level 4. However, a few interviewees and evidence from one of two CHRBP case studies revealed that departments were encouraged to identify gaps and manageable opportunities in process areas where they felt they could implement the changes by the deadline. Thus, a number of potential opportunities were not pursued in favour of achieving the deadline. As well, according to some interviewees, common HR business processes would only be achieved when implemented at Level 5.

Finally, a few respondents said that they did not feel that all of the CHRBP has in fact been fully implemented. These respondents indicated that their departments continued to use processes previously in place.

Initial adoption of GC standard

As described above, the initial pilot of the GC standard involved a cluster of five departments implementing PeopleSoft v8.9 (Agriculture and Agri-Food Canada, the Canadian Food Inspection Agency, Fisheries and Oceans Canada, Health Canada and the Public Health Agency of Canada). According to all lines of evidence, the five departments successfully implemented PeopleSoft v8.9. The evidence from the document review and interviews suggests this initial adoption achieved its expected outcomes and encountered few challenges.

Under the HRSM initiative, four departments (Library and Archives Canada, Canadian Institutes of Health Research, Public Services and Procurement Canada, and Employment and Social Development Canada) were to have implemented PeopleSoft v9.1 by March 31, 2015. The evaluation found that while the basic system (PeopleSoft v9.1) was implemented, functionalities such as the reporting module would only become operational in the future under the My GCHR initiative.

Contribution to better alignment of HR systems investments to GC-wide priorities

All lines of evidence found that by moving to a single configuration of PeopleSoft v9.1 and to the CHRBP, the HRSM initiative has begun to align HR systems investments for those departments that participated in the HRSM initiative to modernize back-office systems. The My GCHR initiative, through the adoption of PeopleSoft v9.1 by additional departments, continues to reinforce the alignment of HR systems investments with GC-wide priorities.

According to key informants and case studies, although the HRBSP component sought to implement PeopleSoft v8.9 rather than v9.1, HRBSP nevertheless played an important role in aligning HR systems investments for participating departments and demonstrated that moving to a common HR system was possible.

Interoperability and contribution to one set of data

The document review and interviews with key informants confirmed that an interoperability frameworkFootnote 4 was developed under the HRSM initiative. Because a second phase of the interoperability component was not funded, the GC standardFootnote 5 for interoperability was not developed as planned. Subsequently, Employment and Social Development Canada used the framework to build a prototype.Footnote 6 Key informants indicated that the main impact of not proceeding with the component was a delay in the achievement of one intermediate outcome: “one set of reliable data and improved data exchange across the GC back office.”

Outside the context of the HRSM initiative, according to documents and key informants, those at the Secretariat responsible for interoperability worked with Employment and Social Development Canada to implement an architecture and solution platformFootnote 7 based on the interoperability framework. This platform is described in documentary evidence as “repeatable,” and it is anticipated that this work can be “lifted and shifted … to serve the entire Government of Canada.”Footnote 8

With one system (PeopleSoft v9.1), the ability to have one reliable set of data is possible. However, without interoperability fully realized and the absence of a reporting module in the system as part of the HRSM initiative, one reliable set of data had not been achieved.

Achievement of intermediate outcomesFootnote 9

Contribution to more consistent efficient and effective HR service delivery across the GC

The reviews of documents and administrative data showed some progress toward more consistent HR service delivery across pilot departments because of the CHRBP and an increase in self-service functionality and automation of some routine HR transactions. Examples identified to evaluators included a new centralized HR structure that holds HR staff accountable, the completion of the outstanding signatures of directors general on organizational charts, documentation of grievances and employee awards in employee files, and standardized tools.

According to documents and comparisons between baseline and performance data, improvements have been recorded in the satisfaction of managers with the quality and timeliness of HR advice, in HR processing times, in service standards for staffing and classification, and in the assignment of competencies to work descriptions.

Furthermore, as evidenced in the Management Accountability Frameworks for the 20 largest departments, departments reported wide-ranging benefits from CHRBP implementation, including improvements in consistency, efficiency, effectiveness, technology, engagement, clarity of roles, service delivery and governance.

Nevertheless, concern was voiced by a few respondents about the sustainability of the HRSM components within departments, owing to lack of monitoring after implementation. The evaluation found evidenceFootnote 10 among some interviewees that their departments had not fully implemented the new CHRBP standard or PeopleSoft v9.1 and, of these, a few have continued to operate parallel or “shadow” HR shops to offer direct support to managers or HR-related information systems. According to these key informants, shadow HR shops exist because not all departments buy into or understand the new allocation of accountabilities (for example, between HR professionals and managers) required under the CHRBP and because of issues with the new software either not generating reports or because of software incompatibility. These few departments indicated that they would continue to run parallel systems until PeopleSoft v9.1 is fully functional and their confidence in the new system increases.

Contribution to lower GC-wide total cost of HR management system ownership

It is too early in the HRSM initiative for evaluators to fully assess the achievement of the long-term outcome, given that departments are still adopting PeopleSoft v9.1Footnote 11 (My GCHR) and that the interoperability component was not developed. Consequently, estimates of lower GC-wide costs of HR management system ownership were not examined.

However, evaluators tried to identify instances where the initiative contributed to lower GC-wide costs of HR management system ownership in the document review, which identified some savings through cost avoidance. Although interviewees also confirmed avoidances, there is limited documentary evidence that costs have in fact dropped. The document review of project documentation noted that possible cost avoidances might occur under the HRBSP and PeopleSoft v9.1 Joint Build because departments would no longer undergo individual system upgrades and because the central and standard system would manage upgrades across the enterprise.

Key informants generally believed that the ownership costs of HR management systems are lower as only one licence is now required and there are fewer older systems requiring costly maintenance. However, there were no data on ownership costs to help evaluators determine the contribution to cost reduction. Notwithstanding the observations of the key informants described above and evidence of possible cost avoidance in documents, the use of parallel systems by a few departments would delay expected cost savings and, in some case, increase HR costs, particularly for departments operating two systems.

Improved data exchange, information, interoperability capability and capacity across the GC back offices

As discussed in the “Evolution” section, evaluators were unable to assess the achievement of this outcome because this HRSM activity ended after funding earmarked for the second year of the project was removed from the fiscal framework. As a result, the project was terminated in 2013. Since work on the interoperability standard has resumed within the Secretariat, this activity could be examined as part of a future evaluation of My GCHR.Footnote 12

Efficiency and governance

Conclusion: Although the GC-wide cost of HR management system ownership is expected to be lower, the evaluation was not able to find concrete evidence that this has occurred.

Regarding the governance and delivery of the initiative, the Secretariat did a number of things well that led to success. These included taking a supportive, guiding and community-building role for the CHRBP component, engaging and encouraging interdepartmental collaboration, and accessing funds to support the Secretariat’s guidance role.

The evaluation also identified a number of areas where the Secretariat’s role hindered success, including not having a clear and defined role as a business owner when implementing the system, causing departments to become confused about who was involved in the decision‑making process; not having a central oversight of the governance of the initiative, but rather applying what some key informants called a siloed approach to governance by component; not taking an active role early enough; not providing departments enough time for implementation of the PeopleSoft v9.1 Joint Build and CHRBP; not providing enough support (for example, tools and guidance) to small departments early in the process; and not providing GC-wide funding to departments.

Efficiency

As noted earlier, actual costs savings were not determined through the evaluation since it is too early in the initiative’s life cycle to identify them. However, the evaluation attempted to learn which delivery approaches might contribute to decreased costs. These approaches included the Secretariat’s role in early planning and support to departments, the willingness of departments to invest in the process, and collaboration between departments and the Secretariat.

To improve the efficiency of the Secretariat’s management of government-wide transformation initiatives, the evaluation evidence identified alternatives and improvements. Earlier and better consideration of the context (for example, size and structure) and capacity of small departments would have resulted in the development and dissemination of tools and guidance for small departments that could have improved the efficiency of the implementation of the CHRBP standard as well as PeopleSoft v9.1 in pilot departments. Consideration of context would also have included recognition that participation in complex transformation initiatives represents a financial burden for small departments:

  • efficiencies would have been gained if opportunities to learn from other departments had not been limited
  • more, earlier and better involvement, communication, coordination and tools provided by the Secretariat would have facilitated the implementation by departments, particularly of the CHRBP standard, where some departments did not fully understand the standard or the expectations of their participationFootnote 13
  • less siloed GC-wide governance and business models and clearer mandates would have resulted in more efficient use of time at meetings, but also in more efficient use of time between meetings

Governance and delivery

Key informants’ views on the governance approach for the HRBSP component were positive overall, though the governance of the other components and the overall initiative were viewed as uneven and feedback was mixed.

The document review and interview evidence revealed that the Secretariat was at times unclear about how it set direction as the business owner. Moreover, evaluators observed that there was no central oversight to coordinate all HRSM components. Rather, each component was governed and operated independently. In the view of the evaluators, this led to fewer opportunities to adapt to the changes within components. For example, better coordination could have helped communicate the impacts of the termination of interoperability funding, and processes could have been standardized to a lower level of detail to facilitate system implementation.

For the PeopleSoft v9.1 Joint Build component, some interview respondents indicated that governance was ineffective. Some key informants and case study participants identified tensions among decision makers that necessitated the escalation of issues to higher levels of management. This caused delays and created operational risks in the delivery of the component. Regarding clarity of roles and responsibilities, there were mixed views, with some respondents indicating they were clear and a few stating that information on who was doing what was confusing. The department’s team primarily dealt with Public Services and Procurement Canada, and the Secretariat was never involved in the decision‑making process. The governance structure for the PeopleSoft v9.1 Joint Build had to be adapted along the way in order to clarify roles and responsibilities.

Regarding the interoperability component, interviewees familiar with it felt that the approach to governance was siloed and that there was poor understanding of the component, resulting in ineffective project oversight.

CHRBP interdepartmental engagement received more positive feedback, particularly for the clarity of roles and responsibilities and structure. A few respondents felt it was ineffective in terms of decision making and influence. In terms of central governance, some respondents felt that the CHRBP’s centrally led governance structure was not relevant for them because their departments did not participate in it. A few respondents explained that the central governance did not address their departments’ needs for support in the implementation of the standard.

Lessons learned

Beyond the efficiency-related findings, evaluators identified the following additional evidence:

  • Large-scale GC-wide transformations should establish a governance approach whereby consensus is encouraged and delays are avoided by having a single authority for decisions.
  • Governance and other project characteristics of GC‑wide transformation initiatives should recognize cross-functional roles and how the initiative plays a role in, and is influenced by, larger initiatives.
  • Interdepartmental collaboration and partnerships, sharing and support should be encouraged early and should be supported by the Secretariat through the sharing of tools, including IT platforms, and meetings.
  • The timing and degree of implementation of the initiative should consider a department’s capacity to implement it. Where capacity is limited and additional funding or support is not available, allowances should be made early in the process to manage expectations.
  • GC-wide transformation initiatives should be monitored after implementation to determine whether outcomes are achieved, to identify where interventions are required to achieve outcomes, and to measure cost savings and other efficiencies. Post-implementation monitoring would also reinforce the importance of the initiative among departments, and if the monitoring reveals cost savings have been achieved, departments will be more likely to buy into subsequent GC-wide transformation initiatives.

Conclusions and recommendations

Conclusions

  • The evaluation found that some departments, particularly small ones, did not have the necessary financial or human resources to fully implement the HRSM initiative requirements in a timely manner.
  • The evaluation found that the HRSM initiative was not monitored after implementation. Post‑implementation monitoring would have allowed for course corrections and reinforcement of the importance of the initiative among departments to avert the use of “shadow” HR systems, where old processes continued to operate in parallel with the new processes.
  • The evaluation found that the components of the initiative’s governance operated in silos and that the interaction among the HRSM components was poorly understood. This resulted in insufficient clarity of the nature and extent of the involvement by participants in the HRSM initiative and uneven buy-in.
  • The evaluation found that the Secretariat should have been involved earlier in its role as business owner and that the short time frame for implementation was a challenge for departments. Once the Secretariat’s role was clarified, the Secretariat’s support was appropriate and appreciated, particularly in the case of the Common Human Resources Business Process component, whose supportive and guiding role during implementation was praised by departments. Some departments would have been open to more direction from the Secretariat.

Recommendations

The recommendations are addressed to the management of the Secretariat’s Back Office Transformation Initiative, the area responsible for the ongoing work of PeopleSoft v9.1 Joint Build and Human Resources Information System. These elements have since evolved to My GCHR and the interoperability standard.

  • To effectively implement transformation initiatives, the Secretariat should allocate financial and human resources, not only to design and implementation, but also to ongoing support and monitoring in order to assess the extent to which the new processes and systems are meeting the needs of departments and operating as intended. The Secretariat should also clarify which department has the responsibility for support and monitoring.
  • The Secretariat should communicate expectations of departments (for example, implementation timelines, the degree and nature of involvement of participants, and the initiative’s expected outcomes) early in the life cycle of a transformation initiative.
  • The Secretariat should ensure that government-wide initiatives have governance and oversight that engage all functional areas at all levels of the governance chain to minimize silos.

Appendix A: methodology approach

The evaluation was guided by an approved evaluation framework, which was a detailed plan of the evaluation activities, questions and indicators. Most of the evaluation indicators were based on the indicators, and the related data collection methods documented in the HRSM Performance Measurement Strategy report.

Evaluation questions

The following questions were addressed by the evaluation:

Relevance

  1. Does the HRSM initiative address a continuing need?
  2. To what extent are the objectives of the HRSM initiative aligned with government priorities and the Secretariat’s departmental strategic outcome?
  3. To what extent is the HRSM initiative aligned with federal roles and responsibilities for HRSM within the federal government?

Performance (effectiveness, efficiency and economy)

  1. To what extent has the HRSM central governance contributed to more coordinated implementation and transformation of the HR service delivery across the GC?
  2. To what extent have departmentsFootnote 14 implemented the Common HR Business Process?
  3. To what extent has the initial adoption of the GC standard (PeopleSoft) occurred?
  4. To what extent has the HRSM initiative contributed to better alignment of HR systems investments to GC‑wide priorities?
  5. To what extent has the HRSM initiative contributed to one reliable set of data?
  6. To what extent has the HRSM initiative contributed to consistent, more efficient and effective HR service delivery across the GC?
  7. To what extent has the HRSM initiative contributed to lower GC-wide total costs of HR management system ownership?
  8. What progress was made towards the development of the GC interoperability framework? What are the implications to the achievement of outcomes from the GC interoperability component not proceeding as planned?
  9. Are there more effective solutions to meeting the needs addressed by the HRSM initiative?
  10. What factors have contributed to the achievement of the expected outcomes of the HRSM initiative? What, if any, have been the main barriers to achieving these outcomes?
  11. Was the delivery of the HRSM initiative efficient?

Methodology techniques

Document and data review

The review of documents and administrative data provided evidence for most evaluation questions. Documents were an important source of information to answer evaluation questions related to relevance and performance. Key documents and data for the review included project charters, close-out reports, Management Accountability Framework (MAF) reports, progress reports and monthly departmental reports, reports to Treasury Board ministers, Employment and Social Development Canada’s implementation schedule and progress reporting, HRIS progress reports and monthly reporting (dashboard), project status reports, lessons learned documents, HRSM governance decisions, meeting minutes, project documentation, onboarding schedules, lessons learned documents, close-out and evaluation reports, and program and baseline performance data. The review was conducted using a review grid to gather the information for each source and each evaluation question. The draft template was pre-tested by the primary reviewer, and all documents were catalogued.

For the administrative data review, in order to incorporate administrative information into this report, all baseline and performance measurement data (submitted in the form of individual Excel databases) were collated. The end result was one database with baseline submission data and performance measurement data side by side for each of the 20 largest departments. Further, an analysis of an online Access database (CHRBP tracking tool) was completed to track gaps and opportunities identified by departments. From this consolidated database key findings were drawn to help inform the evaluation questions.

Key informant interviews

Key informant interviews are a qualitative method used in evaluation. They gather views and factual information from key informants selected from within the federal government and the core public administration. For the evaluation of the Human Resources Services Modernization (HRSM) initiative, the key informant interviews were used to address all 14 evaluation questions in the evaluation framework.

Many different sources were approached to participate in the key informant interviews, including those familiar with the HRSM initiative overall and those familiar with only one or two components of the initiative. From an overarching perspective, members of the Human Resources (HR) Services Modernization Advisory Committee were consulted, as well as members of the HR Services Modernization Secretariat of the Office of the Chief Human Resources Officer (OCHRO). As well, a sample of representatives from various departments that participated in one or more HRSM initiative components was consulted. Where appropriate, some Secretariat and Public Services and Procurement Canada (PSPC) respondents were consulted about their department’s implementation of the HRSM components.

For the Common Human Resources Business Process (CHRBP) component, a sample of the 84 departments that adopted the CHRBP was developed with input from OCHRO. Selection criteria for departments included knowledge of the CHRBP activities, outputs and outcomes at their department, and whether the department had a positive or challenging experience. Selection of departments using these criteria was based on recommendations from OCHRO. Respondents were stratified by size of department (large, medium and small).

Of the 55 interviews planned, 51 interviews were conducted. The number in the “Departments” column (Table 1) is not necessarily the number of departments where interviews were conducted but rather the number of individual interviews conducted with departmental representatives, as in some cases more than one interview was conducted for some departments. Some respondents were able to speak to the evaluation questions from multiple perspectives (for example, many of the Secretariat’s representatives could speak from an overall perspective as well as about a specific component, and many departments were involved in both the CHRBP and the PeopleSoft v9.1 Joint Build project or the Human Resources Information System (HRIS)). The numbers in Table 1 indicate the component for which respondents were specifically identified, except for the Secretariat’s respondents, who had a broader perspective than expected, in which case they were included in the “overall” respondent type. However, when respondents provided evidence for other components, the analysis took their views into account.

Departments were invited to organize group interviews if they preferred. However, each interview of this type would still be weighted as one interview regardless of the number of departments and the number of interviewees in each interview group. The numbers in Table 1 are the number of interviews and not the number of interviewees.

The actual number of respondents who shared a particular opinion is not presented in this report. This is because in most cases, there was a small total number on which to base the findings, and thus the presentation of a number of respondents would be misleading. In many cases, owing to variations in the respondents’ familiarity with the subject matter and their resultant ability to answer some of the question, the total number providing evidence for an evaluation question could be quite low; sometimes only two or three people shared an opinion.

Table 1: key informant interviews conducted, by type
Respondent type by component Secretariat/PSPC Department
Overall: Advisory Committee 4 1
Overall: OCHRO 5 N/A
CHRBP: Small department N/A N/A
CHRBP: Medium-sized department N/A 9
CHRBP: Large department N/A 10
CHRBP: Total 1 28
HRSBP N/A 4
Joint Build (v9.1) or HRIS 2 5
Interoperability 1 0
Total 13 38

Instead, a qualitative scale was used to depict the proportion of those who answered a question and who shared an opinion:

  • Large majority: findings reflect the views and opinions of at least 75% of respondents
  • Most: findings reflect the views and opinions of at least 50% but less than 75% of respondents
  • Half: findings reflect the views and opinions of 50% of respondents (“almost half” and “just over half” are also used when one less or additional respondent shares an opinion)
  • Some: findings reflect the views and opinions of at least 25% but less than 50% of respondents
  • A few: findings reflect the views and opinions of at least two respondents but less than 25% of respondents

Case studies

The case studies were intended to provide a source of information on lessons learned that could be used in subsequent HR modernization projects.

Six case studies were conducted, two for each of three components of the HRSM initiative (CHRBP, Human Resources Business Solutions Pilot (HRBSP) and PeopleSoft v9.1 Joint Build, and HRIS). The unit of analysis for each case study was the department, and the case study focused on the implementation, outputs and outcomes associated with that component. Other evaluation questions related to relevance and governance were also explored, again with the focus on the component at each department.

Each case study included a review of the documents and administrative data pertaining to the department and as well as the implementation and outcomes of the component for which they were selected. As well, interviews were conducted with up to two individuals, who had knowledge of the component and its implementation and outcomes, from each department.

Appendix B: logic model

(Source: HR Modernization Performance Measurement Initiative Framework, January 2014)

logic model. Text version below:
Appendix B: logic model - Text version

Appendix B consists of a flowchart of the logic model used for the Human Resources Services Modernization Initiative.

The flowchart has seven horizontal bars that represent the seven main components of the Human Resources Services Modernization Initiative:

  1. initiatives
  2. activities
  3. outputs
  4. immediate outcomes
  5. intermediate outcomes
  6. long-term outcome
  7. TBS strategic outcome

Each of these seven components relates to five initiatives:

  1. central governance
  2. the Common Human Resources Business Process
  3. the Human Resources Solutions pilot project (the PeopleSoft version 9.1 Joint Build and Human Resources Information System)
  4. the Human Resources Information System replacement
  5. the enterprise-wide human resources data interoperability standard

Under the initiative of central governance:

  • activities include establishing and coordinating a human resources strategic vision, action plan and delivery model
  • outputs include direction, strategies, monitoring tools and annual reports, and integrated and coordinated Human Resources Services Modernization Initiative work streams

Under the Common Human Resources Business Process initiative:

  • activities include overseeing and enabling the horizontal implementation across the Government of Canada of the Common Human Resources Business Process standard
  • outputs include implementation strategies and tools, reporting on departments’ implementation progress, guidance and support for implementation, and an adapted Common Human Resources Business Process that reflects the legislative and business environment

Under the Human Resources Solutions pilot project initiative:

  • activities include expanding the Agriculture and Agri-Food Canada configuration of PeopleSoft version 8.9 to a cluster of five departments, and evolution of the PeopleSoft version 8.9 standard configuration to version 9.1
  • outputs include implementing the Agriculture and Agri-Food Canada cluster configuration and the standard configuration of PeopleSoft version 9.1 for Government of Canada adoption

Under the Human Resources Information System replacement initiative:

  • activities include replacing the legacy system with a single configuration of the Government of Canada PeopleSoft standard for 32 small departments and agencies
  • outputs include a single instance of PeopleSoft version 9.1 that is ready for adoption by 32 small departments and agencies, and pilot projects in two organizations

Under the enterprise-wide human resources data interoperability standard initiative:

  • activities include incorporating the blueprint for the human resources data interoperability standard into the development of the Government of Canada interoperability framework, and validating interoperability prototypes by proof of concept
  • outputs include the Government of Canada interoperability framework, and the interoperability solution architecture

Immediate outcomes of all five initiatives will be as follows:

  • implementation of the Common Human Resources Business Process
  • initial departmental adoption of the Government of Canada standard
  • better alignment of human resources system investments to business priorities across the Government of Canada
  • the Government of Canada interoperability standard is ready for development
  • contribution to one set of data

Intermediate outcomes of all five initiatives will be as follows:

  • consistent, more efficient and effective human resources service delivery across the Government of Canada
  • lower total cost across the Government of Canada for human resources management system ownership
  • improved data exchange, information interoperability capability, and capacity across the Government of Canada back office (this outcome depends on funding for implementation of the interoperability tool (enterprise service bus))

The long-term outcome of all five initiatives will be the transformation of human resources management to enable achievement of the Government of Canada’s strategic goals and a more robust capacity to manage HR.

The TBS strategic outcome of all five initiatives will be that government is well managed and accountable, and resources are allocated to achieve results.

Appendix C: Management Response and Action Plan

The recommendations are addressed to the management of the Secretariat’s Back Office Transformation (BOT) Initiative as the area responsible for the ongoing enterprise work on the PeopleSoft version (v.) 9.1 Joint Build and Human Resources Information System. The Enterprise Program Management Office (EPMO), in its role as business owner of BOT on behalf of the Secretariat and in coordination with the Chief Information Officer Branch (CIOB) and the Governance, Planning and Policy (GPP) Sector, agrees with the evaluation’s recommendations. EPMO, in consultation with CIOB and GPP, proposes the following action plan to address the recommendations below:

Recommendation 1

To effectively implement transformation initiatives, the Secretariat should allocate financial and human resources, not just to the design and implementation, but to ongoing support and monitoring in order to assess the extent to which the new processes and systems are meeting the needs of departments and agencies and operating as intended. The Secretariat should also clarify which organization has this responsibility.

Proposed action Start date Targeted completion date Office of primary interest
Action item 1: A multi-year business case will inform multi-year funding for transformation initiatives that include Financial, Human Resources and Information Management. The business case covers the requirements for design and implementation, as well as ongoing support and monitoring. Quarter (Q) 1, 2016 to 2017 fiscal year Q3, 2016 to 2017 fiscal year EPMO
Action item 2: A performance measurement framework (Management Accountability Framework (MAF) and other tools) is under development for the BOT enterprise and will identify the outcomes, as well as ownership for outcomes pre- and post-onboarding for departments and the business owner. The Secretariat is establishing the business owner role for BOT initiatives to ensure clear ownership for expected outcomes. The EPMO will ensure alignment with the Policy on Results. Q1, 2016 to 2017 fiscal year (Q4 for MAF) Maturity review in Q4, 2016 to 2017 fiscal year EPMO

Recommendation 2

The Secretariat should communicate expectations of departments and agencies (such as implementation timelines, the degree and nature of all players’ involvement, and the initiative’s expected outcomes) early in the life cycle of a transformation initiative.

Proposed action Start date Targeted completion date Office of primary interest
Action item 1: The EPMO will develop a BOT Organizational Change Management Program and Engagement Strategy across the Government of Canada. The strategy will deliver executive change management training to departments in order to provide them with the tools and guidance for their staff to understand the implications of BOT in their day-to-day activities. EPMO has been working with the Canada School of Public Service on this initiative. Q1, 2016 to 2017 fiscal year End of 2016 to 2017 fiscal year EPMO
Action item 2: The Secretariat will communicate the expectations of departments early in the life cycle of a transformation initiative by ensuring that each one has its own project-specific engagement and change management strategies. Q1, 2016 to 2017 fiscal year Ongoing throughout onboarding BOT initiatives

Recommendation 3

The Secretariat should ensure that Secretariat-led government-wide initiatives have governance and oversight that engage all functional areas at all levels of the governance chain to minimize silos.

Proposed action Start date Targeted completion date Office of primary interest
For a more efficient and modernized back office, the Secretariat will strengthen its governance leading to an integrated approach to enterprise-wide decision making. Action item 1: A formalized back office governance structure is currently in place that includes the BOT Steering Committee and supporting Planning Committee. An enhanced organizational governance structure for BOT initiatives will be considered, by the end of the fiscal year, to ensure that Shared Services Canada and Public Services and Procurement Canada as service providers and departments are aligned. Q1, 2016 to 2017 fiscal year Maturity review of governance committees in Q4, 2016 to 2017 EPMO
Action item 2: Individual BOT initiatives have the Secretariat as business owner, and their own management boards or steering committees with departmental involvement to provide oversight and governance for functional area decision making and alignment. Q1, 2016 to 2017 fiscal year Maturity review of governance committees in Q4, 2016 to 2017 (ongoing quarterly) BOT initiatives
Action item 3: The EPMO will address the strategic requirement for Secretariat-led sustained transformation leadership and governance for enterprise projects and initiatives that contribute to the modernization of internal services. (The transformation initiatives are referred to as BOT.) As part of the EPMO oversight role and in assessing that processes and systems are meeting the needs of departments and agencies and operating as intended, the EPMO will develop and implement an independent validation and verification framework, including annual plans, to allow for proactive responses and course corrections. Q3, 2016 to 2017 fiscal year Maturity review of governance committees in Q4, 2016 to 2017 (ongoing quarterly) EPMO
Action item 4: EPMO updates to the Resource Committee on a quarterly basis focusing on the risks and funding of the BOT initiatives. Q3, 2016 to 2017 fiscal year Ongoing quarterly EPMO
Report a problem or mistake on this page
Please select all that apply:

Privacy statement

Thank you for your help!

You will not receive a reply. For enquiries, contact us.

Date modified: