Audit of Information Technology General Controls Phase I – IT Governance

Table of contents

• Introduction
• Background
• Audit objective and scope
• Methodology
• Statement of conformance
• Audit findings and recommendations
• Governance structure
• IT policies, procedures and guidance
• IT governance risk management
• Application portfolio and enterprise architecture management
• Conclusion
• Appendix A– Management response and action plan
• Appendix B – References (non-exhaustive)

Introduction

Background

1. The focus of Information Technology (IT) within the federal government is changing. The Chief Information Officer of the Government of Canada stated in the Government of Canada IT Strategic Plan 2016-2020 that “the role of IT is being transformed from a back office function that provides services to a strategic business partnership that brings IT innovations to the table to address an organization's business needs.”

2. Within this context, there have been many changes related to the provision of IT services at the Public Service Commission of Canada (PSC) over the past 2 fiscal years. A new Chief Information Officer (CIO) was hired in 2015 who has led significant changes within the Information Technology Services Directorate (ITSD). This includes a number of staffing actions at all levels including at the supervisor, manager and executive level. In addition, the directorate was recently re-located within the organization. On November 1, 2017, ITSD was moved from the Services and Innovation Branch to the Corporate Affairs Sector as part of an overall governance shift to better align resources to deliver on the PSC's mandate and vision.

3. ITSD officials work with, and support, PSC senior management in carrying out their respective mandates and adjusting to an ever-changing operating and IT environment. The directorate's vision is to be “an agile professional organization that is a strategic partner in the delivery of innovative business solutions.” ITSD plays a leading role providing and supporting an IT infrastructure that is responsive, modern, adaptable, flexible and secure. The directorate also provides the full range of information management services to the organization. Finally, the CIO and ITSD staff make significant contributions to broader Government of Canada IT initiatives and innovations.

4. Phase I of this audit focused on IT governance, which is defined by the Institute of Internal Auditors as consisting of the “leadership, organizational structures, and processes that ensure that the enterprise IT function supports the organization's strategies and objectives.” This involves the management of IT operations and projects in a manner that is consistent with organizational objectives and supports alignment between IT and program delivery. As such, it is important that IT governance structures support IT integration into the organizational governance framework.

5. The PSC IT governance structure includes 8 committees with 2 main levels of responsibility — strategic and operational. Strategic level committees act as primary decision-making bodies for the PSC on operational items, including IT expenditures. Operational level committees focus on information sharing, analysis and the provision of advice and recommendations.

6. The strategic level committees are the Executive Management Committee (EMC), which is the primary decision-making forum on the operations of the PSC, and the Information Management / Information Technology Committee (IMITC). IMITC's mandate is to support EMC decision-making around IM- and IT-related developments by providing a forum for reviewing, evaluating and recommending strategies, plans and priorities related to IT initiatives. Furthermore, Corporate Affairs Sector management committee meetings provide a venue for the CIO to have strategic level discussions on IT-related governance and operational matters with the Vice-President and management team. The CIO is also invited to the PSC Meeting of the Commission to provide presentations on the state of IT in general, or on specific projects as required.

7. Operational level committees include the Project Review Committee, the Directorate Management Committee, the IM/IT Business Operation Team, the Architectural Review Committee (ARC) and the Change Advisory Board (CAB). Other committees are established as required to advance important PSC IT initiatives.

8. ITSD's budget allocation was approved for increase in February 2016 by EMC for a 3-year interim capacity increase to address technical / IT debt and to introduce a culture of innovation into the organization. In fiscal year 2017 to 2018 the PSC allocated approximately $12.5M to ITSD to support IT program management and service delivery. As of January 1, 2018, there are 90 employees (excluding students and casuals) who work in ITSD to manage and deliver IT services to the PSC.

Audit objective and scope

9. The overall audit objective was to determine the existence and effectiveness of Information Technology General Controls in ITSD at the PSC. Specifically for Phase I, the objective was to provide assurance with respect to whether there is an adequate management control framework in place to govern IT operations and mitigate risk.

10. The audit of Information Technology General Controls was conducted in 2 phases. Phase I focused on IT governance and Phase II focuses on internal controls in place over IT operations, IT and physical security, system development and change management, and business continuity. The Phase II audit report will be presented at the June 2018 Internal Audit Committee meeting.

11. The Phase I scope included IT governance elements that were in place in fiscal years 2016 to 2017 and 2017 to 2018. The scope does not include information management controls, as these were assessed in a previous internal audit in 2014 to 2015. IT controls related to financial reporting systems were also excluded, as these were included in an internal review completed at the end of 2016 to 2017.

Methodology

12. The following audit procedures were performed:

• interviews with PSC management, selected IT governance committee members and staff
• walk-throughs of key processes
• reviews and analysis of documents
• process mapping

Statement of conformance

13. The audit is in conformance with the Internal Audit Standards for the Government of Canada as supported by the results of the quality assurance and improvement program.

Audit findings and recommendations

Governance structure

14. Criterion. It was expected that the IT governance structure had defined roles and responsibilities and met Treasury Board requirements (see Appendix B). It was also expected that the PSC had adequate processes in place to manage the relationship with Shared Services Canada (SSC).

15. Conclusion. The PSC has an established IT governance framework that forms part of the overall organizational governance structure. Each committee has defined roles and responsibilities that are documented. The processes put in place to manage the relationship with SSC are adequate with some room for enhancement. Two opportunities for improvement were identified related to reviewing committee terms of references to ensure that they are in line with current expectations, and assessing the implementation of the business agreement with SSC to ensure there are no outstanding risks to the PSC.

IT governance structure

16. An IT governance structure that includes 8 committees has been established and approved by the President of the PSC. Each committee has defined roles and responsibilities that are documented. The documentation includes, but is not limited to, the identification of committee members, committee mandates and frequency of meetings.

17. Operational level committees focus primarily on reviewing the technical and architectural aspects of IT, as well as projects that are either primarily IT focused or have an IT component. These committees provide advice to presenters and make recommendations to strategic level committees for decision-making purposes.

18. At the strategic governance level, IMITC was involved in the development of the IT Strategic Plan 2017 to 2020, which included resource allocations, risk management and IT alignment with overall PSC strategic objectives components. EMC approved the plan and also approved the allocation of resources to ITSD to support the provision of IT services within the PSC.

19. In addition, EMC performs an annual IT prioritization exercise to assess the funding required to support effective IT management at the PSC. In March 2018 the focus of the EMC retreat was on finalizing sector budgets for fiscal year 2018 to 2019, which includes approving the IT budget for PSC and the projects that will receive funding in an IT prioritization exercise.

20. The audit noted some shortcomings that affect the functioning of the IT governance structure. The role of some of the committees was not always clear, and some committee terms of references had not been updated. In addition, the communication of information, analysis, decisions and recommendations among the governance committees was not always considered adequate to support decision-making.

21. The audit team was also informed that the value-added of the strategic committees is at times challenged by focusing on detailed technical or operational decisions that may be better suited at the operational committee level or the CIO directly. For example, some records of discussion and committee documentation did not sufficiently identify and support the recommended approach for particular items. As such, there is an opportunity to improve coordination of information among the committees and improve records of discussion to support decision-making processes. This is of particular interest due to the high level of new hires in the past year within ITSD, from the working level and through the levels of management.

22. Recommendation 1. It is recommended that the CIO work with IMITC and operational level governance committee chairs to review and update (where required) their committee's terms of reference document to ensure that they support current requirements. This review should include assessing the manner in which records of discussion are documented to ensure clarity of the issues, the advice provided and the expectations of the next level committee review.

Management of the relationship with SSC

23. The Shared Services Act specifies that the PSC is required to use SSC as its IT service provider. The PSC's relationship with SSC has 2 primary components: executive direction and business operations. The executive direction component includes bi-weekly PSC-SSC meetings where the CIO and ITSD directors meet with their SSC counterparts to discuss strategic initiatives and larger operational issues. A particular goal is ensuring that the budgeted work can be performed in a given time period to prevent potential funding lapses.

24. The management of the business operations component involves regular meetings with members of the team under the Director of IT Solutions Engineering and Service Management, and the Manager Infrastructure and Operations team. The purpose of these meetings is for SSC to respond to PSC operational issues, changes and needs. There are also regular communications between SSC and the PSC as required.

25. A business agreement signed by the presidents of the PSC and SSC dated March 15, 2017, articulates the roles and responsibilities of each organization, the governance structure, the IT service delivery and performance expectations, and the dispute resolution mechanisms. The audit team was informed by ITSD officials that the relationship with SSC is adequate and generally functioning as intended.

26. ITSD is expected to review the IT services provided by SSC and make a determination on the adequacy of service levels. The audit team was informed that the PSC does not receive metrics regarding service delivery from SSC in accordance with provisions established in the business agreement. As such, the PSC has not been provided with a dashboard containing information on performance metrics regarding SSC service delivery levels, or information required to support the PSC's monitoring of acceptable IT systems use. This is important because the PSC is obligated to monitor employee use of IT systems as per Treasury Board policy but can only do this by obtaining data from SSC. There is a risk that PSC employees could misuse systems and ITSD would be unable to monitor and address this situation.

27. In order to risk manage this situation, ITSD officials have put compensatory controls in place to obtain some information to review services obtained from SSC. For example, ITSD officials have built relationships with their SSC colleagues through joint meetings, and they obtain some information and data through these sources. The audit team was also informed that there is a concern that the PSC may not be receiving sufficient support from SSC for new development projects. This could have a significant impact on key PSC IT-related projects on the horizon that have government-wide implications.

28. Recommendation 2. It is recommended that ITSD conduct a review of all expected deliverables outlined in the business agreement with SSC, and report to EMC on performance and potential risks to PSC related to the delivery of IT services, and the provision of data and information to support PSC CIO decision-making.

IT policies, procedures and guidance

29. Criterion. It was expected that the PSC had IT policies, procedures and guidance in place that were aligned with Treasury Board policy requirements.

30. Conclusion. The PSC has policies, procedures and guidance in place that support consistent IT application that are aligned with Treasury Board policy requirements. A number of internal policy and guidance documents have not been recently updated. The CIO is awaiting publication of the new Treasury Board IT policy framework in order to assess the extent to which internal PSC policies will have to be renewed, updated or cancelled.

IT policies and procedures

31. In the past, the PSC developed a suite of internal IT policies and guidance documents to support and supplement Treasury Board policies and directives. The audit found that some of these internal policies had not been recently updated. For example, internal policies related to IT security and IT governance had not been updated since 2011. The recent approach taken within the PSC is to only review and update internal IT policies when a change or update is identified or required.

32. The CIO informed the audit team that since Treasury Board officials are currently reviewing and updating all IT policies and related directives, the goal is for ITSD to perform a detailed review of all existing internal policy and guidance documents once the new Treasury Board policy suite is published. The ITSD concept is for PSC employees to adhere to Treasury Board IT policy suite documents and only provide supplemental PSC guidance where required (for example, PSC BlackBerry travel procedure covering lost and stolen mobile devices).

33. The CIO provides periodic and annual reports to the Treasury Board of Canada Secretariat on PSC compliance with IT policy requirements. A Treasury Board assessment of PSC IT policy items and operational performance is conducted annually as part of the overall PSC Management Accountability Framework assessment.

IT governance risk management

34. Criterion. It was expected that the PSC had an effective process for IT governance risk management.

35. Conclusion. The PSC has effective processes in place to manage and mitigate IT risks related to the achievement of corporate objectives. ITSD specific operational risk management processes that clearly identify directorate specific risks and mitigation strategies have not been fully documented. However, operational risks are reported during Sector Management Committees, CIO-Vice-President bilateral discussions and in quarterly sector risks and issues reports to the Vice-President's Office. In addition, there are effective processes in place to manage IT risks from a project perspective, including through the governance committees.

IT governance risk management

36. The PSC's annual integrated business planning exercise includes a risk assessment process to identify and rank organizational risks and identify potential mitigation strategies. These risks are included in the PSC's risk registry and in the PSC's Integrated Business Plan. For fiscal-year 2017-18, one of the corporate risks stated, “As the PSC shifts IT resources to align to Government of Canada direction for greater consolidation of systems and support, that its ability to service and maintain current PSC systems may be hindered.” Risk response strategies were identified to mitigate this important risk. Throughout the year the CIO assesses progress against key Integrated Business Plan IT-related deliverables and provides quarterly updates on results. Furthermore, the CIO must assess risk mitigation strategy implementation in response to corporate risks and report on progress.

37. While ITSD provides input into overall corporate risk discussions, a formal, documented approach to identifying directorate-specific risks to successfully supporting business owners in achieving IT-related objectives and potential mitigation strategies has not been formally documented. Some examples of IT operational risks not currently captured in corporate risk documents include: the impacts of software releases, resource management, IT committee mandates and decision-making, the relationship with SSC and innovation capacity.

38. For IT projects and projects with an IT component, there were effective processes in place at the PSC to identify, manage and develop mitigation strategies to reduce the likelihood and impact of identified risks materializing. As such, the audit found that IT-related projects risks were managed effectively on a project-by-project basis.

39. Recommendation 3. It is recommended that ITSD formally document operational risk analysis, reporting and monitoring processes to provide clarity on ITSD-level risk management and the effectiveness of proposed mitigation strategies performed.

Application portfolio and enterprise architecture management

40. Criterion. It was expected that the PSC had an application portfolio and enterprise architecture management framework that enabled business and IT alignment.

41. Conclusion. The PSC application portfolio and enterprise architecture management functions are focused on improving IT-business alignment and reducing the PSC's technical debt related to legacy systems. The PSC's current application portfolio size is noted as too large for the PSC's size. An opportunity was identified to review and update enterprise architecture process documentation through the EMC-approved Enterprise Architecture Program investment.

Enterprise architecture

42. The Architecture Review Committee (ARC) — formerly the Technical Review Committee (TRC) — is an advisory committee that is comprised of ITSD managers. It is responsible for monitoring the Change Advisory Board (CAB), supporting enterprise architecture (EA), and providing architectural recommendations to the Division Management Committee.

43. The audit found that key documents, namely the ARC and CAB Terms of Reference, the IT Management Lifecycle Reference Guideline, the Project Management Framework, the PSC Security Governance Framework, and the System Decommissioning Methodology have not been recently updated to ensure that no process ambiguities have been created in the transformation of the TRC to the ARC. The audit noted that there is a risk that overlaps or gaps may exist in the distribution of responsibilities related to production release schedule approvals, or the comprehensiveness and adequacy of the processes in place to support the discharge of responsibilities related to how analysis is to be carried out when architectural change involves security considerations.

44. The audit team was informed that the transition of the ARC from a decision-making body to an advisory body has presented some challenges to members understanding their roles and responsibilities. The transition means that ARC members will have to focus on providing advice, which requires members to consider issues from a broad, horizontal perspective, and committee members need to be able to have both an operational and strategic perspective to provide value-added comments.

45. In addition, the ARC is not always able to perform its advisory function effectively due to the document processes that are in place to support committee deliberations, particularly regarding project documentation. The lack of clear documentation standards is exacerbated by the fact that there is a relatively high level of turnover of committee members. The audit team was informed that the Enterprise Architecture Program Implementation Project, which is planned to be completed in 2019, may address elements of documentation standardization.

46. ITSD officials have developed processes on an ad hoc basis to mitigate some of the above-noted concerns. However, there has been no formal mapping or documentation of revised processes. Without proper documentation, there is a risk that expected procedures may not be understood or followed, which could impact ARC member architectural review processes and decision-making. The audit team was informed that the project to implement a mature EA program that was approved by EMC in March 2017 is currently in progress.

Application portfolio management

47. As of January 2018, there are more than 200 applications installed on employee computers across the PSC, many without ITSD's authorization or knowledge. The audit noted current ITSD initiatives to streamline the number of applications being used within the PSC, including the adoption of measures allowing only approved software to run on Windows 10 devices (once Windows 10 has been implemented), and requiring that commercial off-the-shelf applications be vetted by the CAB before being installed within the organization.

48. Recommendation 4. It is recommended that ITSD continue to review key enterprise architecture policy and guidance documentation as per the EA program project to ensure that they are up-to-date and support the effective management of enterprise architecture within the PSC.

Conclusion

49. Overall, the PSC has an adequate management control framework in place to govern IT operations and mitigate corporate and IT project risks. An IT governance framework is in place and forms part of the overall PSC governance structure. The processes in place to manage the relationship with SSC are adequate with some room for enhancement. PSC IT policies, procedures and guidance support the consistent application of IT and are aligned with Treasury Board requirements. The CIO is awaiting publication of the new Treasury Board IT policy framework in order to assess the extent to which internal PSC policies will have to be renewed, updated or cancelled. EMC's approval of a 3-year interim capacity increase for ITSD in 2016 is positioning the organization to catch up on technical debt to reduce operational risks and position the organization to take on innovative projects. Finally, application portfolio and enterprise architecture management functions are focused on improving IT-business alignment and reducing the PSC's technical debt.

50. Four areas for improvement are identified in the report. The first relates to reviewing the mandates and terms of references of existing committees to ensure that they are in line with current expectations. A second area is in regards to assessing the implementation of the business agreement with SSC to ensure that there are no outstanding risks to the PSC. A third area concerns formal documentation of ITSD’s risks and mitigation processes related to delivering IT services within the PSC. Finally, there is an opportunity to improve the documentation related to the enterprise architecture processes. Four recommendations are made in the report to address these opportunities for improvement.

Appendix A– Management response and action plan

Appendix B – References (non-exhaustive)

• COBIT-5 Framework, Information Systems Audit and Control Association
• Global Technology Audit Guides, Institute of Internal Auditors
• Policy Framework for Information and Technology, Treasury Board of Canada
• Policy on Internal Audit, Treasury Board of Canada
• Policy on Government Security, Treasury Board of Canada
• Policy on Management of Information Technology, Treasury Board of Canada
• Directive on Management of Information Technology, Treasury Board of Canada
• Operational Security Standard: Management of Information Technology Security, Treasury Board of Canada
• Standard on Web Interoperability, Treasury Board of Canada
• Top 10 IT Security Actions to Protect Government of Canada Internet-Connected Networks and Information, Canada Security Establishment