Audit of Service Level Agreements

Audit Report

Office of Audit and Evaluation
June 2014

Executive Summary

What we examined

A service level agreement is a formal agreement between two or more parties (between departments, between a department and a common or shared service provider, or between various levels of government) that articulates the terms and conditions of a particular service relationship.

Shared Services Canada (SSC) was created in August 2011. At that time, the Department inherited 1,553 service level agreements from Public Works and Government Services Canada with a total value of $311,539,726. Two of these agreements involved the management of the Blue Pages, a telephone directory listing government organizations and officials. The Blue Pages include a section for the House of Commons.

The first Blue Pages service level agreement (SLA 6800 – National Coordination of Blue Pages Listings) related to the national coordination and updating of the Blue Page listings. This agreement was valid from April 1, 2010, to March 31, 2013, and was valued at $238,422. The second Blue Pages service level agreement (SLA 8541 – Managed Telecommunication Services) pertained to the management of telecommunications services and was valid from April 1, 2011, to March 31, 2016. It was valued at $8,050,215.

The objective of this audit was to provide assurance that appropriate governance and internal controls were in place for the House of Commons service level agreements related to the national coordination of Blue Pages listings.

Adequate governance processes and internal controls for service level agreement lifecycle activities ensure that SSC services meet client expectations. Specifically, adequate oversight structures, roles, responsibilities, tools and issue resolution procedures for House of Commons service level agreements are required to help ensure SSC and House of Commons expectations are met.

What we found

• No verification process for contracts with third parties – There was no formal process for ensuring that contracts with third-party service providers were in place prior to the finalization of service level agreements with the service provider (TELUS) or the client (House of Commons).
• No client issue management process – There was no defined process for tracking and logging client issues or implementing continuous improvement activities in respect of such issues.
• Service level agreement reporting process not defined – There was no formal process for ensuring the clear definition of SLA reporting requirements, including both external and internal reporting requirements.

Yves Genest
Chief Audit and Evaluation Executive

1. The Government of Canada established Shared Services Canada (SSC) on August 4, 2011, to consolidate, streamline and improve the information technology (IT) infrastructure services and achieve excellence in the delivery of email, data centre and network services across the federal government.
   
2. At the time of its creation in August 2011, SSC inherited 1,553 service level agreements (SLA) from Public Works and Government Services Canada (PWGSC) with a total value of $311,539,726. Two of these SLAs were with the House of Commons (HoC) and pertained to the Government of Canada Blue Pages listings. The Blue Pages are a telephone directory listing government organizations and officials. The first SLA (SLA 6800 – National Coordination of Blue Pages Listings) was signed by PWGSC in July 2010 for the national coordination and updating of the Blue Pages listings. This SLA was valid from April 1, 2010, to March 31, 2013, and was valued at $238,422. The responsibilities outlined in the SLA were transferred to SSC when the Department was formed.
   
3. The second SLA (SLA 8541 – Managed Telecommunications Services) pertained to the management of telecommunications services, including the payment of publishing invoices related to the Blue Pages. This SLA was valid from April 1, 2011, to March 31, 2016, and was valued at $8,050,215.

Objective

4. The objective of this audit was to provide assurance that appropriate governance and internal controls were in place for the HoC SLAs related to the national coordination of Blue Pages listings. The criteria used in this audit are presented in Annex A.

Scope

5. The scope of the audit included a review of HoC National Coordination of Blue Pages Listings SLA and HoC Managed Telecommunications Services SLA controls related to:
   
   • Governance and oversight;
   • People, processes and tools; and
   • SLA lifecycle activities such as monitoring and reporting.

Methodology

6. A risk-based audit program was developed to provide more detail on how the audit criteria would be addressed. The program refers to the following audit procedures:
   • Review of policies and procedures related to the management of SLAs;
   • Review of strategic plans, committee terms of reference and meeting minutes;
   • Review of SLA management training and tools provided to departmental staff and management; and
   • Interviews with individuals identified as key players in the management of the HoC SLA.

Statement of assurance

7. Sufficient and appropriate procedures were performed and evidence was gathered to support the accuracy of the audit conclusion. The audit findings and conclusion are based on a comparison of the conditions that existed as of the date of the audit against established criteria agreed upon with management. This engagement was conducted in accordance with the Internal Auditing Standards for the Government of Canada and the International Standards for the Professional Practice of Internal Auditing. A practice inspection has not been conducted.

Detailed Findings and Recommendations

Summary of strengths

8. Governance – SSC created the Optional Services Task Force, which was mandated to review services defined in SLAs inherited from PWGSC to determine whether they fell within the scope of SSC services.
   
9. Processes and Tools – SSC inherited the use of the Business Intake Tracking System (BITS) and SLA pricing templates. These processes and tools were well implemented at SSC and were being applied to the HoC SLAs.
   
10. SLA Approval – SLA 6800 and SLA 8541 had been properly approved in accordance with the business intake procedure at the time of their creation.

Contracts with third parties not verified

11. SLAs for the provision of service may involve services to be provided under contract by a third party (vendor). We expected to find a process for ensuring that contracts with third parties for the delivery of services to partners were in place before the SLAs were signed (COBIT 4.1 DS1.5 and DS2.3).
    
12. We found that the BITS process used to create SLAs did not include a requirement or checkpoint to review the SLA scope and ensure that contracts were in place with third-party service providers before the SLAs were signed.
    
13. Prior to the establishment of SSC, PWGSC contracted the Blue Pages listings services for the HoC to telephone companies, such as Bell, TELUS and MTS, through local access service (LAS) contracts. Following changes in the telecommunications industry, including divestitures of directory publishing businesses by the telephone companies and the subsequent acquisition of these businesses by the Yellow Pages Group (YPG), Blue Pages listing services were provided by YPG. Because of these changes, Blue Pages listings were excluded from the scope of the LAS contracts renegotiated in 2010. Therefore, the renegotiated LAS contracts could not be used as a basis for payment, and no other contracting vehicle was put in place by PWGSC. In July 2010, PWGSC realized that no contract was in place with YPG and that the invoices could not be paid. To correct the situation and pay the YPG invoices, PWGSC began the approval process for paying invoices without a written contract (called a confirming order).
    
14. In August 2012, the YPG’s automated system generated a collection notice to a Member of Parliament indicating that he had an outstanding balance for the period from April 2010 to June 2012. On September 26, 2012, PWGSC approved the confirming order, and the unpaid invoices were subsequently paid. After November 2012, additional invoices continued to accumulate, and contractual negotiations to resolve this issue were still under way at the time of the audit.
    
15. Without a formal process for ensuring that contracts are in place with third-party service providers, there is an increased risk that agreed-upon services will not be delivered as provided for in the SLA.

Client issues not tracked or processed for lessons learned

16. We expected to find that client issues pertaining to SLAs would be formally tracked, logged and resolved in a timely manner, and that a process for continuous improvement would be in place (COBIT 4.1 DS1.3, DS1.4 and DS1.7).
    
17. We found that contact names for issue escalation were provided in the HoC SLAs that were reviewed (SLA 6800 and SLA 8541). However, there was no formal tool or process for tracking and logging client issues, nor was there a process for implementing continuous improvement activities associated with specific client issues.
    
18. Our review of the Member of Parliament’s invoice non-payment issue revealed that specific resolution actions were being taken (e.g. contract negotiations with YPG for unpaid invoices). However, it also revealed that:
    • The status of resolution of the issue was unknown to interviewees, given the lack of a process or tool for tracking issue resolution plans and progress; and
    • No continuous improvement activities had been performed in respect of SLA management processes in response to the non-payment issue.
19. Without a defined process for tracking, reporting on and addressing client issues, there is an increased risk that issues will not be resolved in a timely manner, which increases reputational and business risk.

Service level agreement internal reporting requirements not defined

20. We expected to find a process for clearly defining the internal reporting requirements for SLAs, including what was being reported, how the information was being reported and to whom it was being reported (individuals or governing bodies) (COBIT 4.1 DS1.2, DS1.3 and DS1.4).
    
21. We found that there was no process for clearly defining the internal reporting requirements associated with the HoC SLAs that we reviewed (SLA 6800 and SLA 8541). As a result, no internal reporting was being done.
    
22. Without clear internal reporting processes, there is an increased risk that relevant stakeholders and governing bodies are not being provided with reports in a timely manner and are therefore unable to perform the proper oversight and guidance function with respect to SLA lifecycle management activities.

Service level agreement external processes not always followed

23. We expected that external reporting requirements would be defined and met as specified in the SLAs (COBIT 4.1 DS1.2, DS1.3 and DS1.4).
    
24. A review of SLA 6800 (National Coordination of Blue Pages Listings) with the HoC revealed that external reporting requirements were defined on the basis of standard SLA templates. However, these requirements had not been met (e.g. semi-annual service reporting and semi-annual service review reporting).
    
25. A review of SLA 8541 (Managed Telecommunications Services) with the HoC revealed that external reporting requirements were defined but only partially met. One of the three required quarterly reports had not been provided to the HoC.
    
26. Failure to meet external reporting requirements defined in SLAs increases the risk that service requirements and levels are not being met, which in turn increases reputational and business risk.

Conclusion

27. We found that there were weaknesses in the governance processes and internal controls for the management of HoC SLAs related to the national coordination of Blue Pages. We found that the BITS process used to create SLAs did not include a requirement or checkpoint to review the SLA scope and ensure that contracts were in place with third-party service providers before the SLAs were signed.
    
28. We found that there was no formal tool or process for tracking and logging client issues or implementing continuous improvement activities associated with specific client issues.
    
29. We found that there was no process for clearly defining internal reporting requirements associated with the SLAs reviewed. The external reporting requirements were defined but not always met.

Management Response and Action Plans

Annex A - Audit Criteria

The following audit criteria were used in this audit. The criteria are based on the Control Objectives for Information and Related Technology (COBIT 4.1) framework, in particular the DS1 domain (define and manage service levels).

Free PDF download

To access the Portable Document Format (PDF) version you must have a PDF reader installed. If you do not already have such a reader, there are numerous PDF readers available for free download or for purchase on the Internet:

• Adobe Reader
• Foxit Reader
• Xpdf
• eXPert PDFReader