Audit of the Management of Customer Revenue Agreements
Table of Cotents
Executive summary
Shared Services Canada (SSC) delivers on its mandate and achieves results for Canadians through five core areas of responsibility: Email and Workplace Technology, Cyber and Information Technology (IT) Security, Data Centres, Customer Relationships and Service Management, and Telecommunications. SSC recovers costs from its customers through a variety of customer revenue agreements.
From April 1, 2018 to March 31, 2019 SSC managed over 13,000 new or outstanding customer business requests (BRs). These requests range from small to large and from simple to complex. The complex ones typically span across multiple SSC service lines over multiple years. The BRs go through a revenue agreement development process whereby SSC works with its customers to design and implement solutions. The agreements then serve as the basis for SSC to recover costs from its customers.
Customer revenue agreements account for 25% of SSC’s total funding envelope. Failure to properly renew service agreements can affect SSC’s ability to deliver reliable and secure services to customers.
This audit examined the effectiveness of the processes for the establishment, management, and renewal of customer revenue agreements. The audit was approved by the President following recommendation by the Departmental Audit Committee in the 2019 to 2022 Risk-Based Audit Plan.
The following areas were examined to determine the effectiveness of the customer revenue agreements process:
- Management of agreements: Does SSC have adequate and effective controls and procedures to establish and manage customer revenue agreements?
- Renewals: Are customer revenue agreements renewed as required?
- Internal communication: Does SSC convey accurate, consistent and timely information on customer revenue agreements to internal stakeholders (for example SSC service lines)?
- External communication: Does SSC communicate effectively with its customers?
The audit found the following:
- processes related to the establishment of customer agreements were well documented, but there were issues in providing expected timelines to customers, and the renewal of agreements. There were gaps and inconsistencies in the files examined, including lack of confirmation of completion of requests, lack of justification of exceptions to the process, and inconsistencies in the financial information found in agreements versus the corresponding invoices
- there was lack of guidance available to Client Executive teams to identify which services typically had on-going costs. The Business Intake Tracking System was not being used effectively to flag renewals, and there was a lack of information available to confirm that agreements had been renewed
- the escalation processes used by Client Executive to manage agreement disputes were found to be effective, although generally informal
- roles and responsibilities were defined but were not clearly communicated, or consistently followed by the relevant stakeholders
- regular reporting was being provided to SSC senior managements and other stakeholders (SSC and customers)
- client executive teams had strong relationships with their customers, but tools and information shared with customers were inconsistent. Agreements and invoices were missing critical information for customers to properly reconcile invoiced costs with services provided
- some controls for the establishment of customer revenue agreements were adequate, however, the management and renewal of agreements were inconsistent
In conclusion, agreement processes should be improved to allow SSC to effectively recover service costs from its customers.
Begonia Lojk
Acting Chief Audit and Evaluation Executive
A. Introduction
1. Background
SSC delivers on its mandate and achieves results for Canadians through five core areas of responsibility:
- email and workplace technology
- cyber and IT security
- data centres
- customer relationships and service management
- telecommunications
Section 9 of the Shared Services Canada Act allows the SSC to charge customers for services through the use of a variety of customer revenue agreements.
From April 1, 2018, to March 31, 2019, SSC managed over 13,000 new or outstanding customer BRs or service requests. These service requests range from small to large and simple to complex. Complex BRs typically span across multiple service lines and over several years. When customers need new IT infrastructure services, they submit formal IT BRs to SSC. The departmental Client Executive teams track the requests while working with various stakeholders (that is, service design, costing, service operations, and project management teams) to ensure understanding of the requirements to develop customer revenue agreements. Many services include on-going service maintenance, licencing and support costs, all of which should be listed in service agreements. Once an agreement is approved, SSC delivers and invoices for its services based on the agreement.
2. Rationale for the audit
Customer revenue agreements account for 25% of SSC’s total funding envelope. The terms and conditions of the agreements should clearly define the goods and services to be provided, the costs to be recovered and the basis of payment. SSC counts on the renewal of customer service agreements to ensure costs are recovered for on-going costs, including licensing, maintenance and support of SSC’s IT infrastructure. Failure to properly renew service agreements could affect SSC’s ability to deliver reliable and secure infrastructure services to customers.
3. Audit authority
The Audit of the Management of Customer Revenue Agreements was approved by the President, following recommendation by the Departmental Audit Committee of the 2019 to 2022 Risk-Based Audit Plan.
4. Objective of the audit
The objective of this audit was to provide assurance that SSC had adequate and effective processes for the establishment, management and renewal of customer revenue agreements.
5. Scope
The scope included agreements for which work was underway between April 1, 2018 and June 30, 2019. The scope included all processes, procedures and tools to establish, manage and renew customer revenue agreements, including the required internal and external communication. The scope excluded service pricing, service design, revenue allocation and SSC’s service catalogue.
6. Methodology
The audit was conducted in accordance with the International Internal Audit Standards for audit engagements and the Treasury Board Policy on Internal Audit. It was conducted by means of:
- interviews
- document reviews
- file reviews
- data analysis
- walkthroughs of key systems and processes
7. Statement of conformance
In my professional judgment as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the opinion provided and contained in this report. The opinion is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed to by management. The opinion is applicable only to the entity examined. The engagement was conducted in conformance to the requirements of the Policy on Internal Audit, its associated directive, and the Values and Ethics Code for the Public Sector and the Institute of Internal Auditors Code of Ethics. The evidence was gathered in compliance with the procedures and practices that meet the auditing standards, as corroborated by the results of the quality assurance and improvement program. The evidence gathered was sufficient to provide senior management with proof of the opinion derived from the internal audit.
B. Findings, recommendations and management response
1. Management of agreements
Line of Enquiry 1: SSC has adequate and effective controls and procedures to establish and manage customer revenue agreements.
1.1 Process guidance
Audit Criterion: There are clearly defined and documented processes for the establishment and management of customer revenue agreements.
It was expected that there were clearly defined and documented processes for the establishment and management of customer revenue agreements.
The audit team found that policies and guidelines relating to the establishment of customer agreements were documented and available to stakeholders.
The following gaps/issues were also found:
- the benefits of using drawdown agreements was not clearly communicated to Client Executives, services lines and customers
- guidelines for the renewal of agreements did not contain procedures for the renewal of on-going costs from drawdown and project recovery agreements
- there were no timelines for the establishment of agreements
- guidelines for the identification of customer work priorities were unclear
These findings are further explained below.
The drawdown agreement, was introduced to reduce administrative burden for customers and to speed up service delivery. While some guidance related to drawdown agreements was available, the benefits of using this type of agreement were unclear to stakeholders. Guidelines for the renewal of agreements covered only standard service agreements; they did not outline processes for the renewal of on-going costs of project recovery agreements.
Timelines for the establishment of customer revenue agreements provide a baseline for escalation and follow-up procedures for both SSC and customers. The audit found that these timelines were not communicated.
While the criteria for determining work priorities were defined, they were unclear. Work priorities (Urgent, High and Medium) are defined by the customer in the business requirements document. Both urgent and high priority business requirements must include justification by the customer. From April 1, 2018 to June 30, 2019, SSC received 15,499 requests, with 45% of these being either “Urgent or High” priority. This high volume of urgent/high priorities resulted in Client Executives involving their customers to determine and assign work priorities on a case by case basis.
1.2 Tracking of Agreement Information
Audit Criterion: There are processes and controls in revenue agreements to accurately track information.
It was expected that agreement information was accurately tracked in the official system of record. Shared Services Canada records customer revenue agreements in its Business Intake Tracking System. Accurate and complete information is required to ensure that services are completed and costs are recovered in a timely manner.
A random sample of 35 agreements was tested to determine if key information in the agreements (customers, agreement types, status, start date, end dates, agreement amount) matched official system records.
The audit found that, with minor exceptions, customer names, agreement types, and start and end dates contained in the agreement matched the records contained in the Business Intake Tracking System. In 16 of 35 files tested there was no information in the agreement file to indicate that goods and services had been delivered (for example an email from the service line or customer to the Client Executive to confirm completion of the request). In 6 of 35 files tested, the costs documented in the service agreement did not match the amount in the Business Intake Tracking System, and there was no revised cost information on file to support these changes. Processes for tracking agreements differed among Client Executive Teams. There is a risk that these discrepancies are affecting the timely and accurate recovery of customer revenues.
1.3 Implementation
Audit Criterion: The processes and controls related to the management of customer revenue agreements are implemented consistently in accordance with documented policies and guidelines.
It was expected that key processes and controls were implemented consistently in accordance with policies and guidelines. The testing included a sample of 40 customer revenue agreements to verify that:
- agreement types were determined appropriately in accordance with established criteria
- exceptions to year-end deadlines were approved
- agreements were in place before work was started
- standard documents were retained in agreement files
- custom pricing was appropriately approved
- customers were promptly invoiced
- agreement costs matched invoiced costs
SSC had defined five types of service agreements: recovery agreements, project recovery agreements, service agreements, service level agreements and drawdown agreements. Criteria for determining which type of service agreement to use are found in SSC’s Agreement Guidelines document. With minor exceptions, agreement types were determined according to these documented criteria.
Customer requests and service agreements signed after SSC’s year-end intake deadline require approval from an authorized service line representative. These deadlines allow SSC to effectively manage its capacity to deliver service requests. Of the 40 transactions tested, 8 were received after the year-end deadline. Five of these 8 had appropriate justifications and approvals on file. Three transactions were for work completed by the services lines with no year-end exception approvals or agreements in place. The service line approvals are required to validate that capacity exists within the service line to take on additional requests and complete them prior to the end of the fiscal year. If demand for services outpaces SSC’s capacity to deliver, customer year-end requests may not be fulfilled.
Funding is allocated to branches within SSC based on approved forecasted revenue and signed customer revenue agreements. While urgent government business sometimes requires SSC to undertake work prior to the approval of a signed agreement, this should be done by exception. The audit found that in 6 of 40 files tested, work had been started prior to a signed agreement. If service lines allocate resources to work before customer agreements are approved, there may not be sufficient resources remaining for operations or new requests.
Each customer revenue agreement file should contain key documents outlining business requirements, service costing and the agreement itself. With minor exceptions, these key documents were contained in client files.
SSC’s enterprise price estimation tool was developed to standardize and automate the pricing process. When a customer requirement exceeds $150,000, custom pricing requires approval by SSC’s Financial Strategy and Costing division. The audit found that these approvals were being obtained as required.
Customer agreement revenues account for approximately 25% of SSC’s total funding envelope. SSC’s Guide on Revenue Allocation requires invoices to be processed on a timely basis. With minor exception services were invoiced on a timely basis; invoices were issued within 30 days from record of completion.
A sample of 40 agreements was tested to verify if agreement costs and invoiced costs matched. Thirteen of 23 invoiced costs did not match the signed customer revenue agreement and there were no justifications on file to support the revised costs.
Recommendation 1
High priority
The Senior Assistant Deputy Minister Service Delivery and Management Branch should ensure that appropriate justifications for changes to agreement costs, year-end exception approvals, and service confirmations are retained.
Management response
Management agrees with this recommendation and will develop and strengthen the business intake process as well as communications with the Client Executive to ensure that procedures surrounding retention of documents supporting cost changes, year-end exceptions and service confirmations are clear.
2. Renewals
Line of Enquiry 2: Customer revenue agreements are renewed as required.
2.1 Tracking renewals
Audit criterion: There are processes and controls to accurately track agreement renewals.
It was expected that SSC’s Business Intake Tracking System included controls to identify and track agreements that need to be renewed. It was also expected that guidance on services with on-going costs were available to the Client Executive team and that the guidance was used to accurately track and renew on-going service agreements.
Agreements with on-going service costs should be flagged for renewal in SSC’s Business Intake Tracking System. The audit team tested a sample of 34 agreements which had been identified for renewal in 2019-20 by the Enterprise Business Intake and Demand Management Team to verify that they were flagged for renewal and that they were in fact renewed. In 20 of the 34 agreements tested were not flagged for renewal in the system. There was no guidance available to identify services with on-going costs that should be flagged for renewal. Of the 34 agreements for renewal tested, 10 were not renewed as of October 1, 2019. Without effective controls for the identification and tracking of agreements for renewal, SSC may fail to renew customer agreements and incur additional on-going operational costs.
2.2 Process guidance
Audit Criterion: There are established processes, procedures and tools that support the agreement renewal process.
It was expected that processes and procedures were established to communicate identified agreements for renewal, to renew the agreements, and to notify the service lines when agreements are renewed or cancelled. The audit team found that a list of agreements to be renewed was sent annually to Client Executive teams. Because agreements are not consistently flagged for renewal in the system, this list was created manually by the Enterprise Business Intake and Demand Management team. It was then up to the Client Executive team to confirm with the customer whether or not they would like to renew the service. Both the Client Executives and the customers confirmed that this process was taking place.
There were no notifications from Client Executive teams to the service lines to confirm if agreements had been renewed or cancelled. There is a risk that service lines incur unnecessary on-going costs that are not recoverable.
2.3 Resolution processes
Audit Criterion: There are resolution processes in place to manage agreement disputes.
It was expected that processes were in place to manage and escalate customer disputes. The audit team found that escalation processes were not formally defined, however, there were various channels to raise customer concerns to SSC senior management and the Account Management Board. Customers interviewed confirmed that escalation procedures, while informal, were generally effective.
Recommendation 2
High priority
The Senior Assistant Deputy Minister Service Delivery and Management Branch should:
- develop and provide guidance to Client Executive teams on the tracking of agreement renewals
- inform the branch service lines when agreements are renewed or cancelled
Management response
Management agrees with this recommendation. The Business Intake Tracking System currently has the functionality to track agreement renewals. An updated Responsible Accountable Consulted Informed matrix regarding this process is currently being developed that will clearly identify roles and responsibilities regarding agreements. Enterprise Business Intake and Demand Management will prepare and distribute additional reports to the Service Lines and Client Executives to strengthen communications surrounding renewals and cancellations of service agreements.
3. Internal communication
Line of Enquiry 3: SSC conveys accurate, consistent and timely information on customer revenue agreements to internal stakeholders.
3.1 Roles and responsibilities
Audit Criterion: There are clearly defined roles and responsibilities for the development of agreements and delivery of services.
It was expected that there were clearly defined roles and responsibilities for all stakeholders involved in the agreement process. The audit found that roles and responsibilities were defined but they were not clearly communicated to stakeholders. The customer revenue agreement process is complex, internal stakeholders had difficulties navigating the process and coordinating with other stakeholders. Where contact information was not kept up to date in the Business Intake Tracking System or in the Government Electronic Directory Service, stakeholders had difficulty reaching required officials.
3.2 Internal stakeholder engagement
Audit Criterion: Defined roles and responsibilities are being followed to produce timely and consistent information.
It was expected that defined roles and responsibilities were being followed as intended. For example, that service lines were appropriately engaged in requests, and that notifications were sent by the service line upon completion of service requests.
A sample of business requirements documents was reviewed to determine if service lines were engaged in client requests. Most services were defined in the business requirements document and the enterprise price estimation tool, however, enabling services (such as Platform Services) were not clearly identified in the business requirements document. The Platforms Services Directorate is an enabling service that supports database, hosting and middleware. The Platforms Services service delivery model was not uniformly delivered across customer departments, and the Platform Services team was not consistently engaged in the completion of customer business requests. We found updates to the Enterprise Price Estimation tools were being made, however these updates were not being approved and released on a timely basis.
Upon completion of a service request, SSC’s Guide on Revenue Allocation requires the responsible service line to notify the Client Executive team that the work is complete. A sample of 40 transactions was tested to determine if service lines were notifying Client Executives when the requested work was complete. Notifications of work completion were found in only 4 of 40 agreement files. Follow up interviews indicated multiple channels were being used to notify service delivery managers of the completion of the service including: emails, phone calls, meetings, change management requests, and project status updates. These inconsistent notification methods pose a risk that requests are not being marked as complete and subsequently charged to the customer in a timely manner.
3.3 Reporting
Audit Criterion: Customer revenue agreement results are regularly and appropriately communicated to senior management.
It was expected that SSC senior management was receiving regular updates on the status of customer revenue agreements. The audit team found that reports were regularly provided to stakeholders and to senior management. The audit team tested a sample of 33 transactions presented in the Enterprise Business Intake and Demand Management Dashboard which matched information contained in the Business Intake Tracking System. Updates on the status of customer revenue agreements were regularly presented to SSC senior committees. Regular reporting on the status of business requests, high risk agreements (unsigned and high dollar) and forecasted revenue was provided to the Senior Assistant Deputy Minister Service Delivery Management Branch.
Recommendation 3
High priority
The Senior Assistant Deputy Minister Service Delivery and Management Branch should:
- clarify roles and responsibilities to stakeholders
- develop and implement a process for SSC service lines to notify Client Executives when customer requests are completed
Management response
Management agrees with this response. An updated Responsible Accountable Consulted Informed matrix will clarify roles and responsibilities regarding agreements including notification of work completion from Service Lines to Client Executive to update Business Intake Tracking System.
Recommendation 4
Medium priority
The Senior Assistant Deputy Minister Chief Financial Officer Branch should ensure the Enterprise Price Estimation tool is updated in a timely manner.
Management response
Management agrees with the recommendation. To ensure future releases of the Enterprise Price Estimation Tool (ePET) are timely, and this audit recommendation is fully satisfied, and annual release schedule (including a communication plan) will be established.
4. External communication
Line of Enquiry 4: SSC effectively communicates with customers.
4.1 Tools and processes
Audit criterion: There are consistent tools and procedures for customer communication.
It was expected that consistent tools such as templates and service costing strategies were used to engage customers.
The audit found that costing practices among service lines were inconsistent. Some service lines required customers to sign agreements for on-going costs prior to the installation of new services while others waited until initial agreements expired for approval of on-going costs. The inconsistency of practices resulted in delays in the negotiation of new agreements and difficulties in tracking on-going renewal costs at SSC. Customers also had difficulty tracking and budgeting on-going costs.
4.2 Customer engagement
Audit criterion: SSC has effective two-way communication with customers.
It was expected that:
- SSC met regularly with customers
- there were effective communication channels to receive and address customer concerns
- agreements and invoices were clear and communicated effectively to customers
The audit team met with representatives of four SSC customer organizations, all of which praised the relationships with Client Executive teams. The audit found that there is regular communication between customers and Client Executive teams, and that customers felt the teams worked hard to resolve issues.
Year-end deadlines were clearly communicated but some services had reached their annual capacity long before the year-end deadline. SSC is working with Treasury Board to develop tools to help better align work requests received by SSC with established government-wide priorities. A pilot is being developed, and although a completion date is yet to be determined the resulting tools will help SSC to improve its demand forecasts.
Agreements and invoices were missing critical information, such as work and cost details. Customers were thus unable to identify the work they were being invoiced for and to reconcile charges with services rendered by SSC. Templates for agreements and invoices contained minimal client identifiers. Costing summaries were not consistently shared with customers. In many cases, total costs reported in the invoice did not match total costs in the customer revenue agreement. Commoditized services did not have an associated revenue agreement but were billed using similar processes and therefore added to existing difficulties in the reconciliation of costs to services. Client services could be improved by including additional details in agreements and invoices.
Recommendation 5
Medium priority
The Senior Assistant Deputy Minister Chief Financial Officer Branch should ensure that details on invoiced one-time and on-going service costs are available and accessible for internal users and clients.
Management response
Management will continue to communicate the costing process for all services to internal stakeholders.
Management agrees with the second part of this recommendation related to ensuring that details on invoiced one-time and on-going services costs are available and accessible for internal users and clients.
Recommendation 6
Medium priority
The Senior Assistant Deputy Minister Service Delivery and Management Branch should:
- ensure that changes to agreement costs are proactively communicated to clients
- incorporate the following in customer revenue agreements:
- the location of the work
- summary pricing estimates
- service descriptions
Management response
Management agrees with this recommendation. Clients will be notified of costing changes and the Business Requirements Document and Customer Revenue Agreements will incorporate the additional information suggested in this recommendation.
C. Conclusion
In my opinion, as the Chief Audit Executive, overall SSC had adequate and effective processes for the establishment of customer revenue agreements, however, overall SSC did not have adequate and effective processes for the renewal and management of these agreements.
The audit found that SSC had adequate processes for some aspects of customer revenue agreements, specifically:
- policies and guidelines relating to the establishment of customer agreements were documented and available to stakeholders
- there was regular communication between customers and Client Executive teams and a belief by customers that SSC worked hard to resolve issues
- escalation procedures for the resolution of issues were generally effective
The following gaps may affect the timely and accurate recovery of customer revenues:
- some invoiced costs did not match the signed customer revenue agreement amounts, with no justification on file to support the revised costs
- due to weak controls for the identification and tracking of renewal of agreements, SSC may fail to recover all of its costs from customers
- there was a weakness in the internal communication at SSC with respect to agreement renewals and cancellations
The above may impact SSC’s ability to deliver reliable and secure services to customers and result in SSC incurring unnecessary on-going costs that would otherwise be recoverable.
These findings are important because 25% of SSC’s funding envelope depends on revenue generated from customer agreements. Customer revenue agreement processes should be improved so that SSC can effectively recover service costs from its customers in an accurate and timely manner.
Annex A – Lines of enquiry and audit criteria
| Category | Audit Criteria |
|---|---|
| 1. Management of agreements - SSC has adequate and effective controls and procedures to establish and manage customer revenue agreements. | |
| 1.1 Process Guidance | There are clearly defined and documented processes for the establishment and management of customer revenue agreements. |
| 1.2 Tracking of Agreements | There are processes and controls in revenue agreements to accurately track information. |
| 1.3 Implementation | The processes and controls related to the management of customer revenue agreements are implemented consistently in accordance with documented policies and guidelines. |
| 2. Renewals - Customer revenue agreements are renewed as required. | |
| 2.1 Tracking Renewals | There are processes and controls to accurately track renewals. |
| 2.2 Process Guidance | There are established processes, procedures and tools that support the agreement renewal process. |
| 2.3 Resolution processes | There are resolution processes in place to manage agreement disputes. |
| 3. Internal communication - SSC conveys accurate, consistent and timely information on customer revenue agreements to internal stakeholders. | |
| 3.1 Roles and Responsibilities | There are clearly defined roles and responsibilities for the development of the agreement and delivery of the services. |
| 3.2 Internal Stakeholder Engagement | Defined roles and responsibilities are being followed to produce timely and consistent information. |
| 3.3 Reporting | Customer revenue agreement results are regularly and appropriately communicated to senior management. |
| 4. External communication - SSC effectively communicates with customers. | |
| 4.1 Tools and processes | There are consistent tools and procedures for customer communication. |
| 4.2 Customer engagement | SSC has effective two way communication with customers. |
Annex B – Acronyms
| Acronym | Definition |
|---|---|
| BR | Business Requests |
| IT | Information Technology |
| SSC | Shared Services Canada |
Annex C - Audit recommendations prioritization
Internal engagement recommendations are assigned a rating by the Office of Audit and Evaluation in terms of recommended priority for management to address. The rating reflects the risk exposure attributed to the audit observation(s) and underlying condition(s) covered by the recommendation along with organizational context.
| Rating | Explanation |
|---|---|
| High priority |
|
| Medium priority |
|
| Low priority |
|
Page details
- Date modified: