Audit of Project Cost Management

Executive summary

In alignment with its mandate, SSC develops and implements IT projects, including projects for the transformation and consolidation of government-wide IT infrastructure. Project cost management includes the processes involved to estimate costs, determine the budget and establish an authorized cost baseline, and control costs by monitoring project costs versus spending and managing changes to the cost baseline. Given the large number of SSC-led IT projects, the significant dollar values of these projects and the amount of resources tied to them, ensuring that project financial considerations are adequately addressed and provisioned for is of high importance.

The objective of this audit was to provide assurance that governance, risk management and control processes for the cost management of SSC-led projects are in place and working effectively.

Overall findings are as follows:

The audit team has developed recommendations to remediate audit findings. To address these recommendations, management will develop Management Action Plans.

Begonia Lojk
Acting Chief Audit and Evaluation Executive

A. Introduction

1. Background

Project cost management includes the processes involved to plan and estimate costs, determine the budget and establish an authorized cost baseline. It also includes processes to control costs by monitoring project costs and managing changes to the cost baseline. The planning and estimation of costs for projects may include processes that establish the policies, procedures and documentation for the planning, managing, expending, and controlling project costs. It can be defined further to include the following:

Project cost management is important because cost information helps managers at all levels understand the financial impact of the decisions they make and the initiatives they propose. Therefore, the main objective of project cost management is to ensure that project costs remain within budget and that exposure to the risk of cost overruns is adequately mitigated. To this end, SSC has developed project-related cost management processes and tools.

2. Rationale for the audit

SSC has several ongoing IT projects which involve a significant portion of resources (human resource, level of effort, professional services, IT investment, and so on). Developing project costs, allocating and maintaining budgets and managing costs (for example, monitoring and reporting costs and assessing deviations) are critical to ensuring that there are resources available for implementing the project. Examining project costing, budgeting and baselining along with cost monitoring, change control and reporting, is necessary to examine potential causes of cost discrepancies at any point in the project lifecycle, whether due to unrealistic initial costing and budgeting or inadequate project cost monitoring, controlling and reporting. Given the large number of SSC-led IT projects, the significant dollar values of these projects and the amount of resources tied to them, ensuring that project financial considerations are adequately addressed and provisioned is of high importance.

3. Audit authority

The audit was included in the 2018-19 Annual Update of the 2017 to 2020 Risk-Based Audit Plan, which was recommended by the Department Audit Committee and subsequently approved by the President on June 20, 2018.

4. Objective of the audit

The objective of this audit is to provide assurance that governance, risk management and control processes for the cost management of SSC-led projects are in place and working effectively.

5. Scope

The scope of this audit includes all processes pertinent to the cost management of SSC-led projects that have passed Gate 3 and started between April 2016 and June 4, 2018, per the Project Management Centre of Excellence SSC-led Master List. Refer to Annexes B, C and D for more details on the SSC project gating process. This audit examined the project cost development, budgeting and baselining process along with cost monitoring, change control and reporting.

The determination of amounts to be charged or cost-recovered from Partners by SSC and the derivation of IT costs supporting partner technological initiatives are out of scope for this audit.

6. Methodology

This audit assessed the processes in place for developing and monitoring project costs for a sample of SSC-led projects. The audit criteria are based on industry best practices in the areas of internal controls and project management identified in Annex A. The following audit procedures were carried out to test the adequacy and effectiveness of SSC’s cost management processes:

For HCCS, the audit team only examined financial monitoring and reporting. Although the HCCS project was costed in 2012 using a different process, it was selected for review due to some concerns regarding the financial health of this project. For the EVAS project, the audit team only examined the cost development processes since it had not reached Gate 3, the gate at which costs are reviewed and monitored.

7. Statement of conformance

In my professional judgment as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the opinion provided and contained in this report. The opinion is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed on with management. The opinion is applicable only to the entity examined. The engagement was conducted in conformance to the requirements of the Policy on Internal Audit, its associated directive, and the Internal Auditing Standards for the Government of Canada and Code of Ethics. The evidence was gathered in compliance with the procedures and practices that meet the auditing standards, as corroborated by the results of the quality assurance and improvement program. The evidence gathered was sufficient to provide senior management with proof of the opinion derived from the internal audit.

B. Findings, recommendations and management response

1. Costing process methodology

Audit criterion: An adequate costing process is in place and working effectively.

Establishment of a standardized costing process is integral to promote consistent and accurate costing across projects. Through interview discussions and review of project documentation, it was expected that SSC had a documented costing process. This includes:

Finding: SSC does not have a documented and consistent costing process.

Upon examination, it was noted that SSC created the Financial Strategies and Costing Group in 2016, which reviews project costing prior to project baselining, however, it was noted that a documented process for project costing did not exist.

Without a documented process to cost projects, project costs may be incomplete and/or inaccurate and may result in unexpected changes to costs (that is, surpluses or deficits). This may impede effective organizational decision making. The audit team noted several instances where projects were costed based on high-level designs and experienced significant changes to costs during execution.

In addition, the absence of a documented costing process may lead to un-supported project costs and governance committees approving cost figures without a sufficient amount of detail. A judgmental sample of $69.6 million in costs associated with the six projects was selected for review. While all project teams were able to provide an explanation for how cost line items were developed, only one out of six projects was able to provide documented support for all cost line items sampled by the audit team. One other project (Enterprise Perimeter Security), lacked significant supporting documentation in the amount of $11.7million. Without documented evidence for cost items, project teams and governance committees are unable to effectively challenge, and update cost figures based on new information as the project progresses.

Furthermore, a contingency value represents funds requested to cover cost estimate uncertainty. The audit team expected to find that a documented process to consistently establish contingency values across projects was in place.

Finding: SSC does not have documented criteria and procedures to develop contingency values for projects.

Upon examination, the audit team observed the following with respect to application of contingency values to projects:

Without a documented process to establish project cost contingency values, SSC may not be able to effectively plan for investment decisions and fund unanticipated changes to project costs.

The Project Management and Delivery Branch Operating Guide outlines that project budget, scope and timelines are developed up until project baselining at Gate 3. Service lines, project managers, Cost Management Advisors, Financial Management Advisors, Chief Financial Officer and project owners must be involved in project costing prior to project baselining at Gate 3. The audit team expected to find evidence of involvement of the Cost Management Advisors, service line, project managers, Financial Management Advisors and Chief Financial Officer in project cost development.

Finding: Key individuals are involved in the costing process. Some exceptions were noted.

Upon conducting documentation review and interviews, the audit team observed that six out of six projects involved service lines for input and feedback on preliminary project costs through attestation or project baselining at Gate 2 or 3. The audit team also observed the following:

Cost Management Advisors

The Project Management and Delivery Branch Operating Guide specifies that project costs must be reviewed/approved by the Cost Management Advisor prior to Gate 3 approval. The audit team observed that in four out of the six projects the Cost Management Advisor reviewed the project costs prior to Gate 3. For the remaining two projects, the Cost Management Advisor role was not implemented prior to Gate 3.

The Project Management and Delivery Branch Operating Guide requires Cost Management Advisor review of Task Financial Authorizations from Gates 0 to Gate 2, however, based on interviews and discussions, the audit team noted that the Cost Management Advisors do not review the Task Financial Authorizations.

Project Manager

The Project Management and Delivery Branch Operating Guide explains that project managers are accountable for developing project costs and securing project funding, which occurs at Gate 2. For three of the projects, the project managers were involved in cost development prior to requesting project funding. For the remaining three projects, project managers were assigned under an older process, where project managers were involved later in the budgeting and costing process. In the latter case, interviews and discussions with project managers informed the audit team that they often had difficulty understanding how budgets and costs were developed and the assumptions underlying the interrelationships between cost elements and how they impact the overall project cost.

Financial Management Advisors

The Project Management and Delivery Branch Operating Guide and other project financial management documentation require the Financial Management Advisor to sign the financial situation report which includes the monthly project and assign an Internal Order Code prior to Gate 2. Through interviews and documentation review, the audit team observed that Financial Management Advisors signed off on monthly branch forecasts that relate to all six projects. Additionally, all projects included an Internal Order Code on their Gate 2 presentation.

In conclusion, SSC has not implemented a documented and consistent costing process.

Recommendation 1

High priority

We recommend that the Senior Assistant Deputy Minister Chief Financial Officer Branch establish and document a cost development framework for projects that includes, but is not limited to, the following areas:

  • an overall costing process/ methodology for SSC led projects
  • ensure that the required stakeholders are involved in the costing process
  • requirement to identify costing assumptions and data sources
  • a consistent approach to determine the value of contingencies
  • requirement to collect and retain substantiating evidence for cost figures
Management response

Management agrees with the recommendation. SSC has already taken action to address this recommendation by developing a Directive on Costing. In addition, a Guideline to Costing will be developed.

2. Budgeting process

Audit criterion: Projects are supported by an effective budgeting process, establishing cost baseline and funding requirements.

The Project Management and Delivery Branch Operating Guide describes the project approval process to establish project budgets, including cost baseline (completed upon Gate 3 approval) and funding requirements.

As at March 2018, the following approvals are required:

Finding: Projects obtained governance approvals when baselining costs.

The audit team noted that all six projects obtained approval or conditional approval from the Project Management Board at Gate 2 and all six projects identified the source of funds that covered total project costs. Five of the six projects reviewed by the audit team did not require Financial Investment Management Board (FIMB) approval as this Committee had not been created (FIMB approval required as of March, 2018). Only one of the six projects (Endpoint Visibility Awareness and Security) required and obtained FIMB Gate 1 approval. Another project required and obtained Financial Investment Management Board approval for a change request.

The project financial management documentation explains that projects are required to obtain approval on monthly forecasts by the delegated manager. Projects are required to have final forecasts approved by the Director General and the Senior Assistant Deputy Minister. The audit team expected to find evidence of stakeholder sign-offs on monthly and final financial forecasts.

Finding: Projects obtained approvals for financial forecasts.

During examination, the audit team observed that all six projects provided evidence of monthly financial forecast sign-off by the Assistant Deputy Minister and Financial Management Advisor. Conversely, Branch officials explained that the final project forecast is approved at the Project Management Board. The audit team also noted that all projects provided evidence that the Assistant Deputy Minister (or a representative) was present at the Gate 3 meeting, where projects were baselined and a forecast was presented.

In conclusion, projects are supported by an effective budgeting process, establishment of a cost baseline and funding requirements.

3. Standardized costing tool

Audit criterion: Standardized costing tools are in place.

The Treasury Board Secretariat Guidelines on Costing includes a standardized cost base template along with costing checklists. The Guidelines explain that the use of the templates and checklists will promote a transparent, high quality and accurate costing exercise. To enhance the credibility of the costing exercise, data sources and assumptions for cost figures should be documented. The audit team expected that a standardized costing tool and checklist would exist to capture the full cost of projects.

Finding: A standardized costing tool does not exist at SSC.

Upon examination, the audit team observed that for all six projects examined, project costing was primarily conducted through use of non-standardized, project-created excel spreadsheets, along with use of various tools (for example, Enterprise Price Estimating Tool, Task Financial Authorizations and other tools) for cost components.

Not using a standardized tool for project costing, may lead to incomplete or inaccurate costing of projects, as projects may not fully consider elements, including but not limited to, ongoing costs, one-time costs, contingency values, funding sources. Incomplete costs may be presented to governance committees. For example, for two out of five projects, the Project Management Board required the project teams to reconfirm costing with the Deputy Chief Financial Officer at Gate 3 to ensure inclusion of all elements. Inclusion of the missing cost elements prior to the governance committee reviews would have increased efficiency as the project teams progressed through governance.

Based on interviews and discussions, it was noted that the Cost Management Advisors are in the process of developing a costing tool which includes the data sources, documenting assumptions, timelines and various cost categories.

In conclusion, a standardized project costing tool is not in place and this may lead to inaccurate and incomplete costing of projects.

Recommendation 2

High priority

We recommend that the Senior Assistant Deputy Minister Chief Financial Officer Branch, in consultation with the Senior Assistant Deputy Minister Project Management and Delivery Branch, implement a standard costing tool to ensure that key costing information is captured. This should include the following:

  • data sources
  • assumptions
  • ongoing costs
  • one-time costs
  • contingency values
  • funding sources
Management response

Management agrees with the recommendation. SSC has developed a Project Cost Estimating tool that is currently being finalized that will ensure that key costing information is standardized and captured.

4. Cost training requirements

Audit criterion: Cost training requirements are in place.

Cost training is important to ensure that methodologies are consistently understood and applied across projects, promoting accuracy in the cost process. In addition to having the necessary costing tools, templates, and processes in place, stakeholders must be involved in the costing process. The audit team conducted interviews with individuals involved in the costing of the six projects and expected that cost training requirements were implemented.

Finding: Cost training requirements are not in place.

Upon examination, the audit team observed that there is no mandatory cost training required at SSC. Costing knowledge is obtained via on the job training, enrollment leading to professional designations (Chartered Professional Accountant, and Project Management Professional) and other external training.

Without mandated cost training requirements, SSC may apply inconsistent methodologies to develop and review project costs.

In conclusion, cost training requirements are not in place.

Recommendation 3

Medium priority

We recommend that the Senior Assistant Deputy Minister Chief Financial Officer Branch, in consultation with the Senior Assistant Deputy Minister Project Management and Delivery Branch, implement and communicate project costing training requirements for all stakeholders involved in project costing, which may include but not limited to:

  • Cost Management Advisors
  • Financial Management Advisors
  • Project managers
  • Service line personnel
  • Business analysts
Management response

Management agrees with the recommendation. Over the coming year, SSC will establish and implement a training strategy that will provide key stakeholders with the necessary knowledge to execute on their responsibilities.

5. Cost monitoring

Audit criterion: Shared Services Canada has an adequate and effective cost monitoring process in place to support timely identification and resolution of financial issues.

According to the Project Management and Delivery Branch Operating Guide, monthly Earned Value is used to identify financial issues through identification of financial variances (over or under the project baseline). A resolution is reached to address the identified variances in the project costs. In this case, budget surpluses or deficits are addressed through a change request. Earned Value reporting includes metrics to identify budget surpluses and deficits, such as the Variance at Completion value. Variance at Completion is the expected variance at the end or the completion of the project. It is the variance between the total budget for the project and the project costs to complete the project (which includes the project costs incurred to date and estimated costs to complete the project). Variance at Completion is the project budget surplus or deficit that is expected at the completion of the project.

Variance at Completion is calculated based on the actual expenditures to date, plus the forecasted cost to complete the project and compares it with the total budget for the project. As such, the project manager is required to provide a rationale for the Variance at Completion and present it to the Director General. A change request form must be completed when a project requires a change to project baselined budget, timeline or scope.

The audit team expected that projects would report Earned Value on a monthly basis and that where variances exist (that is, a Variance at Completion value), a documented rationale would explain the variance.

In addition, it was noted that project managers did not use SIGMA as the only source for EV reporting, as it did not contain the most up-to-date expenditure information, mainly as a result of employees not entering this information into SIGMA on a timely basis. As a result, project managers used spreadsheets to manually keep track of additional expenditures that were yet to be posted to and approved in SIGMA.

Finding: An effective and adequate cost monitoring process is in place to identify and resolve financial issues. Financial variances have explanations, with some exceptions.

The audit team observed that over the 10-month period tested (between October 2017 and July 2018) that all projects reviewed were documented and explained the Variance at Completion in the earned value reports. In cases where no documented rationale existed, all projects were able to explain the rationale for the Variance at Completion value with one exception – the HCCS project. This project had one Variance at Completion value for which no specific explanation had been documented. In this instance, the project manager provided only a high-level explanation, to the audit team for why Variance at Completion values existed.

Incomplete documentation that does not explain a Variance at Completion budget surplus/deficit within Earned Value reports may indicate that these values are not being discussed in adequate detail. In conclusion, an effective and adequate cost monitoring process is in place at SSC to identify and resolve financial issues and financial variances have explanations, with some exceptions.

Recommendation 4

Medium priority

We recommend that the Senior Assistant Deputy Minister Project Management and Delivery Branch enforce the effective and adequate financial monitoring process by ensuring that variances outside the authority of the project manager are documented and explained through a Change Request presented to governance for approval in a timely manner.

Management response

Management agrees with the recommendation that variances outside the authority of the Project Manager are documented and explained through a Change Request presented to governance for approval in a timely manner.

6. Project change requests

Audit criterion: Shared Services Canada has an adequate and effective process in place to manage changes to project costs.

The Project Management and Delivery Branch Operating Guide explains that when a change in budget, timeline or scope from parameters specified in Gate 2 is required the following actions are required:

Finding: The change request process is not consistently followed.

The audit team reviewed 12 change requests associated with the projects sampled. All change requests were approved by the Project Management Board and were assessed to determine impacts on other projects. The executive sponsor (or a designate) was present at the time of change request approval for all 12 change requests.

The audit team observed that although change requests include a signature line for the executive sponsor, project manager and Financial Management Advisor, no change requests included all the required signatures on the document.

The audit team noted five out of eight change requests created after Gate 4 were reviewed by a Cost Management Advisor. Additionally, for 9 out of 12 change requests, the change request listed the name of the business sponsor (for example, Director General) instead of the executive sponsor (for example, Assistant Deputy Minister). The lack of change request approvals may result in absence of rigour over the change request values process and in turn may lead to an inefficient use of SSC financial resources. Personnel from the Financial Strategies and Costing Division noted that change requests are often approved by the Project Management Board in large batches, which often does not allow sufficient time to ensure a thorough scrutiny of these items.

In conclusion, while a process is in place to manage changes to project costs, this process is not consistently followed.

Recommendation 5

Medium priority

We recommend that Senior Assistant Deputy Minister Project Management and Delivery Branch document and enforce the change request approval requirements for all projects.

Management response

Management agrees with the recommendation. Effective February 2019, SSC has implemented through its Project Management and Delivery Branch Operating Guide update an adequate and effective process to manage changes to project costs which has been approved by governance and communicated widely.

7. Financial reporting tools

Audit criterion: Financial reporting tools are reliable, accurate, and provide sound information for decision making.

In May 2017, SSC released project management cost monitoring tools, such as Earned Value reporting, to strengthen project oversight and visibility. SIGMA is used as SSC’s financial system of record. The audit team expected that this tool would be fully implemented and be useful to enable sound decision making.

Finding: SSC’s financial system, SIGMA, is used along with manually tracked excel spreadsheets for cost monitoring and reporting.

All six projects selected used SSC-provided tools and templates for cost monitoring. Additionally, all six projects sampled used SIGMA as a system to track budgets against actual expenditures, however, there is still a significant reliance on manually tracked excel spreadsheets to capture all expenditures due to information not be entering in SIGMA on a timely basis. The audit team identified no significant errors or inability to complete reports as a result of the use of the current suite of tools to manage costing information.

In conclusion, while reliable, accurate financial reporting tools are in place, stakeholders such as project managers, Financial Management Advisors and Cost Management Advisors, identified areas for improvement with respect to manual processes.

C. Conclusion

The audit team has concluded that governance, risk management and control processes for cost management of SSC-led projects are in place. Shared Services Canada has project management tools in place that are aligned with industry best practices and are being followed for financial reporting and monitoring. There is an effective and adequate cost monitoring process in place to identify and resolve financial issues and financial variances. In addition, the SSC Project Management Operating Guide has documented key processes related to project cost management to allow for a consistent, repetitive and disciplined approach for project cost management and reporting. Finally, the SSC Project Governance Framework requires that all SSC-led IT projects obtain senior management oversight and approval to ensure transparency and accountability in regard to project budgets, fund allocation and financial management.

There are some opportunities for improvement, specifically the need for a documented costing process/methodology, active engagement of key stakeholders in the costing process, the application of a consistent approach to the determination of contingency values, the identification of costing assumptions and data sources, and increased cost training for staff.

Annex A – Specific lines of enquiry and audit criteria

Globally accepted industry best practices in the areas of IT controls, project management and public sector management were used for this audit. The frameworks include the Project Management Body of Knowledge, COBIT 5 and the Management Accountability Framework. COBIT 5, created by the Information Systems Audit and Control Association, is a holistic, internationally-accepted framework that aids organizations in aligning objectives with the governance and management of IT, encompassing full end-to-end business functions. The Project Management Body of Knowledge is the Project Management Institute’s flagship publication and is the fundamental resource for effective project management in any industry. The Management Accountability Framework, used by the Treasury Board Secretariat as an oversight tool, establishes expectations for sound public sector management practices and performance.

The aforementioned frameworks were leveraged to develop audit criteria. The following audit criteria were used in the conduct of this audit:

Audit Criteria Criteria Description
1. Costing process An adequate costing process is in place and working effectively.
2. Project budgeting Projects are supported by an effective budgeting process, establishing cost baseline and funding requirements.
3. Costing tools Costing project tools are in place and working effectively.
4. Training Costing training is in place, adequate and working effectively.
5. Project cost monitoring and oversight SSC has an adequate and effective cost monitoring process in place to support timely identification and resolution of financial issues.
6. Project change requests SSC has an adequate and effective process in place to manage changes to project costs.
7. Financial reporting tools Financial reporting tools are reliable, accurate, and provide sound information for decision making.

Annex B – Elements of the cost management process

1. Estimating project costs

Developing an approximation of the monetary resources needed to complete the project work.

2. Determining project budget

Aggregating the estimated costs of individual activities or work packages to establish an authorized cost baseline.

3. Controlling costs

Monitoring the status of the project to update the costs and manage changes to the cost baseline.

Annex C - Timeline of key elements of the cost management process

The diagram below outlines a timeline for the introduction of key elements of the costing process. This timeline was considered when we tested projects against process requirements.

Long description

The diagram depicted outlines a timeline for the introduction of key elements of the cost management process. The time line takes place 2016 to 2018 and is broken up by year. The first point occurred in April 2016 with the introduction to Project Management Branch. The following point occurred January of 2017, which was the beginning of project reviews with Cost Management Advisor. Following that event in May 2017, Earned Value and Task and Financial Authorization were introduced as well as the first version of the Operating Guide. In March of 2018 Finance and Investment Management Boardwas introduced. The last point on the timeline occurs April 2018 with the release of the second version of the operating guide.

Annex D - Project governance framework gates

The project governance framework has been recently modified to include six project gates, as presented in the diagram below. Key governance committees involved in the process are as follows:

Long description

This image depicts the breakdown of the different Project Governance Framework Gates. Staring on the left with Gate 0, moving to the right in consecutive gate numbers and ending on the right with Gate 6. Gate 0 is the business intake. Gate 1 is idea generation. Gate 2 is initiation. Gate 3 is planning. Gate 4 is operational readiness. Gate 5 is Deployment. And the final gate, gate 6 is Closeout.

Annex E - Audit recommendations prioritization

Internal engagement recommendations are assigned a rating by OAE in terms of recommended priority for management to address. The rating reflects the risk exposure attributed to the audit observation(s) and underlying condition(s) covered by the recommendation along with organizational context.

Recommendations legend
Rating Explanation
High priority
  • Should be addressed as priority for management (that is, within the next six to 12 months)
  • Controls are inadequate. Important issues are identified that could negatively impact the achievement of organizational objectives
  • Could result in significant risk exposure (for example, reputation, financial control or ability to achieve Departmental objectives)
  • Provide significant improvement to the overall business processes
Medium priority
  • Should be addressed over the next year or reasonable timeframe
  • Controls are in place but are not being sufficiently complied with. Issues are identified that could negatively impact the efficiency and effectiveness of operations
  • Observations could result in risk exposure (for example, reputation, financial control or ability of achieving branch objectives) or inefficiency
  • Provide improvement to the overall business processes
Low priority
  • Changes are desirable within a reasonable timeframe
  • Controls are in place but the level of compliance varies
  • Observations identify areas of improvement to mitigate risk or improve controls within a specific area
  • Provide minor improvement to the overall business processes

Page details

Date modified: