Review of the Departmental Financial Management System Renewal Project: chapter 3


2. Objective, Scope and Methodology

Objective

The overall objective of this SUD Review was to assess the appropriateness of the project management approach used to implement SAP at ECCC.

Annex 1 of this report summarizes the criteria used to support review activities, consistent with CobiTFootnote1 and Deloitte’s approach to SUD reviews.

Scope

The scope of the review included the following components:

  • financial system and business process transformation;
  • business case alignment of project management (i.e. through the project plan, approach, communications and coordination of project activities);
  • implementation of the IT solution and change management (i.e. business process design, configuration and software development, user acceptance testing, data conversion and cutover, and management of technical IT infrastructure); and,
  • business and user readiness (i.e. business processes, data maintenance, access to SAP, user readiness and training).

The scope of the review included project phases and tasks defined in the Accelerated SAP (ASAP) project management methodology and approach, which provides a framework for the above-mentioned components (e.g. resource staffing and key roles, implementation into business areas, implementation into IT production and maintenance, and transition to support).

In addition, the scope of the review included steps taken to prepare the agreement that supports the ongoing operation of the host-client partnership between AAFC and ECCC.

The review also covered a follow-up on a recommendation from the Corporate Accountability and Administrative Renewal / Internal Control over Financial Reporting 2013 internal audit. The recommendation dealt with the timely finalization and approval of key project documents. Project documents inspected during the review included final versions of the DFMS Renewal Project Charter, DFMS Renewal Business Case, DFMS Fit-to-AAFC SAP Cluster Solution Assessment Summary Results (an analysis of the ability of AAFC’s SAP system to meet ECCCs functional business requirements), and DFMS Renewal Project Implementation Phase - Preliminary Business Transformation Strategy. All were found to have been approved and signed by senior management at ECCC.

To the extent that on-time completion of software configuration and testing activities were critical to ECCC’s project timelines, the work of the technical SAP team at AAFC was in scope of the SUD review. As well, project tasks that involved interaction between the DFMS Project Team and the AAFC SAP team regarding SAP implementation processes were included in the scope of the review. However, the scope did not include the review of project management activities at AAFC.

Methodology

A System Under Development (SUD) review is carried out during the design, development and installation of a software system (SAP in this case) and the transformation of business processes dependent on the system to be implemented. A SUD review typically focuses on project management practices and controls that are in place to ensure that the project’s outcomes are delivered on time, are of high quality and meet the expectations of the project’s sponsor and stakeholders.

The methodology used to conduct a SUD review also differs from the approach taken on other internal audits or reviews in its real-time approach to reporting its findings. Findings are documented as the conduct of the review proceeds, and reporting findings to management on a timely basis is important. A SUD review also differs from a SUD audit and other internal audits in the degree to which project management controls are examined; there is less formal control testing in a review. The review approach was adopted at ECCC to reduce the risk that DFMS Renewal Project deadlines and deliverables would be affected by the review. This approach was presented to ECCC’s External Audit Advisory Committee (EAAC) at its meeting in November 2014.

The review team pursued four broad lines of enquiry related to the execution of the DFMS Renewal Project and to ECCC’s readiness for the introduction of the SAP-based DFMS on April 1, 2015. These lines of enquiry were: Project Management and Governance, Functional Readiness, Deployment Readiness, and Business Process Readiness.

Based on evidence gathered through the examination of documentation and interviews with project stakeholders, review criteria were assessed and a conclusion for each criterion was determined. Where a significant gap existed between expected findings and the observed practice, the risk of the gap to the success of the project was evaluated, and recommendations for improvement were documented.

The review team prepared and submitted to management two interim reports which included findings and recommendations to date: the first was submitted in December 2014 and the second in February 2015. Fact sheets were used to advise senior management of findings and recommendations, allowing corrective action to be taken by the project team as soon as possible. Findings and recommendations reflect work undertaken until the end of the Conduct Phase of the review (March 2015).

A briefing on the contents of the interim reports was presented to the EAAC at its meeting in March 2015.

Statement of Conformance

This review conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program, and applied in the context of a review.

In our professional judgement, sufficient and appropriate procedures have been conducted and evidence gathered to provide reasonable assurance of the accuracy of the conclusions reached and contained in this report. However, the controls were not tested and the conclusions were based on a comparison of the situations as they existed at the end of the fieldwork (March 2015) against the review criteria.

Page details

Date modified: