Follow-up on Internal Audit: Halifax-class Modernization/Frigate Equipment Life Extension (HCM/FELEX) Project

December 2013

7050-11-48 (CRS)

Reviewed by CRS in accordance with the Access to Information Act (AIA). Information UNCLASSIFIED.

Caveat

The result of this work does not constitute an audit of the HCM/FELEX Project audit. Rather, this report was prepared to provide reasonable assurance that the Management Action Plans (MAP) that resulted from the March 2011 HCM/FELEX project audit were implemented as stated and as such have addressed the associated recommendations.

Acronyms and Abbreviations

Introduction

In keeping with the Treasury Board Policy on Internal Audit,¹ Chief Review Services (CRS) is required to undertake audit follow-ups to assess the implementation status of MAPs developed in response to previous CRS audit recommendations. In accordance with the CRS Risk-based Audit Plan for fiscal year 2013/14 to 2015/16, this audit follow-up was selected to ensure that some systemic capital acquisition process issues, as well as concerns specific to the HCM/FELEX project have been addressed.

In March 2011, CRS completed an audit of the $2.8-billion mid-life upgrade of the 12 Halifax-class ships first delivered between 1991 and 1996. The objective of the audit was to provide assurance that the HCM/FELEX project has effective governance, risk management and control frameworks in place to ensure a cost-effective and timely operational capability.

The audit concluded that to mitigate the potential loss of operational capability and ensure the timely delivery of all ships, the Project Management Office (PMO) needs to implement better risk management and control processes. Nevertheless, some innovative management practices were observed. The HCM is governed by a Committee of Sponsors that includes Assistant Deputy Minister (Materiel) (ADM(Mat)), Chief of the Naval Staff (C Navy), Assistant Deputy Minister Acquisitions from Public Works and Government Services Canada, and the Chief Executive Officer from each of the principal contractors. Regarding contract management, the final holdback in the Combat System Integration contract will be released after the warranty period to provide an incentive to the contractor to fulfill warranty obligations. As well, the project Combat System subsection risk management plan complied with Department of National Defence (DND) guidance and was more advanced than the project risk management plan in order to mitigate integration risk. The findings of the 2011 audit are summarized as follows:

Project Schedule. Planned delivery of the 12 modernized frigates within a period of 75 months could pose schedule risks due to potential slippage in mid-life-refits, the Tiered Readiness Program and delays in Combat System Integration.

Contract Basis of Payment. The Combat System Integration contract basis of payment did not provide optimal value for the Crown. DND allowed a reduction of $48 million in holdbacks during the project’s Request for Proposals amendment process.

Cost and Schedule Control. The Defence Resource Management Information System, the departmental Enterprise Resource Planning system for material acquisition and support, was not being fully utilized due to system shortfalls.

Requirements Traceability. Not all requirements could be traced in the Dynamic Object Oriented Requirements Suite tool used by the HCM/FELEX project, and there is no policy on the use of the Dynamic Object Oriented Requirements Suite in DND.

Navy Warehouse Space. Warehouse construction and lease costs of up to $25 million could be avoided if supply managers reviewed and disposed of dormant inventories consuming warehouse space at the time of the audit.

Project Cost Estimates and Validation. The Project Approval Guide did not require capped Requests for Proposals to be cost-validated by the Assistant Deputy Minster (Finance and Corporate Services) (ADM(Fin CS)) until the project expenditure approval stage, and HCM/FELEX project released a capped Request for Proposals to industry before project approval.

Risk Management. The project Risk Management Plan was not in accordance with the departmental Integrated Risk Management Policy, and the project risk management practices were not effective.

In order to address these issues, eight recommendations were put forward (see Annex A). The Vice Chief of the Defence Staff (VCDS) and ADM(Mat) developed a set of MAPs to address the audit findings specific to their area of responsibility.

Methodology

This audit follow-up is based on a review of documentation and evidence to assess the progress made in implementing the MAP. The following methods were used:

• Analysis of data from the Audit and Evaluation Recommendation Management System (AERMS)² and the Capability Investment Database;
• Interviews with personnel from the HCM/FELEX project office and the VCDS and ADM(Mat) staff; and
• Review of key project documentation and policies pertaining to the MAP.

Statement of Conformance

The audit follow-up conclusions contained in this report are based on sufficient and appropriate audit evidence gathered in accordance with procedures that meet the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing. The audit follow-up thus conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program. The opinions expressed in this report are based on conditions as they existed at the time of the audit follow-up, and apply only to the entity examined.

Overall Assessment

The follow-up on the March 2011 audit identified that progress has been made in some areas. For example, there have been improvements in project risk management, human resource forecasts and the Senior Review Board endorsement of capped Requests for Proposals and contract cash flow.

At the time of follow-up, there were nine of eleven MAPs designated as fully implemented in the AERMS. Upon further examination, two of eleven MAPs required additional measures to fully address the issues. As for those MAPs that have yet to be fully addressed at the time of the follow-up, the majority will be implemented by 2014 due to the systemic nature of the issues. An implementation status of the MAP items can be found in Annex A.

Until all MAPs are fully implemented, there remain risks in the following areas:

• Requirements Traceability. The absence of a departmental policy or guidance on the use of the Dynamic Object Oriented Requirements Suite tool to trace project requirements to contract specifications and deliverables will make it difficult to ensure that all combat system requirements are met by the contractor.
• Navy Warehouse Space. Dormant inventories are currently consuming significant amounts of warehouse space in Halifax and Esquimalt. Additional warehouse construction and lease costs will have to be incurred in order to accommodate stripped-out equipments and new material for HCM/FELEX unless surplus stock is disposed of.

For the purpose of monitoring the implementation of the two action plans that have not yet been fully implemented, the AERMS status of these action plans will be revised by CRS to reflect the results of this audit follow-up.

Annex A—Summary of Management Action Plan Status

Project Schedule

Management Action Plan and AERMS Status

C Navy has reviewed the schedule criteria and has set priorities as follows:

1. ▓▓▓▓▓▓▓▓▓ Halifax-class ships will always be available per coast.
2. The mid-life refit schedule will be aligned with the 48-month refit.
3. Tiered Readiness Program overlap on the same coast is to be avoided if possible.
4. As closely as practical, the oldest of the class will be refitted first and the newest last.
5. All 12 mid-life refits will be completed within an 81-month period.

Recognizing that priorities are not static in time, should schedule slippage occur in the future, these priorities will be balanced with operational requirements of the fleet and institutional capacity in terms of size and composition of the fleet, with the industry capacity as well as the scope and funding constraints placed on the HCM/FELEX PMO.

Through the HCM governance structure, the PMO is conducting ongoing oversight of these criteria during the refits utilizing the integrated schedule tool jointly developed and managed by C Navy formations, and the three principal contractors. When major scheduling issues arise which require the current refit schedule to be significantly altered, the PMO will revisit all relevant criteria and conduct the necessary sensitivity analyses. The five criteria that focus only on the operational concerns will be balanced with other cost and schedule impacts as well as the impact on other key stakeholders and their activities related to the refits.

OPI: C Navy and ADM(Mat)/Director General Maritime Equipment Program Management (DGMEPM)/HCM FELEX PMO

Target Date: Ongoing

AERMS Status: Full implementation

Assessment of Progress by CRS

The project office did set priorities for the schedule criteria but at the same time recognized that the order of priority may not remain static depending on the operational requirements and other constraints at any particular time. The project did not perform sensitivity analysis. Instead the HCM/FELEX PMO, C Navy and Canada Industry governance structure was used to address schedule slippage.

In September 2012, the Senior Review Board³ revised the Halifax-class ship availability to ▓▓▓▓▓▓▓▓▓ ships on the east coast and ▓▓▓▓▓ ships on the west coast in the Restricted Readiness/Tiered Readiness Program.⁴ The ship availability was revised due to delays in the Tiered Readiness Program schedule. The first two ships are taking more than the 12 months initially planned⁵ and two of the follow-on ships will require two months more than the nine months initially planned for their Tiered Readiness. The most recent fleet plan estimated the time of completion for the twelve HCM/FELEX mid-life refits to remain within 81 months.

MAP Status: Full implementation

Management Action Plan Summary Status

Fully implemented.

Contract Basis of Payment

Management Action Plan and AERMS Status

This recommendation will be instituted in the Project Approval Guide via the current re-write which is already under way.

OPI: VCDS/Chief of Programme

Target Date: October 2011

AERMS Status: Full implementation

Assessment of Progress by CRS

The Project Approval Directive, which replaced the Project Approval Guide in October 2011, requires the review of contract cash flow and holdbacks prior to contract award to be a standard Senior Review Board agenda item.

MAP Status: Full implementation

Management Action Plan Summary Status

Full implementation.

Cost and Schedule Control

Management Action Plan and AERMS Status

3.a) General ledger accounts for major acquisition and in-service support contracts that exceed five years in duration will be brought to the attention of the Director Financial Accounting.

OPI: ADM(Mat)/Chief of Staff Material/Director Material Group Comptroller

Target Date: April 2011

AERMS Status: Full implementation

3.b) The Defence Resource Management Information System tool will be more fully utilized once the multi-year (> five years) commitments are included in the Defence Resource Management Information System and there is more flexibility in adjusting numbers between years, budget elements and commitments.

OPI: HCM/FELEX PMO

Target Date: Resolution of Defence Resource Management Information System shortfalls

AERMS Status: Under way

Assessment of Progress by CRS

3.a) Long-term commitments can be entered in the Defence Resource Management Information System for 25 years. Director Financial Systems Operation and Integration works with cost centres to authorize those long-term commitments that have valid requirements. Otherwise, the maximum commitment is for six years. This limitation on future commitments is due to the annual “system maintenance”⁶ that requires manual update of cost centre coding if it is no longer in use.

MAP Status: Full implementation

3.b) The HCM FELEX PMO office has entered commitments until fiscal year 2018/19, and the project is expected to close out in January 2019.

MAP Status: Full implementation

Management Action Plan Summary Status

Fully implemented

Requirements Traceability

Management Action Plan and AERMS Status

ADM(Mat) is in the process of a major revision to the Project Management Manual governing Materiel Acquisition and Support projects for which ADM(Mat) is the functional authority. We will ensure that appropriate policy direction and guidance concerning requirements traceability is included in the manual.

OPI: ADM(Mat)/Director General Material Systems and Supply Chain/Director Materiel Policy and Procedures

Target Date: December 2011

AERMS Status: Full implementation

Assessment of Progress by CRS

Some information that pertains to the Dynamic Object Oriented Requirements Suite tool and related training can be found in ADM(Mat) Knowledge Network (KNet). The process, procedures and guidance for requirements management and traceability in the Material Group are detailed in the Mat KNet Project Management Statement of Operation Requirements page and within the DND System Life Cycle Model/Engine; in particular system Engineering Process (LCO2) and the Enabling Processes Requirements Management (TM-04). The latter is scheduled for publishing in December 2013 and contains the detailed step-by-step process for requirements traceability, including when a tool should be employed.

With respect to policy, the Project Approval Directive, chapter 7, provides departmental direction for requirement development, management and traceability. This includes guidance on the appropriate use of Dynamic Object Oriented Requirements Suite tool to assist in requirement management and traceability. Complex projects that have a dedicated operational requirements staff should investigate the use of the tool. It is the intention of DND to expand the use of this tool.

OPI: ADM(Mat)/Director General Material Systems and Supply Chain/Director Materiel Policy and Procedures

Target Date: December 2013

MAP Status: Preparation for implementation

Management Action Plan Summary Status

This management action plan is incomplete.

Navy Warehouse Space

Management Action Plan and AERMS Status

ADM(Mat) and the Canada Operational Support Command have recently launched a Canadian Armed Forces-wide Inventory Rationalization Programme to immediately reduce national inventory and to increase awareness of materiel flow, while building a long-term systematic approach to rationalize inventory holdings. The programme is comprised of four phases, starting with “Definition and Preparation” from November 2010 to January 2011, and will conclude with the systematic rationalization of inventory from March 2012 and onward. Because this is a mandated corporate programme, DGMEPM will participate fully in this initiative.

OPI: ADM(Mat)/DGMEPM

Target Date: Implement January 2011 onwards

AERMS Status: Under way

Assessment of Progress by CRS

Dormant stock issues at Esquimalt and Halifax still exist. Under the Inventory Management Modernization and Rationalization Project, ADM(Mat) has established an Inventory Management Working Group in consultation and collaboration with Level One stakeholders as well as with representatives from ADM(Mat). The Inventory Management Working Group is expected to undertake a review that will address material management process in addition to undertaking an accelerated review and rationalization of dormant inventory. This process is planned to be completed by 31 December 2018.

OPI: ADM(Mat)/DGMEPM

Target Date: December 2018

AERMS Status: Preparation for implementation

Management Action Plan Summary Status

Preparation for implementation.

Project Cost Estimates/Validation

Management Action Plan and AERMS Status

This recommendation will be instituted in the Project Approval Guide via the current re-write, which is already under way.

OPI: VCDS

Target Date: October 2011

AERMS Status: Full implementation

Assessment of Progress by CRS

The Project Approval Guide was revised in October 2011 and is now known as the Project Approval Directive. The Project Approval Directive requires the Senior Review Board to endorse cost-capped Requests for Proposals before they are promulgated to industry.

MAP Status: Full implementation

Management Action Plan Summary Status

Fully implemented.

Human Resource Staffing

Management Action Plan and AERMS Status

7.a) Longer-range forecasts of civilian Project Management Personnel Requirements should be in place. Director Material Group Human Resources, in its annual call for military Project Management Personnel Requirement, will include direction for forecasting civilian project resource requirements. This process is launched each year in May and is completed at 30 September.

OPI: ADM(Mat)/Director Material Group Human Resources

Target Date: October 2011

AERMS Status: Full implementation

7.b) Consistent staffing time targets should be adopted. Director Material Group Human Resources developed its own staffing target timelines for collective staffing processes as the timelines set by Assistant Deputy Minister (Human Resources – Civilian) (ADM(HR-Civ)) are currently not achievable. Director Material Group Human Resources will work with our Service Centre Manager from ADM(HR-Civ) to determine if there is flexibility to set more realistic staffing timeline targets. If this is not achievable, it will be necessary to maintain the status quo or risk not starting staffing processes in time to fill vacancies in a timely manner.

OPI: ADM(Mat)/Director Material Group Human Resources

Target Date: February 2011

AERMS Status: Full implementation

7.c) Non-generic experience requirements should be included in the Statement of Merit.

As of 11 August 2010, non-generic experience criteria are being included as mandatory criteria for the recent competitions for the Ship Refit Manager in the HCM/FELEX PMO, as well as for engineering specialists in the Aerospace Equipment Program Management division.

OPI: ADM(Mat)/Community Management Office

Target Date: Complete

AERMS Status: Full implementation

Assessment of Progress by CRS

7.a) It was evident from the July 2012 Project Management Board briefing by ADM(Mat) that a five-year Civilian Project Management Personnel Requirements has been implemented.

MAP Status: Full implementation

7.b) Since it has not been possible to set more realistic staffing timeline targets, Director Material Group Human Resources will continue to maintain the existing target of seven months.

MAP Status: Full implementation

7.c) Statement of Merit Criteria for seven HCM/FELEX PMO project management positions that were staffed were reviewed. When compared to the relevant generic Statement of Merit Criteria, the project management staffing included some non-generic criteria as well.

MAP Status: Full implementation

Management Action Plan Summary Status

Fully implemented.

Risk Management

Management Action Plan and AERMS Status

The HCM/FELEX PMO updated the project Risk Management Plan on 13 August 2010 to comply with departmental guidance and ensure risk management practices were consistent with the Plan.

OPI: HCM/FELEX PMO

Target Date: Complete

AERMS Status: Full implementation

Assessment of Progress by CRS

The project Risk Management Plan was updated in March 2011. It addresses the March 2011 audit observations regarding the need for the alignment with the Departmental Risk Management guidelines. The PMO risk management practices are also in accordance with the established project Risk Management Plan.

MAP Status: Full implementation

Management Action Plan Summary Status

Fully implemented.

___________________________________________________________________________________________________________________________

Footnote 1 Policy on Internal Audit http://www.tbs-sct.gc.ca/pubs_pol/dcgpubs/ia-vi/ia-vi_e.asp.

Footnote 2 AERMS is a system that tracks the progress of the MAP items.

Footnote 3 The Senior Review Board is co-chaired by C Navy and ADM(Mat).

Footnote 4 Restricted Readiness is one of three readiness categories that indicate ship response time. After the Tiered Readiness Program, Standard Readiness will be attained followed by High Readiness.

Footnote 5 The first ship is taking 30 months and the second ship is taking 15 months.

Footnote 6 System Maintenance is performed annually by Director General Financial Operations. This results from the "annual master data review" memo that is sent out to all commands and the financial coding restructuring that is performed annually by the Receiver General.