Audit of the A and B Jetty Recapitalization Project
Alternate Formats
Assistant Deputy Minister (Review Services)
October 2014
7050-11-7-5 (CRS)
Reviewed by CRS in accordance with the Access to Information Act (AIA). Information UNCLASSIFIED.
- CAF
-
Canadian Armed Forces
- Comd
-
Commander
- CRS
-
Chief Review Services
- CSC
-
Canadian Surface Combatant
- DCC
-
Defence Construction Canada
- DCPD
-
Director Construction Project Delivery
- DND
-
Department of National Defence
- ADM(IE)
-
Assistant Deputy Minister (Infrastructure and Environment)
- FAA
-
Financial Administration Act
- FAM
-
Financial Administration Manual
- FY
-
Fiscal Year
- JSS
-
Joint Support Ship
- OPI
-
Office of Primary Interest
- PAD
-
Project Approval Directive
- RCN
-
Royal Canadian Navy
- VCDS
-
Vice Chief of the Defence Staff
- SOR(I)
-
Statement of Operational Requirements (Infrastructure)
- ADM(Fin CS)
-
Assistant Deputy Minister (Finance and Corporate Services)
Results in Brief
In 2011, Chief Review Services (CRS) conducted the Analysis of Capital Infrastructure Projects. Based on attributes such as the materiality, duration, location and departmental priority of the project, the A and B Jetty Recapitalization Project was ranked as one of the top projects that could benefit from audit coverage. It was thus included in the CRS Fiscal Year (FY) 2013/14 to 2015/16 Risk-Based Audit Plan. The objective of this audit was to assess whether adequate governance processes, controls, and risk management practices were in place to help ensure value for money for the A and B Jetty Recapitalization Project.
Overall Assessment
- Project operational requirements need to be updated and aligned with interdependent projects to ensure the desired outcomes.
- Documenting and managing project risks can help the project mitigate further cost increases and schedule slippage.
The A and B Jetty Recapitalization Project aims to demolish the existing A and B Jetties, located at Canadian Forces Base Esquimalt—which are approximately 70-year-old, wood-based structures—and replace them with modern steel-and-concrete pile jetties to berth existing and planned vessels for the Royal Canadian Navy (RCN). The existing jetties have already exceeded their useful life, which has led to their gradual deterioration and functional limitations in naval operations. The project is being delivered in a phased approach, where the jetties are currently being designed and the construction of the jetties should be completed in 2021. In November 2013, the project team advised senior management that the forecasted cost to complete the project will be higher than the indicative estimate of ▓▓▓▓▓▓▓▓▓▓▓▓▓ million that was previously submitted. The project team is progressing towards developing a substantive estimate to move forward with the construction of the jetties.
Findings and Recommendations
Statement of Operational Requirements (Infrastructure) and the Designed Length of A Jetty. The Statement of Operational Requirements (Infrastructure) (SOR(I)) is the project document that should clearly state the project’s operational requirements. Throughout the design phase of a project, assumptions, trade-offs and constraints are examined and refined, resulting in the evolution of operational requirements. As this occurs, care must be taken to ensure that the SOR(I) is clear and reflects the most current operational requirements. The SOR(I) and other key decision documents for the A and B Jetty Recapitalization Project currently contain ambiguities that make it unclear whether the project team must ensure that the recapitalized A and B Jetties are capable of accommodating the RCN’s future Canadian Surface Combatants (CSC), or if the project team is only required to take the CSCs into consideration when designing the new jetties. The current design of A Jetty will accommodate the length of the current Halifax Class Frigates and Iroquois Class Destroyers. However, A Jetty is not being designed to accommodate berthing the four planned CSCs according to the assumed length stated in the SOR(I). Therefore, A Jetty may require an extension, should CSC length exceed current vessel length, in order to provide sufficient operational berthing space for the CSCs. The ambiguity in the SOR(I) should be resolved to ensure that the SOR(I) and all other project documentation going forward reflects the most current operational requirements, and that the A and B Jetty requirements align with the documentation for other relevant major ship projects, such as that for the CSCs.
Recommendation 1. The Commander (Comd) RCN should amend the A and B Jetty Recapitalization Project SOR(I) to ensure that it reflects the most current project operational requirements and vessel characteristics, and should develop a plan to be able to berth the CSCs at Esquimalt Dockyard when the CSCs begin delivery in approximately 2026.
Director Construction Project Delivery (DCPD) Quality Manual. Elements of the DCPD Quality Manual are outdated and not consistent with the Vice Chief of Defence Staff (VCDS) Project Approval Directive (PAD). The DCPD Quality Manual provides detailed procedures and guidance to manage infrastructure projects. However, it is missing some key project requirements with respect to formal risk management, as prescribed by the PAD. Similar to other infrastructure projects at the Department of National Defence (DND), the project team for the A and B Jetty Recapitalization Project relies on the DCPD Quality Manual, and the project is, therefore, not fully compliant with departmental direction that is provided in the PAD.
Recommendation 2. The Assistant Deputy Minister (Infrastructure and Environment) (ADM(IE)) should update the DCPD Quality Manual and implement a process to ensure that it is continually in line with the PAD.
Risk Management. The project team for the A and B Jetty Recapitalization Project has not yet developed a formal risk management plan or a risk register. A risk consultant has been hired to produce a formal risk management plan and a risk register by end of FY 2014/15. Although a formal risk management plan and risk register have not been developed, the project has put in place a project review committee to deal with project issues.
Recommendation 3. ADM(IE) should continue developing a formal risk management plan and a risk register, and ensure that the risk register is in accordance with the PAD, and that it be kept current throughout the life of the project to effectively manage ongoing project risks.
Allowance and Contingency. The development of allowance and contingency for infrastructure projects is a risk management process that quantifies risks and allocates additional funding to projects to deal with these risks. However, DND’s policy and guidance are not clear and consistent on the calculation and usage of allowance and contingency for all infrastructure projects. As such, allowance and contingency may be inconsistently calculated and used by infrastructure projects including the A and B Jetty Recapitalization Project.
Recommendation 4. Assistant Deputy Minister (Finance and Corporate Services) (ADM(Fin CS)) should clarify the methodology that should be used to calculate allowance and contingency for infrastructure projects, and communicate this methodology to ADM(IE).
Recommendation 5. ADM(IE) should ensure that personnel have a common understanding of the methodology to calculate allowance and contingency for infrastructure projects and when both should be used.
Certification of Payments. Following a recommendation from the Auditor General of Canada, a cyclical review of controls over Defence Construction Canada’s (DCC) sections 34 and 33 was conducted as part of this audit. The audit also tested DND’s controls over its financial management processes. It was found that further evidence to support completeness of the Financial Administration Act (FAA) section 34 process would improve the level of assurance that the certification process is being followed by DND and DCC. An updated Financial Administration Manual (FAM) chapter now requires that a checklist supporting verification related to section 34 be kept on file. However, ADM(Fin CS) is currently drafting a policy memo to clarify the use of the checklist and, therefore, ADM(IE) has not yet communicated the revised FAM requirement to personnel who certify section 34.
Recommendation 6. ADM(IE) should communicate the revised section 34 certification process to relevant ADM(IE) and DCC personnel and ensure they are aware of its proper application.
Note: For a more detailed list of CRS recommendations and management response, please refer to Annex A—Management Action Plan.
Rationale for Audit
In June 2011, CRS conducted the Analysis of Capital Infrastructure Projects to determine potential infrastructure projects that would benefit from audit coverage. The analysis considered the dollar-value of projects, departmental priority, and dependencies of related equipment projects. Based on this analysis, the A and B Jetty Recapitalization Project was included in the CRS Risk-Based Audit Plan for FYs 2013/14 to 2015/16. The A and B Jetty Recapitalization Project had a high level of materiality, with an estimated cost of ▓▓▓▓▓▓▓▓▓▓▓▓▓ million, and a high departmental priority due to its critical role in supporting several RCN equipment projects. The A and B Jetties are the operational berthing facilities for the RCN’s major vessels at Canadian Forces Base Esquimalt Dockyard location.
The RCN is entering a dynamic new phase that involves upgrading and expanding the operational capability of the fleet to ensure it is able to meet the needs of Canadians for decades to come. DND is currently planning to procure new vessels through the CSC Project and new support ships through the Joint Support Ship (JSS) project. It is also adding the Arctic Offshore Patrol Ships, which is a new capability. The plans are to berth these vessels along with Victoria Class submarines at the recapitalized A and B Jetties in Esquimalt. Until the new vessels arrive, the A and B Jetties will berth the Victoria Class submarines, the Halifax Class Frigates, the Iroquois Class Destroyers, and the Protecteur Class supply ship that are assigned to RCN.
Background
In the early 2000s, individual projects to recapitalize A Jetty and B Jetty were started; however, after the initial options analysis work, a lack of funding halted both projects. Commencing in 2005, all new funding provided to DND was given on an accrual basis. The adoption of this Accrual Budget meant that the Department could receive cash up front to acquire new assets, although the cost of the assets would only be recognized over their useful life, thereby making large undertakings more affordable. The individual projects to recapitalize A Jetty and B Jetty were combined into one large project to take advantage of operational synergies and cost savings, and the new project team was able to restart at the identification phase of the project. An RCN berthing plan was developed to determine which vessels would be berthed at both jetties. Among the vessels to be berthed at the jetties were the Arctic Offshore Patrol Ships. Since this was a new capability that would require berthing space additional to what was already available, the Arctic Offshore Patrol Ships project, also an accrual budget project, agreed to contribute approximately $100 million towards funding for the A and B Jetty Recapitalization Project.
This Project aims to demolish the existing A and B Jetties, located at Canadian Forces Base Esquimalt, which are approximately 70 year old wood-based structures, and replace them with modern steel-and-concrete pile jetties to berth existing and planned vessels for the RCN. The existing jetties have already exceeded their useful life, which has led to their gradual deterioration and functional limitations in providing support to naval operations. The Project is being delivered in a phased approach where the jetties are currently being designed, and two jetties’ construction will be done in sequence. That construction should be complete in 2021. In November 2013, the project team advised senior management that the forecasted cost to complete the project will be higher than the indicative estimate of ▓▓▓▓▓▓▓▓▓▓▓▓▓ million that was previously submitted. The project team is progressing towards developing a substantive estimate to move forward with the construction of the jetties.
Objective
The objective of this audit was to assess whether adequate governance processes, controls, and risk management practices were in place to help ensure value for money for the A and B Jetty Recapitalization Project.
Scope
The audit scope included:
The A and B Jetty Recapitalization Project, from Identification to the Phase I Implementation and to Phases II and III Definition (design). The audit covered work from the year 2000 up to early 2014.
Methodology
The approach included the following:
- interviews and discussions with staff from the RCN, ADM(IE), ADM(Fin CS), VCDS and DCC;
- examination of DND and DCC policies and procedures;
- examination of project documents—project charter, SOR(I), and contracts;
- examination of contract management practices that relate to FAA sections 33 and 34;
- data analysis—Defence Resource Management Information System, Financial and Managerial Accounting System, and Change Orders; and
- site visit with end users at Canadian Forces Base Esquimalt.
Audit Criteria
The audit criteria can be found at Annex B.
Statement of Conformance
The audit findings and conclusions contained in this report are based on sufficient and appropriate audit evidence gathered in accordance with procedures that meet the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing. The audit thus conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program. The opinions expressed in this report are based on conditions as they existed at the time of the audit, and apply only to the entity examined.
SOR(I) and the Designed Length of A Jetty
The SOR(I) has not been kept current and A Jetty may require an extension, should CSC length exceed current vessel (Halifax Class Frigates) length, in order to provide sufficient operational berthing space for the CSCs.
SOR(I)
For the A and B Jetty Recapitalization Project, the SOR(I)1 outlines the requirements needed for the A and B Jetties to achieve the vessel berthing configuration outlined in the RCN berthing plan. The SOR(I) was approved by the project sponsor in January 2011. It is a document that should be updated as required, since the design of infrastructure projects is an evolving process that becomes more definitive as assumptions are developed and examined. As the examination of assumptions, trade-offs and constraints are completed, any results that impact project requirements should be reflected in the SOR(I).
The SOR(I) for this project is inconsistent, as it makes conflicting statements regarding berthing requirements. The mandate in the SOR(I) states that the “A and B Jetty Recapitalization Project is to accommodate … four Halifax Class ships” and that “consideration must be given to the new CSCs.” However, the general berthing requirements section states that “the A and B Jetty Recapitalization Project must be capable of accommodating four Frigates/Destroyers which will be replaced by the CSCs,” beginning in 2026. Therefore, the SOR(I) does not make it clear whether the project team needs to consider only if the jetties should accommodate the CSCs or if the jetties need to be designed and built to accommodate the CSCs as per the assumed length stated in the SOR(I). In other key documents, such as the Project Charter2 and the Project Brief,3 the requirement appears to be clarified as it states that the new A and B Jetty “facility must be designed to accommodate …the new Canadian Surface Combatant.” The SOR(I) should be clarified because it is the document that communicates the characteristics of the operational requirements.
In addition to the ambiguity concerning the CSCs within the SOR(I), there are portions of the SOR(I) that have not been kept current and require amending to help ensure that the jetties meet the requirements of the RCN going forward. The JSS will be a new vessel added to the RCN fleet and it is planned to be berthed at B Jetty. The SOR(I) states that the JSS will have an assumed overall length of 160 to 170 metres; however, the length of the JSS has been finalized at 173.7 metres. Although B Jetty should still be able to safely berth the JSS, the SOR(I) has not been amended to reflect the finalized length of the JSS. The SOR(I) mandates that the jetties will berth two submarines, although recent communication from the project team4 of the A and B Jetty Recapitalization Project indicates that a decision has been made to berth only one submarine; however, the SOR(I) has not been amended to reflect this decision. Rather than amending the SOR(I) to reflect the updated operational requirements of the project, the A and B Jetty project team decided to focus their limited personnel resources on tasks of higher priority. An amended SOR(I) reflecting the most current project operational requirements and vessel characteristics could benefit stakeholders when making decisions; as well, it could benefit the project team to accurately address the capability deficiencies of the RCN.
The Designed Length of A Jetty
According to project documentation, part of the scope of the A and B Jetty Recapitalization Project is to provide sufficient operational berthing space for four CSCs. The CSC vessels are scheduled to begin delivery in approximately 2026 and will replace the Halifax Class Frigates and Iroquois Class Destroyers. The RCN berthing plan has designated one JSS, two Arctic Offshore Patrol Ships, and two submarines to be berthed at B Jetty and four Frigates and Destroyers/CSCs to be berthed at A Jetty. Based on the current designed length of B Jetty (approximately 439 metres) and the current known overall lengths of the vessels that are planned to be berthed at B Jetty, B Jetty will be long enough to provide sufficient operational berthing space to berth those vessels. However, based on the current designed length of A Jetty (approximately 306.8 metres), it should be able to berth the existing Frigates and Destroyers (approximately 135 metres each), although A Jetty will not be long enough to provide sufficient operational berthing space to berth the four planned CSCs (their current assumed overall length being 160 metres each, as per the SOR(I)). Since A Jetty has not been designed to provide sufficient operational berthing space for the CSCs, this could result in the CSCs not having adequate safety separation between CSCs when two CSCs are berthed alongside A Jetty, and the CSCs could overhang past the end of A Jetty. To avoid overhang past the end of A Jetty, the new CSC would have to be similar in length to the current Frigates and Destroyers, and no longer than approximately 140 metres. If the CSCs’ length will cause an overhang past the end of A Jetty, then A Jetty may require an extension in order to provide sufficient operational berthing space for the CSCs. At the November 2013 Senior Review Board meeting,5 the project team informed senior management that A Jetty was not being designed to accommodate and berth four CSCs at their current assumed lengths, but was being designed to accommodate the current Frigate and Destroyer vessels. Therefore, when the CSCs begin delivery in approximately 2026, there could be an issue regarding berthing space.
It is important for DND and the Canadian Armed Forces (CAF) that equipment and infrastructure requirements are aligned not only to meet operational requirements but to ensure that as much value results from the available funding. The DND/CAF is currently in the process of determining the quantity and design of the CSCs that it will acquire, and it is critical that sufficient operational berthing spaces are ready and available when the CSCs arrive in Esquimalt. The SOR(I) provides the means for the RCN to direct the requirements of the project, and it should help ensure that interrelated projects are aligned to meet the objectives of the DND/CAF. If A Jetty is not designed to have sufficient operational berthing space to berth the four planned CSCs, the DND/CAF may incur additional costs to correct the deficiency at a later date, which could also result in disruptions to operations. It would be more efficient for the RCN to make an informed decision proactively on how it plans to address the possible lack of berthing space for the future CSCs if the CSCs are longer than the current fleet. This lack of alignment between the A and B Jetty Recapitalization Project and infrastructure requirements of the future CSCs represents a similar concern that has previously arisen in other departmental projects where infrastructure and equipment projects were not well aligned.
Summary
The mandate and general berthing requirements in the SOR(I) are not aligned nor are they aligned with other key project documents. The SOR(I) also does not reflect current information regarding vessel dimensions, berthing plans and operational requirements. Therefore, the SOR(I) needs to be amended to reflect the project’s current requirements. The A and B Jetty Recapitalization Project is not currently designing A Jetty to provide sufficient operational berthing space to berth the CSCs based on their current assumed length, so the RCN must develop a plan to resolve this issue before the CSCs are delivered to Esquimalt.
Recommendation
1. The Comd RCN should amend the A and B Jetty Recapitalization Project SOR(I) to ensure that it reflects the most current project operational requirements and vessel characteristics, and should develop a plan to be able to berth the CSCs at Esquimalt Dockyard when the CSCs begin delivery in approximately 2026.
OPI: Comd RCN
DCPD Quality Manual
Elements of the DCPD Quality Manual are not consistent with the PAD and resulted in the A and B Jetty Recapitalization Project being noncompliant with some departmental direction.
PAD and DCPD Quality Manual
DND has one policy document (the PAD) for all projects and one guidance document (the DCPD Quality Manual) for infrastructure projects. The PAD represents policy-based direction and is issued by the VCDS to all staff involved in the development of all investment projects, and it includes high-level guidance for infrastructure projects such as the A and B Jetty Recapitalization Project. The DCPD Quality Manual provides detailed procedures and guidance to specifically manage infrastructure projects managed by ADM(IE). Project managers for infrastructure projects rely on the direction provided in the DCPD Quality Manual to manage their projects. Therefore, it is necessary for the DCPD Quality Manual to be consistent with the PAD in order to ensure that projects are compliant with departmental direction. However, due to personnel shortages, turnover, and other priorities within DCPD, the DCPD Quality Manual has not been updated and kept in line with the PAD.
During this Audit, a number of inconsistencies were found between the DCPD Quality Manual and the PAD. Some of those discrepancies impacted the A and B Jetty Recapitalization Project, including:
- The PAD requires that a risk register be provided to the VCDS analyst every year,6 but the DCPD Quality Manual does not include a requirement to create a risk register.
- The project documentation templates in the DCPD Quality Manual are different from the project documentation templates in the PAD. For example, the PAD Project Management Plan requires a risk management plan while the DCPD Quality Manual template does not. Currently, the project team for the A and B Jetty Recapitalization Project is developing a formal risk management plan and risk register.
If the DCPD Quality Manual had guidelines on when a risk register or risk management plan should be done, the A and B Jetty Recapitalization Project may have produced these risk management documents.
Other discrepancies, not impacting the A and B Jetty Recapitalization Project, documented during the Audit are:
- The DCPD Quality Manual requires infrastructure projects to produce a Project Profile and Risk Assessment.7 The PAD no longer requires this document but instead requires a Project Opportunity and Risk Assessment8 or a Project Brief.
- The PAD states that the project leader9 will be assigned based on the result of the Project Complexity and Risk Assessment.10 The DCPD Quality Manual states that the project leader will be assigned based on the project dollar value and the project risk, and not based on the result of the Project Complexity and Risk Assessment.
Project staff relying on the DCPD Quality Manual may not be following departmental guidelines, which could result in some project documentation not being produced. This could delay infrastructure projects, including the A and B Jetty Recapitalization Project, when they go for approval.
Summary
In general, infrastructure projects, including the A and B Jetty Recapitalization Project, have been and will continue to follow and rely on the DCPD Quality Manual. Since elements of the DCPD Quality Manual are out-of-date and not consistent with the PAD, infrastructure projects may continue to be noncompliant with departmental direction and certain project requirements may not be produced.
Recommendation
2. ADM(IE) should update the DCPD Quality Manual and implement a process to ensure that it is continually in line with the PAD.
OPI: ADM(IE)
Risk Management
The project team for the A and B Jetty Recapitalization Project has not yet developed a formal risk management plan or a risk register.
Risk Management Plan and Risk Register
At the time of the audit, the A and B Jetty Recapitalization Project team had not yet developed a formal risk management plan or risk register. The DCPD Quality Manual, which the project relies on for guidance, does not specify when a risk management plan11 or risk register should be done. Therefore, there is no formal identification, ongoing assessment, ranking, mitigation and quantification of risks for this project—thus, there is no assurance that key project risks that may prevent successful project delivery will be properly mitigated.
Good Practice
In July 2011, the project established a project review committee that meets approximately every four months to ensure stakeholder involvement and to deal with project issues. The project review committee is a value-added practice according to project stakeholders. The DCPD Quality Manual could be amended to state the criteria and timing for establishing project review committees for future projects.
The project team engaged a consultant in November 2012 to develop a formal risk management plan and a risk register. After some delay, work involving the risk management plan and the risk register was started in December 2013, and is scheduled to be completed by the end of March 2015. To help ensure that the project objectives are achieved, all risks should be considered, including project, contract, and interdependent12 risks.
A formal risk management plan and risk register could have helped the project achieve better value for money by mitigating project risks, including risks related to unknown and evolving requirements. During the design stage, additional engineering studies were required to gather further information on the soil and rock bed where the jetties are to be built. These studies and other evolving project requirements had the following cost and schedule impacts on the project:
- The $4.3 million A Jetty design contract has had 18 contract amendments worth approximately ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ increase in the initial contract value;
- The $3.2 million B Jetty design contract (which includes 3 phases of the design contract) has had 49 contract amendments worth approximately ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ increase in the initial contract value;
- According to an updated project schedule, the project may be delayed by 4.5 years beyond the previous departmentally-approved schedule. Based on construction escalation rate estimates from the 2013 DND Economic Model,13 this delay may lead to construction costs increasing by an estimated ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ This delay may also cause operational limitations for the RCN as the JSS is planned to be berthed at the new B Jetty. The JSS is currently scheduled to be delivered in November 2019 while B Jetty should be ready to be used three months later in February 2020. If these schedules remain the same, the risk and impact on operations could be minimal, and easily mitigated as the RCN could berth the JSS elsewhere for three months. However, if more significant delays to B Jetty construction schedule are encountered, it could result in greater operational limitations for the RCN.
Summary
Although risks were discussed at the project review committee meetings, the A and B Jetty Recapitalization Project team has not developed a formal risk management plan or a risk register. Formal risk management could have helped better identify, quantify, prioritize, mitigate, and monitor project risks, and potentially reduce their impact on the project’s cost and schedule. Effectively managing project risks can help the project team focus on the most important risks in delivering a successful project.
Recommendation
3. ADM(IE) should continue developing a formal risk management plan and a risk register, and ensure that the risk register is in accordance with the PAD, and that it be kept current throughout the life of the project to effectively manage ongoing project risks.
OPI: ADM(IE)
Allowance and Contingency
Guidelines on the calculation of allowance and contingency, and the circumstances under which each should be used, are not clear.
The development of allowance14 and contingency15 for infrastructure projects is a risk management process that quantifies project risks. Departmental personnel generally have the understanding that allowance and contingency are added to the project cost estimate to budget for uncertainty and risk that may exist or may materialize. Allowance is part of the expenditure authority controlled by the Project Manager, while contingency can only be spent by the Project Manager after it is authorized for release by the Project Leader and only after the Project Leader receives advice from the Senior Review Board.16
Guidance on the Calculation and Usage of Allowance and Contingency
DND has policy guidance in place regarding the calculation of allowance and contingency; however, there are different definitions of contingency and differing ways to calculate allowance and contingency. Due to the lack of consistent guidance, it is difficult for project teams to follow a consistent approach when calculating allowance and contingency funds for unknown circumstances.
Departmental policy guidance does not specify when allowance and contingency can be used. The DND template used to estimate the A and B Jetty Recapitalization Project costs calculates allowance and contingency as a percentage of the same cost elements (design costs, government contracting agency costs, and project manager costs). Since allowance and contingency are based on the same cost elements, it is not clear whether allowance or contingency should be used only if one of those cost elements caused the costs to increase. This lack of clarity was evident, since project team members of the A and B Jetty Recapitalization Project had different views on when allowance and contingency should be used.
Applicability of Allowance and Contingency
Based on current guidance, the project team for the A and B Jetty Recapitalization Project included ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ for allowance and ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ for contingency17 in the project cost estimate prepared prior to entering the design phase of the Project. The project team has forecasted that by the end of the design phase (scheduled to be in the fall of 2015), they will have used most of the funds allocated for both allowance and contingency. However, based on the current guidance, it is not clear if allowance and contingency were calculated appropriately and if they will be used as intended. The current guidance may not only impact the A and B Jetty Recapitalization Project, but could have ramifications for DND’s entire infrastructure program.
Using the current guidance it was estimated that for FY 2014/15, DND’s infrastructure program could have ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ in allowance and ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ in contingency18 to use to respond to project risks. Based on this guidance, it is difficult to determine whether the infrastructure program’s allowance and contingency will be calculated considering project risks, and/or if they will be used as intended.
Summary
DND’s policy guidance has inconsistent information with respect to allowance and contingency. Clearer guidance will help ensure that allowance and contingency funds for DND infrastructure projects are consistently calculated and used to cover unknown circumstances.
Recommendations
4. ADM(Fin CS) should clarify the methodology that should be used to calculate allowance and contingency for infrastructure projects, and communicate this methodology to ADM(IE).
OPI: ADM(Fin CS)
5. ADM(IE) should ensure that personnel have a common understanding of the methodology to calculate allowance and contingency for infrastructure projects and when both should be used.
OPI: ADM(IE)
Certification of Payments
Further evidence to support completeness of the FAA section 34 process would improve the level of assurance that the certification process is being followed by DND and DCC.
Recommendation from the Auditor General of Canada
The 2012 Fall Report of the Auditor General of Canada Chapter 5—Real Property—National Defence recommended that “National Defence should ensure that the cyclical and semi-annual review controls over Defence Construction Canada are implemented.”19 These controls are identified in the 2008 Memorandum of Understanding between DND and DCC; the Auditor General’s audit found that the Department’s key controls of cyclical and semi-annual reviews of section 34 and section 33 of the FAA were not being fully implemented.
In response to this recommendation, ADM(Fin CS) contracted a third party in July 2013 to conduct the semi-annual reviews required in the Memorandum of Understanding. The first report under that contract, dated June 2014, included a review of the full FY 2013/14. Subsequent reports should consist of semi-annual reviews, as per the Auditor General’s recommendation. To respond to the cyclical reviews element of the recommendation, CRS undertook the review of controls over DCC’s sections 3420 and 3321 as part of this audit. The result of this assessment is detailed below. CRS will continue to perform cyclical reviews whenever an infrastructure project involving DCC is audited.
Section 34 and Section 33 Certification
For infrastructure projects such as the A and B Jetty Recapitalization Project, DCC provides project management and contracting services to DND. As part of their responsibilities, DCC receives invoices and certifies section 34 and section 33 for goods and services received from consultants and contractors. For the services provided by DCC to DND, a monthly invoice is submitted by DCC to the DND project manager; the project manager signs section 34, and ADM(Fin CS) certifies section 33. When signing section 34 and section 33, DND and DCC personnel must follow the same standards as outlined in DND’s FAM.22
The Audit of the A and B Jetty Recapitalization Project verified if DND and DCC followed the proper steps when signing section 34 and 33. For sections 34 and 33, a judgemental sample was selected and all sampled invoices were signed by DND and DCC employees with appropriate delegation of authority. As required, section 33 was always signed for the sampled invoices after section 34, and there was segregation of duties between the person certifying section 34 and the person certifying section 33. The signing of section 34 is considered to be the key control prior to making a payment. The Audit sampled 35 financial transactions associated with the A and B Jetty Recapitalization Project for a review of the certification of section 34 process. The sampled invoices had some minor discrepancies, such as the use of incorrect personnel rates, an incorrect tax rate, and allocating costs to an incorrect general ledger. Yet the available evidence on the sampled invoices made it difficult to assess whether all aspects of the section 34 certification, as prescribed by DND’s FAM,23 had been followed.
At the time that the sampled invoices were certified, DND standards did not require evidence to be maintained on file to certify that all steps of the section 34 process were being followed. As of December 2013, the section 34 FAM chapter was updated, and now requires that a verification checklist for section 34 be maintained on file. However, ADM(Fin CS) is currently drafting a policy memo to clarify the use of the checklist, and therefore ADM(IE) has not yet communicated the revised FAM section 34 to the relevant personnel within ADM(IE) and DCC. ADM(IE) project managers and DCC staff are not currently completing the section 34 checklist as per the updated FAM. Completion of the verification checklist could mitigate some of the discrepancies mentioned above, and it would add assurance that the section 34 process was followed by DND and DCC.
Summary
All the invoices sampled during the Audit were certified by section 34 and section 33. Prior to December 2013, DND’s policy guidance did not require evidence to be kept on file to confirm that each step of the section 34 process had been completed. The updated FAM section 34 chapter now requires that a section 34 verification checklist be maintained on file. However, ADM(IE) has not yet communicated the updated FAM requirement to personnel who certify section 34.
Recommendation
6. ADM(IE) should communicate the revised section 34 certification process to relevant ADM(IE) and DCC personnel and ensure they are aware of its proper application.
OPI: ADM(IE)
General Conclusion
The A and B Jetty Recapitalization Project is a high priority infrastructure project for the RCN that will be an essential foundation to support naval operations. This project is benefiting from good practices—such as their Project Review Committee, which engages key stakeholders in addressing project issues. To help ensure that it delivers the RCN’s operational requirements while ensuring value for money, improvements are needed in the project's governance processes, controls, and risk management practices. The Project’s operational requirements need to be updated and aligned with interdependent projects to help ensure that it attains the desired outcomes. Additionally, the project needs to develop its formal risk management plan and risk register to help mitigate further schedule slippage and cost increases. These improvements can help the project successfully deliver on its aim.
Several updates to the DCPD Quality Manual are required to provide all infrastructure projects with accurate guidance and procedures according to departmental policies; this should benefit the whole infrastructure program, in addition to this project. As well, clearer guidance is needed in developing allowance and contingency estimates for foreseen and unforeseen events. Additionally, better direction regarding certification of payments needs to be communicated to all relevant personnel.
While this project is benefitting from good practices, the recommendations are provided to improve the value for money for the A and B Jetty Recapitalization Project as it moves forward, and to improve current and future departmental infrastructure projects.
Annex A—Management Action Plan
CRS uses recommendation significance criteria as follows:
High—Controls are not in place or are inadequate. Important issues are identified that could negatively impact the achievement of program/operational objectives.
Moderate—Controls are in place but are not being sufficiently complied with. Issues are identified that could negatively impact the efficiency and effectiveness of operations.
Low—Controls are in place but the level of compliance varies.
The SOR(I) has not been kept current and A Jetty may require an extension should CSC length exceed current vessel (Halifax Class Frigates) length in order to provide sufficient operational berthing space for the CSCs.
CRS Recommendation (High significance)
1. The Comd RCN should amend the A and B Jetty Recapitalization Project SOR(I) to ensure that it reflects the most current project operational requirements and vessel characteristics, and should develop a plan to be able to berth the CSCs at Esquimalt Dockyard when the CSCs begin delivery in approximately 2026.
Management Action
To address the recommendation that the A and B Jetty Project SOR(I) should be amended, Comd RCN will promulgate an SOR(I) Amendment Note not later than the end of August 2014. This Note will be appended to the previously Approved SOR(I), and the Project’s CID page, and will clarify the specific operational berthing requirements of the future A and B Jetty facility as they stand today.
OPI: Comd RCN
Target Date: August 2014
The plan to further align the long-standing relationship between the A and B Jetty Project and the CSC Project included the stand-up of a RCN “Major Ship Project PRICIE”24 Oversight Committee and its Infrastructure Working Group earlier this year. This Working Group informs all stakeholders of infrastructure implications associated with CSC fleet/size options and decisions, and advises the Oversight Committee on infrastructure investment recommendations. The Working Group and the Oversight Committee will establish as definitively and as early as possible what, if any, RCN berthing facility infrastructure improvement work must be undertaken as a CSC Project Construction in Support of Equipment investment. The A and B Jetty and CSC project Senior Review Boards will be briefed accordingly.
OPI: Comd RCN
Target Date: October 2016
Elements of the DCPD Quality Manual are not consistent with the PAD and resulted in the A and B Jetty Recapitalization Project being noncompliant with some departmental direction.
CRS Recommendation (Moderate significance)
2. ADM(IE) should update the DCPD Quality Manual and implement a process to ensure that it is continually in line with the PAD.
Management Action
DCPD Quality Manual revision to align with the current PAD is underway. The intent is to update the project approval related sections of the DCPD Quality Manual in order to align with the direction in the PAD by March 2015. Subsequent updates are planned to coincide with major scheduled revisions of the PAD.
OPI: ADM(IE)
Target Date: March 2015
The project team for the A and B Jetty Recapitalization Project has not yet developed a formal risk management plan or a risk register.
CRS Recommendation (High significance)
3. ADM(IE) should continue developing a formal risk management plan and a risk register, and ensure that the risk register is in accordance with the PAD, and that it be kept current throughout the life of the project to effectively manage ongoing project risks.
Management Action
The A and B Jetty project has retained a risk consultant. A risk management plan and risk register will be completed by the end of FY 2014/15. The risk register will be developed in accordance with the PAD, and kept current throughout the project’s life. Additionally, the project has implemented a Project Review Committee to deal with project issues, including managing the ongoing risks.
OPI: ADM(IE)
Target Date: March 2015
Guidelines on the calculation of allowance and contingency, and the circumstances under which each should be used are not clear.
CRS Recommendations (Moderate significance)
4. ADM(Fin CS) should clarify the methodology that should be used to calculate allowance and contingency for infrastructure projects and communicate this methodology to ADM(IE).
Management Action
Director Costing Services is in the process of establishing a cost risk management framework that will include clarifying the definition for contingency and clarify the methodology for quantifying contingency amounts. Director Costing Services and Assistant Deputy Minister (Science and Technology)/Centre for Operational Research and Analysis staff are working together on this initiative, and are integrating the project teams’ earlier in the cost estimation and development process to be able to participate in quantifying cost risks.
OPI: ADM(Fin CS)
Target Date: November 2014
CRS Recommendation (Moderate significance)
5. ADM(IE) should ensure that personnel have a common understanding of the methodology to calculate allowance and contingency for infrastructure projects and when both should be used.
Management Action
The methodology to calculate allowance and contingency for infrastructure projects will be reflected in a revised Submission Project Costing—Construction… and will be communicated through project management professional development sessions for all project management staff. This will ensure that personnel have a common understanding of the methodology and how it should be used for infrastructure projects. .
OPI: ADM(IE)
Target Date: March 2015
Further evidence to support completeness of the FAA section 34 process would improve the level of assurance that the certification process is being followed by DND and DCC.
CRS Recommendation (Low significance)
6. ADM(IE) should communicate the revised section 34 certification process to relevant ADM(IE) and DCC personnel, and ensure they are aware of its proper application.
Management Action
The requirements of the new FAM Chapter 1016-3 Account Verification—Section 34 have been communicated to DCC. Beginning in July 2014, third party contractor invoices processed by DCC on behalf of the Department will include a FAM Section 34 verification checklist. In addition, all other invoices certified by ADM(IE) Responsibility Managers will include a verification checklist also beginning in July 2014.
OPI: ADM(IE)
Target Date: July 2014
Annex B—Audit Criteria
Criteria Assessment
The audit criteria were assessed using the following levels:
Assessment Level and Description
Level 1: Satisfactory
Level 2: Needs Minor Improvement
Level 3: Needs Moderate Improvement
Level 4: Needs Significant Improvement
Level 5: Unsatisfactory
Governance
- Roles and responsibilities are defined, appropriate skills/staff are available to govern the Project, and project approval/monitoring processes with accurate information for decision-making are in place.
Assessment. Level 3—Elements of the DCPD Quality Manual are not up-to-date to reflect current business practices, and are not in line with the PAD. The Project Leader is responsible for approximately 140 infrastructure and environment projects (compared to a Project Leader being responsible for an average of 14 projects for equipment projects). The Project Review Committee meetings provide value-added governance for the Project.
Risk Management
- Risks are identified, assessed, ranked, mitigated, quantified and reported by the Project Office and by the vendors.
Assessment. Level 4—A formal risk management plan and risk register have not been developed; risk assessments mentioned in project documentation are unclear, inconsistent or do not accord with risk management guidelines; a risk consultant has been hired to produce a formal risk management plan and a risk register.
Controls
- Assets are managed and payments are made in accordance with the FAA and Treasury Board/DND policies.
Assessment. Level 2—There were minor discrepancies in the FAA section 34 certification; ADM(IE) has not yet communicated the updated FAM section 34 to ADM(IE) personnel and DCC.
- Cost estimates are based on reliable and relevant information from valid sources, and consisting of appropriate contingency funds.
Assessment. Level 3—DND’s policy and guidance are not aligned with respect to the calculation and use of allowance and contingency for all infrastructure projects.
- Valid objectives and requirements are in accordance with defence construction policy/standards, complete, prioritized, consistent and tracked throughout the project activities.
Assessment. Level 3—The SOR(I) does not have up-to-date information. Project documents stated the aim of the project as being four CSCs to be berthed at A Jetty, although more than two CSCs cannot be berthed at A Jetty at the current assumed length of the CSCs. The number of Change Orders for the B Jetty design contract is significantly above DCC’s average for the number of Change Orders.
- The project schedule is achievable and is managed to avoid impact on operational requirements.
Assessment. Level 4—The project completion date is delayed by approximately 4.5 years from the previous departmentally-approved schedule. This delay may lead to construction costs increasing by at least | | | | | | | | | | and may impact the RCN’s operations if construction of B Jetty is delayed significantly after the delivery of JSS.
- The procurement plan, bid evaluation, and contract terms and conditions provide assurance of value for money.
Assessment. Level 2—The procurement strategy of the A and B Jetty Recapitalization Project to have separate design contracts for A Jetty and B Jetty may have caused schedule delays and affected the value for money of this project. However, for the construction of the A and B Jetties, the strategy is to have one contract which could reduce integration-related schedule delays and improve value for money.
Sources of Criteria
- Project Approval Directive, 2011-2012
- Project Approval Guide (Replaced in 2011)
- DCPD Quality Manual
- Financial Administration Act
- Treasury Board Secretariat Audit Criteria related to the Management Accountability Framework: A Tool for Internal Auditors, March 2011
Table 1. Audit Criteria related to the Management Accountability Framework
Table Summary:
This table shows the source from the Management Accountability Framework corresponding to each audit criterion used in conducting this audit.
Audit Criteria Category | Audit Criteria Description | Reference to Management Accountability Framework |
---|---|---|
Governance | Roles and responsibilities are defined, appropriate skills/staff are available to govern the project, and Project approval/monitoring processes with accurate information for decision making are in place. | AC-3, AC-4, G-6, PPL-2, ST-18, ST-20 |
Risk Management | Risks are identified, assessed, ranked, mitigated, quantified and reported by the Project Office and by the vendors. | RM-2, RM-4, RM-5, RM-6 |
Controls | Assets are managed and payments are made in accordance with the FAA and Treasury Board/DND policies. | ST-1, ST-2 |
Controls | Cost estimates are based on reliable and relevant information from valid sources, and consist of appropriate contingency funds. | ST-10, ST-13 |
Controls | Valid objectives and requirements are in accordance with defence construction policy/standard, complete, prioritized, consistent and tracked throughout the project activities. | RP-2, RP-3 |
Controls | Project Schedule is achievable and is managed to avoid impact on operational requirements. | RM-7, ST-16, ST-22, RP-3 |
Controls | The procurement plan, bid evaluation, and contract terms and conditions provide assurance on value for money. | ST-22, RP-3 |
___________________________________________________________________________________________________________________________
Footnote 1 The SOR(I) is the document provided by the project sponsor that defines the capability deficiency to be addressed and, in broad terms, the project requirements.
Footnote 2 A project charter is a document that covers the objective and scope of the project, along with its risks, constraints, schedule/milestones, resources allocated, organizational accountabilities and reporting relationships, etc.
Footnote 3 A project brief is a document that provides a synopsis of the current state of the project and proposed way ahead in project approval submissions that are presented to either Defence Senior Management or to Treasury Board for approval. According to the PAD (B.7.1.8), the “Project Brief records plans, events, issues and achievements as they happen” on a project.
Footnote 4 2013 Senior Review Board Deck and 2014 Deck for Co-Ordination Meeting with the Esquimalt Harbour Remediation Project.
Footnote 5 The November 2013 Senior Review Board Record of Decision stated that “…when the CSCs arrive at the Dockyard Esquimalt, the possibility exists that there may be an overhang situation due to the potentially longer overall length of the ships of that class. Thus, there could be a berthing space problem.”
Footnote 6 The PAD states that “…it is mandatory for all projects that a Project Risk Register is provided to the applicable DDPC [Director Defence Programme Coordination] analyst for every decision stage and for annual progress reports.” (PAD B.10.3.2)
Footnote 7 The “Project Profile and Risk Assessment …overviews the scope, complexity and risks associated with a project and outlines the management strategy to deal with them.” (Project Approval Guide 4.5.20)
Footnote 8 The Project Opportunity and Risk Assessment “is a departmental project document, providing detailed evidence of risk analysis and assessment.” (PAD B.7.4.1)
Footnote 9 A Project Leader is “accountable for the overall management of the project.” (PAD B.8.3.6)
Footnote 10 The Project Complexity and Risk Assessment is the output of an assessment tool that determines the risk and complexity of a project for the purposes of project approval and expenditure authority.
Footnote 11 The PAD does not explicitly make mention of when a risk management plan should be created. The Internal Audit of Project Risk Management Practices (7050-11-44 (CRS)) recommended that VCDS “update the PAD to require that an RMP [Risk Management Plan] be developed earlier in a project’s life” (Recommendation 2).
Footnote 12 Interdependent risks happen when projects mutually impact one another (interdependent projects). Some of the A and B Jetty Recapitalization interdependent projects include the Esquimalt Harbour Remediation Project, the Fleet Maintenance Facility Cape Breton Shop Consolidation Project, the Arctic Offshore Patrol Ship Project, and the CSC Project.
Footnote 13 The DND Economic Model is the means of measuring the impact of inflation on the purchasing power of DND budget dollars.
Footnote 14 Allowance is not defined in the PAD.
Footnote 15 Contingency “is the quantification of risks that have been identified throughout the risk analysis, risk mitigation and risk management planning exercises.” (PAD B.8.1.14)
Footnote 16 Senior Review Board is an oversight committee that provides challenge and project performance management functions. The Senior Review Board provides advice to the Project Leader. (PAD C.15.11.2)
Footnote 17 The Project Manager and Director Costing Services cost validation working papers provided the breakdown for the estimate for allowance and contingency.
Footnote 18 Calculations of ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ of allowance and ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ of contingency that are made are based on the percentages provided in the DND template to estimate allowance and contingency. These figures are based on an inference from the 2014–2024 Capital Investment Plan (Infrastructure) expected expenditure in FY 2014/15.
Footnote 19 The 2012 Fall Report of the Auditor General of Canada, Chapter 5, 5.39. DND’s response to its recommendation: “National Defence will engage an independent third party to conduct semi-annual reviews of Defence Construction Canada’s (DCC) processes for compliance with sections 33 and 34 of the Financial Administration Act. These reviews will commence in the 2012–13 fiscal year. Further, starting in the 2013–14 fiscal year, National Defence will incorporate consideration of cyclical reviews of the controls over DCC into its risk-based management processes.”
Footnote 20 Certifying section 33 “ensures that all payments requisitioned have been duly certified pursuant to section 34 of the FAA and that they are timely, properly authorized and legal.” (FAM, section 33)
Footnote 21 Certifying section 34 ensures “that the work has been performed, the goods supplied or the service rendered, as the case may be, and that the price charged is according to the contract, or if not specified by the contract, is reasonable.” (FAA, section 34)
Footnote 22 The FAM provides the details for certification of FAA section 34 and section 33.
Footnote 23 FAM Chapter 1016-3, FAA section 34, issued 2001 (FAM version in place at time of sampled invoices).
Footnote 24 This stands for: People, Research and Development, Infrastructure, Concepts of operation and doctrine, Information Technology, Equipment, Support and Sustainment.
Page details
- Date modified: