Follow-up on Internal Audit of the Joint Support Ship Project

Caveat

The result of this work does not constitute an audit of the Joint Support Ship (JSS) project. Rather, this report was prepared to provide reasonable assurance that the Management Action Plans (MAPs) that resulted from the 2011 audit were implemented as stated and as such have addressed the associated recommendations.

Table B-1. Management Action Plan Scorecard. This table shows the ADM(RS) assessment of progress on the MAP.
Recommendation # MAP OPI ADM(RS) Assessment of Progress
1. Capability Cost Trade-Off 1.1 The operational impact and availability of JSS in relation to the number of ships will be articulated in the next revision of the statement of operational requirements to be presented and endorsed at the annual Senior Review Board meeting in 2012. Commander Royal Canadian Navy Full Implementation
2. Project Schedule 2.1 ADM(RS) project scheduling recommendations will be adopted. The JSS PMO will update its Project Master Schedule to ensure that all identified Project Definition activities have appropriate resources assigned with realistic productivity considerations and levels. ADM(Mat) Obsolete or Superseded
2.2 The JSS PMO will ensure the NSPS Secretariat maintains the issue of non-combat work package sequencing as an open action item within its DND/Canadian Coast Guard Project Action Log. This will promote a regular dialogue on this schedule risk between NSPS, DND, Department of Fisheries, and the Canadian Coast Guard on the optimal scheduling of the affected projects. Finally, given the interdepartmental significance, visibility into this risk has been elevated to the Major Crown Project-Interdepartmental Oversight Committee level. ADM(Mat)

Full ImplementationFootnote 4

3. Contract Management 3.1 The recommendations proposed by ADM(RS) in support of contract management will be taken into consideration when developing future contracts in support of JSS, specifically the build contract with the designated NSPS shipyard. ADM(Mat) Preparation for Implementation
4. Contract Management 4.1 ADM(RS) recommendations concerning vendor reporting and statement of work clarity have been implemented in both the Military Off The Shelf and New Design procurement documents. Terms of payment are presently being negotiated to ensure payments will be in line with actual work (milestones) completed. ADM(Mat) Obsolete or Superseded
5. Human Resources Management 5.1 Project Management Support Office, in consultation with Director Materiel Group Operational Research Acquisition Support Team, will undertake to develop a PMO staffing model to supplement the existing departmental methodology and guidance. ADM(Mat)

Full ImplementationFootnote 5

5.2 The JSS PMO has drafted a Human Resource Management Plan that reflects ADM(RS) recommendations to better address succession planning and surge requirements within the project. The Human Resource plan is currently under review and will be promulgated when ready. ADM(Mat) Full Implementation
6. In-Service Support Strategy 6.1 Linking vendor performance to exercising an In-Service Support contract option will be considered in the next In-Service Support Statement of Work and In-Service Support Pro-Forma contract revisions. The criteria, metrics and associated penalties tied to maintenance deferral by the vendor will also be considered and will be articulated in detail in the above-mentioned revised documents. ADM(Mat)

Full ImplementationFootnote 6

7. Risk Management 7.1 ADM(RS) risk management recommendations have been incorporated into the project’s draft Risk Management Plan. This revision of the Risk Management Plan is presently under review and will be promulgated in the fall of 2011. Updates include improved definitions for risk impact levels and the adoption of the five levels of risk severity, in accordance with the DND Integrated Risk Management policy. ADM(Mat) Full Implementation
7.2 The JSS PMO will rescore all project risks in light of the new impact definitions and complete a risk quantification exercise using Project Management Body Of Knowledge’s Expected Monetary Value practices to assess the adequacy of the project’s existing risk mitigation and contingency levels. ADM(Mat) Full Implementation
Legend
Obsolete or Superseded No Progress or Insignificant Progress Planning Stage Preparation for Implementation Substantial Implementation Full Implementation
Table B-1 Details – MAP Scorecard 

Recommendation 1

Table C-1. Progress in Implementing the MAP for Recommendation 1. This table shows the ADM(RS) assessment of recommendations concerning amendments to the Statement of Operational Requirements.
MAP OPI Target Date Progress to Date Assessment
1.1 The operational impact and availability of JSS in relation to the number of ships will be articulated in the next revision of the statement of operational requirements to be presented and endorsed at the annual Senior Review Board meeting in 2012. Commander Royal Canadian Navy November  2012

In 2012 the JSS Statement of Operational Requirements was updated to address the following three issues:

  1. how operations would be impacted under scenarios of various JSS availability;
  2. further information to clarify the operational risk to the fleet under these scenarios; and
  3. information acknowledging that challenges exist to meet full capability requirements within the assigned budget.

Subsequent to the original audit of JSS, the new Defence Policy: Strong, Secure, Engaged (SSE) specified two JSS. The PMO has updated its communication products accordingly.

Full Implementation
ADM(RS) Assessment of Progress on MAP:    Full Implementation
Obsolete or Superseded No Progress or Insignificant Progress Planning Stage Preparation for Implementation Substantial Implementation Full Implementation
Table C-1 Details – Status of the Implementation of the MAP Item for Recommendation 1

Recommendation 2

Table C-2. Progress in implementing the MAP for Recommendation 2. This table shows the ADM(RS) assessment of recommendations concerning project scheduling.
MAP OPI Target Date Progress to Date Assessment
2.1 ADM(RS) project scheduling recommendations will be adopted. The JSS PMO will update its Project Master Schedule to ensure that all identified Project Definition activities have appropriate resources assigned with realistic productivity considerations and levels. ADM(Mat) November 2011

Since the original audit, roles regarding the project schedule have evolved. The Shipyard Contractor now manages the majority of the tasks that make up the greater project schedule, as opposed to the PMO. The PMO now monitors and provides feedback on the Shipyard Contractor’s Integrated Master Schedule.

In particular, the PMO is monitoring and completing several tasks related to the Integrated Master Schedule with the Shipyard Contractor. These actions include:

  1. reviewing/contributing to the schedule created and managed by the Shipyard Contractor, including visibility into the Shipyard Contractor allocation of human resources; and
  2. managing the review and approval of JSS deliverables from the Shipyard Contractor through a detailed tracking sheet that ensures PMO staff know when to expect a deliverable and expected timeframes for PMO review and feedback; working collaboratively with the contractor to determine acceptable durations for the completion of milestones and events; and lastly, by conducting regular communication with the contractor through various methods such as monthly progress reports, teleconferences and meetings.

Due to these changes, this MAP is obsolete and the original risk of project definition activities being delayed due to insufficient schedule controls within the PMO is no longer a significant concern. The risk has largely been transferred to the monitoring of the Shipyard Contractor’s schedule.

Obsolete or Superseded
ADM(RS) Assessment of Progress on MAP: Obsolete or Superseded
Obsolete or Superseded No Progress or Insignificant Progress Planning Stage Preparation for Implementation Substantial Implementation Full Implementation
Table C-2 Details – Status of the Implementation of the MAP Item for Recommendation 2

Recommendation 3

Table C-3. Progress in implementing the MAP for Recommendation 3. This table shows the ADM(RS) assessment of recommendations concerning contract terms and conditions.
MAP OPI Target Date Progress to Date Assessment
3.1 The recommendations proposed by ADM(RS) in support of contract management will be taken into consideration when developing future contracts in support of JSS, specifically the build contract with the designated NSPS shipyard. ADM(Mat) Effective Project Approval

The original audit included a review of the draft NSPS Request For Proposal and identified areas for improvement with respect to contract clauses. It was recommended that ADM(Mat) include these clauses in the JSS implementation build contract. Although the implementation build contract has not been finalized, considerable progress has been made in addressing original concerns in two of the current definition phase contracts – the Design and Production Engineering and Long Lead Items contracts. The following progress was observed:

  1. Better visibility of principal subcontractors was noted for these contracts and one contract included the requirement for subcontract competition. In addition, the discretionary audit clause was required for all subcontracts where it added value.
  2. Both contracts required a ‘most favoured customer’ certification by stipulating that subcontracted rates cannot be higher than the best rates provided to others for like quantity and quality.
  3. The concern over the timing of payments to the Shipyard Contractor for Canada’s share of shipyard improvement costs has been resolved.
  4. The two contracts stated that either party could seek resolution from the courts at any time during the dispute resolution process as recommended in the original audit.
  5. A clause regarding liquidated damages was included in both contracts for Industrial and Regional Benefits policy commitments, but not to protect against delay.

Much progress has been made, however until the future implementation build contract is in place with clauses to address risks raised in the original audit, this MAP cannot be considered complete.
The PMO has agreed to a revised due date of November 2019 for the finalization of the MAP – current expected Project Approval – Implementation date.

Preparation for Implementation
Revised target date: November 2019
ADM(RS) Assessment of Progress on MAP: Preparation for Implementation
Obsolete or Superseded No Progress or Insignificant Progress Planning Stage Preparation for Implementation Substantial Implementation Full Implementation
Table C-3 Details – Status of the Implementation of the MAP Item for Recommendation 3

Recommendation 4

Table C-4. Progress in implementing the MAP for Recommendation 4. This table shows the ADM(RS) assessment of recommendations concerning vendor reporting and Statement of Work clarity.
MAP OPI Target Date Progress to Date Assessment
4.1  ADM(RS) recommendations concerning vendor reporting and Statement of Work clarity have been implemented in both the Military Off the Shelf and New Design procurement documents. Terms of payment are presently being negotiated to ensure payments will be in line with actual work (milestones) completed. ADM(Mat) March 2011

The recommendation and associated MAP 4.1 had many elements that were specific to two 2011 definition phase contracts. These specific definition phase contracts are no longer in place as the scope of work has been completed. This MAP is therefore obsolete. Nonetheless, the follow-up examined the progress made before the obsolescence and observed that some progress had been made:

  1. The terms of payment were updated to link milestones to key deliverables and a time verification clause was added to one of the two contracts.
  2. Additional schedule information was requested from the contractor as was recommended in the original audit. Secondly, a cost confidence level to support the strength of the contractor’s cost estimate was included in one of the two contracts as recommended. Finally, the PMO requested the contractor(s) provide a cost estimate for a third ship, which was included in one of the two contracts.
  3. Recommended clarity in language in the contract’s statement of work was included as recommended.
Obsolete or Superseded
ADM(RS) Assessment of Progress on MAP: Obsolete or Superseded
Obsolete or Superseded No Progress or Insignificant Progress Planning Stage Preparation for Implementation Substantial Implementation Full Implementation
Table C-4 Details – Status of the Implementation of the MAP Item for Recommendation 4

Recommendation 5

Table C-5. Progress in implementing the MAP for Recommendation 5. This table shows the ADM(RS) assessment of recommendations concerning HR planning.
MAP OPI Target Date Progress to Date Assessment
5.2  The JSS PMO has drafted a Human Resources Management Plan that reflects ADM(RS) recommendations to better address succession planning and surge requirements within the project. The Human Resources plan is currently under review and will be promulgated when ready. ADM(Mat) February 2012

The Human Resources Plans from fiscal year 2011/2012 and 2017 (current version) include a section on succession management. It describes how the JSS PMO will identify positions that require added attention, and the activities that can be completed to mitigate possible future turnover of key personnel.

The fiscal year 2011/2012 Human Resources Plan includes sharing of resources from Director General Maritime Equipment Program Management and the Engineering Logistic Management Support contractor to handle surges during the Project Definition phase. Further, the 2017 Memoranda of Understanding with the Canadian Coast Guard establishes how both organizations are expected to manage their various responsibilities required for the review work for project inspections.

Full Implementation
ADM(RS) Assessment of Progress on MAP:  Full Implementation
Obsolete or Superseded No Progress or Insignificant Progress Planning Stage Preparation for Implementation Substantial Implementation Full Implementation
Table C-5 Details – Status of the Implementation of the MAP Item for Recommendation 5

Recommendation 7

Table C-6. Progress in implementing the MAP for Recommendation 7. This table shows the ADM(RS) assessment of recommendations concerning the Risk Management Plan Revisions.
MAP OPI Target Date Progress to Date Assessment
7.1  ADM(RS) risk management recommendations have been incorporated into the project’s draft Risk Management Plan. This revision of the Risk Management Plan is presently under review and will be promulgated in the fall of 2011. Updates include improved definitions for risk impact levels and the adoption of the five levels of risk severity, in accordance with the DND Integrated Risk Management Policy. ADM(Mat) December 2011

The updated Risk Management Plan addressed the four risk management observations from the audit report. The Risk Management Plan and relevant documents included:

  1. the development of the criteria for five threshold levels of risk (in accordance with the DND Integrated Risk Management Policy and Guidelines);
  2. the severity of the risk scale was modified to be more aligned with the impact/probability of the risks;
  3. improved reporting of each project’s respective top risks; and
  4. the adoption of the Project Management Body Of Knowledge Expected Monetary Value practice when quantifying risks.
Full Implementation
ADM(RS) Assessment of Progress on MAP:  Full Implementation
Obsolete or Superseded No Progress or Insignificant Progress Planning Stage Preparation for Implementation Substantial Implementation Full Implementation
Table C-6 Details – Status of the Implementation of the first MAP Item for Recommendation 7

Recommendation 7

Table C-7. Progress in implementing the MAP for Recommendation 7. This table shows the ADM(RS) assessment of recommendations concerning risk rescoring.
MAP OPI Target Date Progress to Date Assessment
7.2  JSS PMO will rescore all project risks in light of the new impact definitions and complete a risk quantification exercise using Project Management Body of Knowledge’s Expected Monetary Value practices to assess the adequacy of the project’s existing risk mitigation and contingency levels. ADM(Mat) March 2012

All project risks in the updated risk register were assessed using the Expected Monetary Value method in accordance with the Project Management Body of Knowledge and had a quantitative measure associated with it. While there was no specific evidence of the PMO assessing the totals against their contingency amounts, they did calculate the total Expected Monetary value. The contingency available to the project, when compared to the total Expected Monetary value, should be sufficient to mitigate the risks.  

Full Implementation
ADM(RS) Assessment of Progress on MAP:  Full Implementation
Obsolete or Superseded No Progress or Insignificant Progress Planning Stage Preparation for Implementation Substantial Implementation Full Implementation
Table C-7 Details – Status of the Implementation of the second MAP Item for Recommendation 7


Back to Table of Contents

Page details

Date modified: