Evaluation of Naval Force Development

November 2021

1258-3-042

Reviewed by ADM(RS) in accordance with the Access to Information Act.  Information UNCLASSIFIED.

Acronyms

Back to Table of Contents

Executive Summary

This report presents the results of the evaluation of the Naval Force Development (NFD) Program conducted during Fiscal Year (FY) 2020/21 by the Assistant Deputy Minister (Review Services) (ADM(RS)) in compliance with the 2016 Treasury Board Policy on Results. The evaluation examines the relevance and performance of the NFD Program over a five-year period FY 2015/16 to FY 2019/20.

Program Description

The NFD Program consists of Defence Program Inventory Program 4.2 Naval Force Development. The NFD Program develops and manages the execution of activities of conceive, design, build, and introduce new or modified capabilities in order to achieve desired effects during defence operations, activities or services.

Departmental Results Framework (DRF) Expenditures: $53 million for FY 2019/20.

Scope

The evaluation focused on the achievement of initial outcomes of the NFD Program and identified impacting issues. Three specific scope focus areas were:

• Extent to which the pillars of conceive, design, build and manage activities contribute to NFD;
• Extent to which capabilities are managed through each phase, including the Accountabilities, Responsibilities, Authorities (ARA); and
• Extent to which NFD manages capability innovation.

Results

Findings were aligned according to the ability of the NFD Program to influence the finding. Some findings can be managed internally by the Royal Canadian Navy (RCN). Other findings are external to the NFD and need to be referred for Departmental considerations. This was done in an Integrated Strategic Analysis (ISA) of the evaluations of the Force Development (FD) programs of the Navy, Canadian Army and Royal Canadian Air Force. Key findings of this evaluation centred around the following areas:

NFD

• Continued need and value of the Program; and
• Governance.

Integrated Strategic Analysis Related

• Innovation;
• Processes and standards;
• Collaboration; and
• Resources and training.

Back to Table of Contents

Evaluation Scope

Coverage and Responsibilities

The evaluation examined the effectiveness of the Defence Program Inventory Program 4.2 Naval Force Development over the FYs 2015/16 to FY 2019/20. The focus was on the immediate outcomes of the Program, which were “prioritized concepts are integrated throughout the design, build and manage phases effectively” and “naval capabilities are effectively governed and managed.” Three specific scope focus areas were:

• The extent to which the pillars of conceive, design, build and manage contribute to NFD;
• The extent to which capabilities are managed through each phase, including the ARAs; and
• The extent to which NFD manages capability innovation.

Key findings fall into two groups: 

• Findings within the control of the RCN  (Internal)
• Findings outside the control of the RCN  (External)  ISA

This evaluation is one of three concurrent FD evaluations conducted in FY 2020/21. Findings from this evaluation containing elements outside the control of the RCN, alongside findings from the Air and Space FD evaluation and Land FD evaluation, are further discussed in the ISA: FD.

Methodology

The issues and recommendations within this report were informed by multiple lines of evidence collected throughout the conduct phase of the evaluation. The evaluation conducted a document review, case studies, survey (n=11), administrative data collection, international benchmarking and interviews.

Out of Scope

The evaluation did not assess the departmental procurement process, materiel management or real property, except for its impact on the NFD Program.

Back to Table of Contents

Program Profile

Program Activities

The NFD Program develops and manages the execution of activities that conceive, design, build and introduce new or modified capabilities in order to achieve desired effects during Defence operations, activities or services. The Program’s responsibilities also include the analysis, experimentation and validation of the joint capabilities, enablers and force structures to be integrated and implemented into the RCN and CAF, while ensuring interoperability with our domestic and international allies and partners. Planning is driven by changes in policy, actual or projected changes in the security environment and/or lessons learned from operations.

Program Stakeholders

Internal to RCN

• Director General Naval Force Development (DGNFD)
• Director General Future Ship Capability (DGFSC)
• Interested parties: All other organizations within the RCN

External to RCN

• Assistant Deputy Minister (Human Resources – Civilian), Assistant Deputy Minister (Information Management) (ADM(IM)), Assistant Deputy Minister (Infrastructure and Environment) (ADM(IE)), Assistant Deputy Minister (Materiel) (ADM(Mat)), Assistant Deputy Minister (Policy), Assistant Deputy Minister (Defence Research and Development Canada) Chief of Military Personnel
• The Royal Canadian Air Force, Canadian Army, Special forces, Canadian Joint Operations Command

External to Department of National Defence (DND)/CAF

• External (Public Services and Procurement Canada, Defense Industry)
• International (Five Eyes, North Atlantic Treaty Organization, etc.)

Program Resources

Figure 1 Summary

Program expenditures for the period include the restated figures for FY 2015/16 to FY 2018/19 and actual amount in expenditures for FY 2019/20.

In 2019, the NFD Program was supported by 66 military FD positions and 15 full- time equivalent civilian positions.²

Source:

1. Defence Resources Management Information System
2. Data provided by Program

Back to Table of Contents

Key Findings: Internal

Naval Force Development generates and manages the capabilities required for successful operations.

SSE states that with new investment, Canada’s Navy will be capable of meeting the anticipated defence and security challenges of the coming decades. This investment includes the revitalization of the entirety of the RCN’s major warship fleet. The scope of this revitalization results in a significant level of effort on the NFD Program in the Conceive, Design and Build pillars. The effort in these pillars will have long- lasting effects as the average lifespan of an RCN vessel is 40 years. This is in addition to the pillar requirements to manage the capabilities of the existing RCN fleets.

Results of the NFD program are that the RCN is equipped with strategically relevant, tactically effective and sustainable forces that are in line with CAF strategic direction in order to meet operational and SSE requirements.

The ultimate outcome for FD is that future defence capabilities are conceived based upon analysis of security trends, emerging threats, and science and technology foresight as they impact Canada’s maritime strategic mission.

Figure 2 Summary

Each current or future RCN vessel requires the capabilities to operate in an ever-changing threat and security environment. The capabilities of these new ships are required to be managed throughout the pillars. The NFD Program responsibilities include:

• Incorporating new capabilities such as the new capability in arctic operations with the Arctic Offshore Patrol Ship (AOPS).
• Reintroducing warfare capabilities, such as Area Air Defence capabilities that have been missing since the decommissioning of the Iroquois-class vessels in 2017 and will return with the introduction of the Canadian Surface Combatant (CSC).
• Enhancing sustainment capabilities such as the replenishment at sea and support that the Joint Support ship (JSS) will provide.
• Refreshing/updating existing warfare capabilities such as the updated combat management systems that will be in CSC.
• Maintaining existing capabilities such as the capabilities provided by the Halifax-class and Victoria-class submarines.

The RCN is adapting to meet the NFD demands of fleet revitalization while maintaining current capabilities.

In order to meet the significant NFD activity requirements noted in finding 1, the RCN is transforming the organization to improve support. The NFD Program now includes the following organizations:

• DGFSC provides dedicated FD support to the new capabilities. The subsections of DGFSC are:
  • Director Naval Major Crown Projects (DNMCP);
  • Director of Naval Innovation (DNI) is a focal point for innovation activities;
  • Director New Capability Integration (DNCI) is responsible for providing advice to Naval leadership, and for bridging the transition from “Build” to the steady-state "Manage" pillar; and
  • Director Digital Navy (DDN).

Figure 3 Summary

Figure 4 Summary

Although the RCN issued organizational directives and guidance, the transformation to the new FD organization is not formally completed. For example:

• The DGFSC organization does not appear on the RCN Defence Wide Area Network (DWAN) website;
• The RCN restructuring of the Maritime Capability Development Board is ongoing; and
• Blurred ARAs exist between the roles of DNI and Director Naval Requirements (DNR) which can lead to overlap in the work of the DNMCP and DNCI.

Once the structure is formalized and performance indicators are identified and collected, the validity of this structure, both for the RCN and other Force Developers, can be assessed.

Back to Table of Contents

Key Findings: External

The implementation of technical advances is required throughout the entire life of an RCN ship.

Continuous innovation and capability insertion are vital in all FD pillars to maintain a capability. An RCN ship takes decades to build and has to adapt to changing technologies over its average lifespan of over 40 years. This is comparable to conceiving a car’s capability requirements in 1995, having it delivered in 2018 and keeping it until 2050. This is a significant challenge due to the need to balance innovation in long-term capabilities within current non-agile procurement processes.

However, the RCN has made efforts to be proactive in incorporating innovative solutions to address these challenges. To support design and build, the RCN is utilizing  tools such as Simulation for Crew Optimization and Risk Evaluation (SCORE), a software package that enables crew modeling and design. It models the potential crew in multiple situations including fatigue issues and double assignments. This was used in JSS to optimize the crew both in number and taskings. SCORE is continually being improved and used for CSC and other capabilities.

The RCN created the DNI organization in November 2020 to provide a single focal point for RCN innovation. Its creation was found to promote stability in the relationship with industry. This organization has the potential to support innovation in all FD pillars. DNI manages the six focus areas for research and innovation.

The Innovation for Defence Excellence and Security is an example of an innovation program used by NFD. It was indicated to be effective for the RCN and allows the RCN to test an array of new capabilities and introduce them into the fleet faster than previously possible.

The consideration of PRICIEG* components is crucial for successful capability development.

A capability does not just consist of equipment, but needs to be enabled with doctrine, infrastructure, people and training. For example, a new ship requires a trained crew and infrastructure with IM support when alongside at the home port. The PRICIEG analysis framework is a tool utilized to measure and manage the impact of integration of capabilities into the CAF service.

The application of PRICIEG elements in NFD has been somewhat inconsistent, with disagreements regarding timing and focus.

100% of survey respondents indicated that PRICIEG is to be considered prior to implementation. However, there was evidence of some uncertainty regarding when PRICIEG should be started since only 71% indicated it commences in the design phase. Stakeholders indicated challenges in the areas of infrastructure and information management noting that detailed infrastructure planning needs to be considered early in projects in order to properly inform decision makers.

In response to the Project Approval Directive, case studies highlight that the RCN does generate a PRICIE analysis and that Gender- based Analysis Plus (GBA Plus) annexes are completed as part of formal project documentation.

However, GBA Plus is a relatively new incorporation, and since the current NFD projects are only in the options analysis and definitions phases, the final result of these efforts are unavailable.

The RCN requires the work across key enablers to ensure capability needs are met.

Collaboration, engagement and technical interoperability are important elements in the capability development process. Successful NFD relies on integration with various key enablers, including ADM(Mat), ADM(IM) and ADM(IE).

A natural tension exists between NFD and project enablers. NFD is focused on delivering capabilities while enablers tend to take a more project lens and operate under the three key constraints of scope, schedule and cost. This change to a project lens could result in misunderstandings of the capability gap that needs to be addressed.

Cost capability trade-offs are conducted throughout all of the project phases. Periodic reviews of cost- capability during the Design and Build phases provide the awareness required to make informed cost-capability trade-off decisions. For example, the Underwater Warfare Suite Upgrade was originally approved for six of the twelve Halifax-class vessels. When funds became available, more analysis was done on the operational impact and the RCN staffed a proposal to include all twelve ships.

In addition to working with key enablers to develop and deliver capabilities, consideration also needs to be given to how the navy works with its key partners, both internal and external to the CAF. Interoperability needs to be considered as part of ongoing capability requirements and includes the ability for the RCN to operate with other CAF elements, other government departments and key allies. While survey results denoted the RCN’s success in this area, interoperability needs to be seen as a cornerstone of NFD capability development and closely managed throughout a project’s life cycle.

The final step in capability delivery for NFD is integration into the fleet. The RCN created DNCI to provide advice to Naval leadership and to bridge the transition from “Build” to the steady-state "Manage" pillar. DNCI enabled the Frigate life extension project to be delivered to the fleet on time and on budget. With the success of the teams responsible for the integration of the Modernized Halifax-class vessels, a similar requirement for JSS, AOPS and CSC was put in place.

Naval Force Development is prioritizing activities to manage resource shortages.

Personnel challenges exist throughout the CAF, and the NFD Program is no exception. In addition to meeting the NFD challenges tied to the unprecedented level of activity in the Program:

• Interviewees and survey results noted a need for additional resources, particularly Project Directors.
• The lack of FD personnel and high turnover were found to be recurring issues which reduced program efficiency and led to loss of institutional knowledge. For example, the current Director of Naval Strategy is the first in seven years to remain in this role for more than one year.
• The lack of formal FD training, noted in finding 7, increases the impact of personnel turnover.

The RCN mitigates the resource challenges by concentrating resources on the higher priority capability issues. Focusing resources and close monitoring of progress supports the RCN in optimizing the FD personnel to move forward vital capability enhancements. Monitoring and optimizing the limited Naval FD personnel is vital to ensure that the most relevant capabilities are achieved and integrated into the RCN. In addition, recently, the DNI liaised with Director Naval Personnel & Technology about a new personnel software to match personnel to work requirements. This will not address the shortage of personnel but does ensure personnel are better suited to their roles.

Figure 5 Summary

Figure 6 Summary

A force development training structure would generate FD expertise both in the RCN and the CAF.

There is a lack of a formal FD training and certification program. This results in varying levels of expertise and a significant amount of on-the-job training that is necessary for personnel to be productive and effective in their roles.

Interviewees and survey respondents noted that training would be more effective if provided before employment in FD positions. A member new to FD is on a steep learning curve, often proficient just in time to move on to their next posting. Interviewees noted this challenge particularly in Naval Strategy Directorate.

There is available training that supports FD activities, such as the Project Manager training provided by ADM(Mat). However, this training does not have a FD-focused component, nor does it provide a formal FD certification. It is possible that this existing training can be leveraged as components of a formal FD certification curriculum.

The RCN is currently supporting a Chief of Force Development-led activity to develop a formal training program for Project Directors.

Figure 7 Summary

A formal FD training structure for initial employment postings and refresher training could support the generation and maintenance of FD expertise both in the RCN and across the CAF.

“Training for FD staff should be timely and relevant. In some cases people don't get training until they have been in the job for over a year making it nearly irrelevant…”

Back to Table of Contents

Conclusions

The NFD Program is vital to support the RCN fleet revitalization identified in SSE. The revitalization of the entire warship fleet while concurrently maintaining current capabilities resulted in an unprecedented level of NFD activity.

Back to Table of Contents

Annex A – Findings and Recommendations

Back to Table of Contents

Annex B – Management Action Plan

Management Action 1

1.1 Update terms of references for DGNFD, DGFSC and all subordinates Directors.

OPI: DGNFD, DGFSC
Target Date: July 2023

1.2 Complete all Establishment Changes required to reflect the new NFD organizational structure.

OPI: DFGNFD, DGFSC
Target Date: September 2022

1.3 Create a DGFSC page under the RCN DWAN website. 

OPI: DGFSC
Target Date: July 2022

Back to Table of Contents

Annex C – Evaluation Context

This report presents the results of the evaluation of the NFD Program conducted by ADM(RS) as a component of the DND Five-year Evaluation Plan (FY 2017/18 to FY 2022/23). It was approved by the Performance Measurement and Evaluation Committee in March 2020, and program representatives were identified to help guide the evaluation.

The evaluation examines the relevance and performance of the NFD Program during the period of FY 2015/16 to FY 2019/20 and was conducted in accordance with the Treasury Board Policy on Evaluation. Although a 2016 evaluation was conducted on Capability Development in DND, this is the first evaluation of the NFD Program as defined in the DRF, established in 2017.

The NFD Evaluation is one of three concurrent FD evaluations conducted in FY 2020/21. Findings from this evaluation, alongside findings from the Land Force Development Evaluation and the Air and Space Force Development Evaluation are presented in the ISA.