Evaluation of Defence Capability Development Program

November 2017

1258-3-003 (ADM(RS))

Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

Alternate Formats

PDF Version
(596 KB, 55 pages)

Assistant Deputy Minister (Review Services)

ADM(IM): Assistant Deputy Minister (Information Management)
ADM(RS): Assistant Deputy Minister (Review Services)
BCA: Business Case Analysis
C4ISR: Command, Control, Communications, Computers, Intelligence, Surveillance and Reconnaissance
CA: Canadian Army
CAF: Canadian Armed Forces
CANSOFCOM: Canadian Special Operations Forces Command
CBP: Capability Based Planning
CDS: Chief of the Defence Staff
CFAWC: Canadian Forces Aerospace Warfare Centre
CFINTCOM: Canadian Forces Intelligence Command
CFJP: Canadian Forces Joint Publication
CFMWC: Canadian Forces Maritime Warfare Centre
CFWC: Canadian Forces Warfare Centre
CIPPR: Capital Investment Program Plan Review
CJOC: Canadian Joint Operations Command
CMP: Chief Military Personnel
DCB: Defence Capabilities Board
DGCSI: Director General Capability and Structure Integration
DND: Department of National Defence
DPS: Defence Procurement Strategy
DSI: Director Structure Integration
FCG: Force Capability Guidance
FTE: Full-Time Equivalent
FY: Fiscal Year

GC: Government of Canada
IP: Independent Review Panel for Defence Acquisition
IRMC: Independent Review Panel for Defence Acquisition
IRPDA: Independent Review Panel for Defence Acquisition
JDB: Joint Doctrine Branch
KMS: Knowledge Management System
L1: Level 1
LL: Lessons Learned
M&S: Modelling and Simulation
NATO: North Atlantic Treaty Organization
OCI: Office of Collateral Interest
OPI: Office of Primary Interest
PAA: Program Alignment Architecture
PAD: Project Approval Directive
PRICIE: Personnel; Research and development / operations research; Infrastructure and organization; Concepts, doctrine and collective training; Infrastructure; Equipment and material
RCAF: Royal Canadian Air Force
RCN: RCN
SJS: Strategic Joint Staff
S&T: Science and Technology
TBS: Treasury Board Secretariat
UAS: Unmanned Aircraft System
UOR: Unforecasted Operational Requirement
UK: United Kingdom
US: United States
VCDS: Vice Chief of the Defence Staff

Overall Assessment

The Capability Development Program remains highly relevant to assure the proper and effective development of capabilities for the CAF.
The program is effective but can be improved with increased attention to joint capabilities and in revising the role of Chief of Force Development (CFD).
Further efficiencies can be obtained in the program through greater use of sustainment planning, modelling and simulation, and in staff training.

Executive Summary

In accordance with the Treasury Board Secretariat’s (TBS) Policy on Evaluation and the Department of National Defence and the Canadian Armed Forces’ (DND/CAF) Five-Year Evaluation Plan, Assistant Deputy Minister (Review Services) (ADM(RS)) conducted an evaluation of the DND/CAF Capability Development Program. The evaluation covered the period of 2010 to 2016. This is the first evaluation of the Capability Development Program conducted by ADM(RS). The purpose of this evaluation was to collect and analyze evidence on the relevance, design, implementation and outcomes of activities related to capability development.

The Defence Capability Development Program provides the analytical basis and knowledge to anticipate foreseeable changes in the threat and security environment and to determine the associated demand for defence capabilities across near and long-term time horizons. This is required in order to enable evidence-based strategic decisions that align the introduction, modification and divestment of defence capabilities and guide the application of existing capabilities within acceptable levels of risk.

Relevance

The program is required to support the development of effective capabilities for the CAF. It meets the three core criteria for ongoing relevance: there is a continued need for the program; it aligns with federal roles and responsibilities; and it aligns with Government of Canada (GC) priorities and with DND/CAF priorities and strategic objectives.

Performance

The Capability Development Program is effective. Evidence suggests that there are mature processes and mechanisms in place to identify capability gaps across the DND/CAF. In addition, sufficient tools, research and experimentation exist to support examining options to resolve the capability gaps. The use of the tools and practices needs to be continually monitored in order to confirm that it is effective. Additionally, the challenge function that is so widely employed in the program should be maintained. Sufficient mechanisms exist to integrate capabilities and guard against duplication in both equipment and structure. The Capability Development Program is governed effectively across the DND/CAF, which has enabled the preparation of a well-identified family of proposed future capabilities.

The DND/CAF’s ability to monitor affordability and project priorities through the Investment Plan (IP) and the Capital Investment Program Plan Review (CIPPR) has promoted efficiency within the program. Efficiencies can be further sought by developing a comprehensive experimentation and modelling and simulation (M&S) campaign plan and by improving the training and development of FD staff.

Table 1 lists the evaluation’s key findings and their respective recommendations.

Table 1. Key Findings and Recommendations of the Evaluation. This table lists the evaluation’s key findings and recommendations.
Key Findings	Recommendations
Key Finding 1. The joint concepts, doctrine and lessons learned (LL) are not fully exploited in support of the Capability Development Program.	Recommendation 1. The conceptual components of joint warfighting, in particular concepts, doctrine and joint LL, need to be properly organized, institutionalized and exploited to improve pan-CAF capability development. OPI: Vice Chief of the Defence Staff (VCDS) Recommendation 2. The joint LL database needs to be updated and improved to increase its functionality. OPI: Canadian Joint Operations Command (CJOC) OCI: VCDS, Assistant Deputy Minister (Information Management) (ADM(IM)) Recommendation 3. VCDS and CJOC should review the purpose, role, functions and responsibilities of the Canadian Forces Warfare Centre (CFWC). OPI: VCDSOCI: CJOC
Key Finding 2. The CFD’s role in the standardization of the Capability Development Program and in the development of joint enablers requires to be revalidated and documented.	Recommendation 4. The role, functions and organization of the CFD need to be revalidated and formalized, including identifying the necessary governance structures to ensure the pan-CAF development of joint enablers. OPI: VCDS
Key Finding 3. Greater use of M&S could make the Capability Development Program more efficient and economical.	Recommendation 5. CFD should develop a comprehensive experimentation and M&S campaign plan for its internal program, and all Level 1s (L1) with capability development responsibilities should exploit to the maximum extent possible experimentation and M&S in their conceive and design phases. OPI: VCDS
Key Finding 4. Capability development staff should be more carefully selected, trained and supported.	Recommendation 6. VCDS should conduct a training needs analysis for capability development staff, which should then influence assignment, training and core activities. OPI: VCDS OCIs: All L1s with capability development staff

Table 1 Details - Key Findings and Recommendations of the Evaluation

Back to Table of Contents

1.0 Introduction

1.1 Context for the Evaluation

This report presents the findings and recommendations of the evaluation of the Defence Capability Development Program. It examines its relevance and performance for the period 2010 to 2016. ADM(RS) conducted this evaluation as a component of the DND/CAF Five-Year Evaluation Plan (2012/13 to 2016/17), and did so in accordance with the TBS Policy on Evaluation (2009). This is the first evaluation of the Defence Capability Development Program in its entirety. A number of audits and other evaluations touch on capability development issues and are discussed in the report where relevant.

The findings in this report are based on review and analysis of program administrative and financial documents, literature review, country comparisons and interviews conducted with select program staff and/or offices of primary interest (OPI) and outside experts. Due to the program being distributed among multiple organizations, an advisory panel was not used. It should be noted that many outside agencies and actors influence the Capability Development Program. Observations may be included herein concerning their influence on the Capability Development Program; however no evaluation or assessment of these external actors was conducted.

1.2 Program Profile

1.2.1 Program Description

The Capability Development Program provides the analytical basis and knowledge to anticipate foreseeable changes in the threat and security environment. It determines the associated demand for defence capabilities across near to long-term time horizons. This enables evidence-based strategic decisions that align the introduction, modification and divestment of defence capabilities and that guide the application of existing capabilities with an acceptable level of risk.

The Capability Development Program includes force development at the national strategic level and capability development at lower levels. Force development is defined as a system of integrated and interdependent processes used to identify, conceptualize and implement necessary changes to existing capabilities or to develop new capabilities.^[1] Capability development is defined as the process by which capabilities are conceived, designed, built and integrated.^[2] The Capability Development Program focuses on determining and describing the conceptual force of the future (Horizon 3, which is 10 to 30 years out) and the operating concepts for the force of tomorrow, which are conceived and translated into capability requirements; that is, the predicted capabilities needed to operate in the near future (Horizon 2, which is 5 to 10 years out).^[3]

Most CAF L1s that have to generate or sustain a capability follow a capability development process. Capability development is a core activity, not only in the three Environments (Canadian Army (CA), Royal Canadian Navy (RCN), and Royal Canadian Air Force (RCAF)) but also in many other organizations, such as Canadian Forces Intelligence Command (CFINTCOM) and Canadian Special Operation Forces Command (CANSOFCOM). CJOC has a limited capability development capacity. Most capability development staffs follow a four-step process as denoted in Figure 1. These steps are as follows:

Conceive – Operating concepts are conceived and translated into capability requirements.
Design – Selected capability requirements are translated into valid designs for force employment—doctrine and structures.
Build – This step integrates the PRICIE^[4] components to realize the implementation of the capability.
Manage – This step manages the day-to-day use of the capability in keeping with readiness requirements.

Figure 1. The Defence Model and the Force Development System.^[5] This figure demonstrates the capability development process, including the conceive, design, build and manage phases.

Figure 1 Details - The Defence Model and the Force Development System

Figure 1 Summary: This figure demonstrates the capability development process, including the conceive, design, build and manage phases. It consists of a process map that has the Defence Model at the top and the Force Development System below. The Defence Model is represented by three blue elongated hexagons aligned horizontally and linked together by a horizontal line. The hexagon on the left is labelled DM/CDS Engagement and Decision, the one in the middle is labelled Overarching Defence Management, and the one on the right is labelled Department Financial Management. Directly below the hexagons are four small side-by-side coloured squares. The left-most square is blue and labelled Policy, the next one to the right is green and labelled Plan, the next one is purple and labelled Programme, and the right-most square is orange and labelled Delivery and Budget.

The lower section of the figure is the Force Development System, which consists of four large side-by-side coloured rectangles (whose long side is vertical). The two rectangles on the left are green and labelled Conceive and Design, respectively. The two rectangles on the right are purple and labelled Build and Manage, respectively. A green diagonal line leads from the green Plan square (above) to the conceive rectangle, and a purple diagonal line leads from the purple Program square (above) to the Manage rectangle.

Along the midline of the Force Development System rectangles there are several elements layered on top of each other as follows:

On the far left there is a blue hexagon (half of which covers the left-most part of the Conceive rectangle), labelled L0 CF/DND Strategy.
Immediately to its right, there is a dark green rectangle (whose long side is horizontal), labelled Capability Based Planning Process. It is located directly in the middle, and atop, the Conceive and Design rectangles.
Immediately to its right, there is another rectangle with gradient colouring from dark green on the left end, where it sits atop the right-most portion of the Design rectangle, to dark purple, where it sits atop the Build and Manage rectangles.
Atop and overlapping both the Capability Based Planning Process rectangle and the Defence Project Management System rectangle is blue hexagon labelled FCG MYEP (which, according to the legend located in the top left corner of the image, means Force Capability Guidance Multi-Year Establishment Plan). It sits above the rectangle representing the Design phase.
Diagonally below and to the right of the FCG MYEP hexagon is another blue hexagon labelled FP&R (Force Posture and Readiness) Decision. It overlaps the rectangles representing the Design and Build phases.
Immediately to the right of the FP&R Hexagon is a long, slim grey rectangle (whose long side is horizontal), labelled Force Posture and Readiness Process (SJS). It sits at the very bottom of the rectangles representing the Build and Manage phases and overlaps both of them.
The next layer consists of four small yellow circles as follows:
- The first and left-most yellow circle is labelled ID DCB, meaning Identification Stage – Defence Capability Board. It sits slightly below, yet overlapping the lowest portion of the FCG MYEP hexagon, as well as slightly touching the top-left corner of the FP&R Decision hexagon. It is entirely above the rectangle representing the Design phase.
- The second yellow circle, located slightly to the right of the first, is labelled OA DCB, meaning Option Analysis Stage – Defence Capability Board. Its top half overlaps the Defence Project Management System rectangle, and its bottom section slightly overlaps the FP&R Decision hexagon. It sits above the exact midpoint between the rectangles representing the Design and Build phases.
- The third yellow circle is labelled DEF PMB, meaning Definition Stage – Project Management Board. It is located slightly further to the right of the second yellow circle. Its top half overlaps the Defence Project Management System rectangle. IT sits above the rectangle representing the Build phase.
- The fourth and right-most yellow circle is labelled IMP PBM, meaning Implementation Stage – Project Management Board. Its top half overlaps the Defence Project Management System rectangle, and it sits at the far right edge of the rectangle representing the Build phase.

Along the very bottom of the figure, there is a long horizontal orange rectangle labelled Lessons Learned / Urgent Operational Requirements / Capability Deficiencies. It has three arrows: one pointing up from “Lessons Learned” to the rectangle representing the Conceive phase, one pointing up from “Urgent Operational Requirements” to the rectangle representing the Design phase, and one pointing up from “Urgent Operational Requirements” to the rectangle representing the Build phase. At the bottom right of the figure, there is another orange arrow curling downwards from the rectangle representing the Manage phase and pointing to “Capability Deficiencies.”

To the far left of the entire figure, there is an image of “Strong, Secure Engaged: Canada’s Defence Policy.”

For the three Environments, their respective warfare centres serve as key components in the coordination of the conceive and design phases of their projects. Most L1s have a requirements staff consisting of subject matter experts who help in the translation of project documents dealing with capability deficiencies.

At the strategic level, the CFD is responsible to the Chief of the Defence Staff (CDS), through the VCDS to harmonize, synchronize and integrate the force development activities of the CAF in order to develop the capabilities required to produce strategically relevant, operationally responsive and tactically decisive military forces.^[6] During the conceive and design phases, the Capability Development Program, is primarily concerned with identifying gaps in current or future defence capabilities and determining the preferred solutions to these identified gaps. The bulk of CFD’s activities are focussed on these two phases. CFD delivers force development through the following four key mechanisms:

serving as the champion for joint enabling functions;
conducting capability based planning (CBP) in order to produce the force capability guidance (FCG);
ensuring the standardization of submissions to strategic project review boards, like the Defence Capabilities Board (DCB), as well as assuring project affordability within the IP; and
monitoring and coordinating quantitative strategic manning issues through the Multi-Year Establishment Plan.

1.2.2 Program Objectives

The Capability Development Program’s objectives are as follows:

establish and monitor the fulfillment of near-term targets for readying force elements and conducting defence operations;
identify lessons from past operations;
assess defence and security trends;
develop and integrate new knowledge and systems/methods for conducting operations;
develop approaches and conduct defence capability analyses at strategic, operational and tactical levels;
conduct present-to-future capability assessments;
design and assess defence alternatives;
provide defence capability oversight and expertise; and
plan for sustainable defence capabilities in future time horizons.^[7]

1.2.3 Stakeholders

The key stakeholders are the CDS, the Deputy Minister (DM), the VCDS and the Environmental Chiefs of Staff. In addition, all L1s and commanders who have a significant role in force employment, force generation, force support and force development are stakeholders, including CJOC, who is the end user for the forces prepared by the other stakeholders. The program is delivered through VCDS/CFD, Strategic Joint Staff (SJS), CJOC (including the CFWC, CANSOFCOM, CFINTCOM, Chief Military Personnel (CMP)) and the Environments (RCN, CA and RCAF, including their respective warfare centres).

Back to Table of Contents

1.3 Evaluation Scope

1.3.1 Coverage and Responsibilities

The evaluation covered Program Alignment Architecture (PAA) Strategic Outcome 5.0 – Defence Capability Development and Research and the following PAA sub-elements: 5.1.1 – Capability Design and Management, 5.1.2 – Concept, Doctrine Development and Warfare Experimentation, 5.2 – Strategic Capability Planning Support and 5.2.2 – Strategic Force Posture Planning Support.

PAA sub-element 5.1.3 – Science and Systems Development and Integration was not included in this evaluation as it was the subject of a separate evaluation in fiscal year (FY) 2013/14.

The evaluation focussed primarily on activities, processes and outcomes in the conceive and design phases of force/capability development and not on the specific deliverables, such as the CBP Final Report or the FCG documents.

1.3.2 Resources

Departmental program spending coverage of this evaluation was $227,447,030 in FY 2014/15 and $194,210, 834 in FY 2015/16. Note that the variance from FY 2014/15 to FY 2015/16 was caused primarily by the use of different attribution rules within the PAA model.

The full-time equivalent count for FY 2015/2016 was 831 (696 military and 135 civilian).

Table 2. PAA Element 5.0 Costs for FY 2011 to 2016. This table lists the funds expended in the four capability development PAA elements from 2011 to 2016.
PAA Elements	Costs for Capability Development ($ millions)^[8]
PAA Elements	2011	2012	2013	2014	2015	2016
5.1.1 – Capability Design and Management	46.0	50.8	58.4	47.1	72.4	88.6
5.1.2 – Concept, Doctrine Development and Warfare Experimentation	70.5	78.2	74.2	81.5	77.8	68.7
5.2.1 – Strategic Capability Planning Support	52.7	39.9	21.3	30.1	75.1	33.0
5.2.1 – Strategic Capability Planning Support	2.7	2.9	3.2	1.9	2.1	3.8
Total Capability Development Costs	171.9	171.8	157.1	160.6	227.4	194.1

Table 2 Details - PAA Element 5.0 Costs for FY 2011 to 2016

1.3.3 Issues and Questions

In accordance with the TBS Directive on the Evaluation Function (2009),^[9] the evaluation addresses the five core issues related to relevance and performance. The methodology used to gather evidence in support of the evaluation questions can be found at Annex B. An evaluation matrix listing each of the evaluation questions, with associated indicators and data sources, is provided at Annex D. Not all indicators may be discussed in the report and some will be modified from their original form.

Back to Table of Contents

2.0 Findings and Recommendations

2.1 Relevance—Continued Need

This section confirms that there is a continuing need for the Capability Development Program within the DND/CAF. It uses the following indicator to make this determination:

The Capability Development Program addresses the future needs of the CAF.

Finding 1: There is a continuing need for the Capability Development Program.

The CAF is required to respond to the national security and defence missions assigned to it by the GC. It does so through the preparations entailed in its Force Posture and Readiness directives and through general preparation and planning. Planning includes considering the typical missions the CAF will be assigned and the resources required to carry them out. Given the variety of potential missions, the CAF requires a broad variety of capabilities available both from the Environments and from many other elements. At any particular point in time, the CAF has a vision of how it would prefer to conduct a specific mission. This is normally described in concepts and doctrine, forces earmarked at various stages of readiness for potential missions and a suite of existing equipment.

Each major operation the CAF conducts is unique unto itself. How CAF capabilities will be employed in a theatre of operation varies greatly depending on factors ranging from weather, to terrain, to the specific adversary. These unique factors require deployed forces to adapt quickly to remain operationally effective. These factors might establish the need for a reorganization of the forces involved, the need for modification to existing equipment, the need for new equipment and even the need for new capabilities for future forces.

While the nature of war may be regarded as immutable, the conduct of war is continually changing. For instance, warfighting techniques and doctrines that were appropriate for the 1960s, such as how counterinsurgencies were fought, may not be suitable in today’s conflicts. These changes are often evolutionary, and occasionally revolutionary. The means are the physical component of war. Although change in organizational structures is also important, change in the physical component is best represented by the continual technological enhancement of the equipment used in warfighting. For example, the modern armoured fighting vehicle is a vast improvement over the original versions, and new platforms, such as Unmanned Aircraft Systems (UAS), can significantly affect how certain combat functions are performed. The ways are the conceptual component of war, and getting them correct can be critical for overall strategic success and ensuring the right means are procured.

A program is required to assure that the CAF can maintain, sustain and improve on the conceptual and physical components of military power. Thus, there is a continuing need for the Capability Development Program to ensure that the CAF is properly informed, organized, trained and equipped to face the challenges of the missions and tasks that will be assigned to it in the future.

Back to Table of Contents

2.2 Relevance—Alignment with Federal Roles and Responsibilities

This section confirms that the Capability Development Program is aligned with the federal roles and responsibilities of the GC. The following indicator was used to assess the alignment with federal roles and responsibilities:

alignment to GC roles and responsibilities.

Finding 2: The Capability Development Program aligns with federal roles and responsibilities.

National defence is an activity solely within the purview of the GC. It fulfils its roles and responsibilities through explicit and implicit actions. The GC’s roles and responsibilities for national defence are expressed explicitly in legislation, Speeches from the Throne, federal budgets and defence policy statements. For example, under the National Defence Act, Section 36, the Minister of National Defence is responsible for deciding how the CAF will be equipped. The GC responsibility is often expressed in The Speech from the Throne. For instance, the Speech opening the Second Session of the 41st Parliament noted that the government “has been living up to our promise to give the Canadian military the tools it needs to get the job done.” More recently, the Minister’s Mandate Letter called for a well-equipped and ready force.^[10]

The GC’s roles and responsibilities are expressed implicitly in how the GC organizes federal departments and agencies to carry out the defence of the nation and in allocating resources to perform this function. Appropriating funds for capability development within the broader defence budget and providing expenditure authority for specific proposals through project approval mechanisms are part of the GC’s overall roles and responsibilities.

The Capability Development Program is the entry point for projects that will eventually lead to defence procurement. As such, it aligns with the GC requirement to assure that the CAF is properly equipped.

Back to Table of Contents

2.3 Relevance—Alignment with Government Priorities

This section confirms that the Capability Development Program is consistent with current GC and DND/CAF priorities. The following indicators were used to assess the alignment with priorities:

alignment to GC priorities; and
alignment to DND/CAF priorities and objectives.

Finding 3: The Capability Development Program aligns with GC priorities and with DND/CAF priorities and strategic objectives.

The national security of Canada and the strategy to obtain it are responsibilities of the GC. The roles, missions and tasks assigned to the DND/CAF are at the discretion of the GC. The following three core roles of the CAF have been largely constant for decades:

defend Canada;
defend North America; and
contribute to international peace and security.^[11]

The GC determines the priorities for the missions and operations within these roles through explicit statements, such as the desire to reengage in peacekeeping noted in the Minister’s Mandate Letter, and through demonstrable practice, such as the deployment of trainers to Iraq and of troops in support of the North Atlantic Treaty Organization (NATO). Regardless of the assigned operation, CAF elements are typically deployed as tactically self-supporting units. This practice helps ensure that any Canadian contribution can be combat effective regardless of its overall role within a coalition structure. The Capability Development Program aligns with GC priorities as it seeks to develop the capabilities necessary within the CAF to allow it to perform its roles, mission and tasks.

The GC sets other priorities for defence purchases as detailed in the Defence Procurement Strategy (DPS). The Capability Development Program aligns with the DPS through the publication of the annual Defence Acquisition Guide and through a more thorough early engagement with industry.

The Capability Development Program is aligned with the DND/CAF priorities and strategic objectives through internal work within the program. In particular, the outcome of the CBP process, the FCG, helps ensure there is alignment between the capabilities needed and the capital funding available.

Back to Table of Contents

2.4 Performance—Achievement of Expected Outcomes (Effectiveness)

This section evaluates the achievement of the Capability Development Program’s expected outcomes with a focus on the following:

Immediate Outcome 1: Capability gaps are identified
Immediate Outcome 2: Options are selected to resolve capability gaps
Immediate Outcome 3: Selected options are integrated into existing warfighting systems
Immediate Outcome 4: The Capability Development Program is properly governed
Intermediate Outcome 1: The proposed future family of CAF capabilities is identified and validated

2.4.1 Immediate Outcome 1: Capability gaps are identified

The evaluation used the following indicators to assess the outcome:

The CAF maintains situational awareness on the changes and evolution of the security environment, which may create capability gaps.
Context documents and processes exist that support the identification of capability gaps.
Concepts, doctrine, LL and warfare centres contribute to the identification of capability gaps.

Indicator: The CAF maintains situational awareness on the changes and evolution of the security environment, which may create capability gaps.

Finding 4: Mature processes and mechanisms to identify capability gaps exist across the DND/CAF.

Strategic planning involves preparing an organization for the future. It is now common within strategic planning to model, project and publish an assessment of the challenges the future security environment (FSE) may hold. These projections are developed by both military and civilian planners. From a military perspective, considerations of the future figure largely in the planning documents of many of Canada’s key military partners.^[12] From a civilian planning perspective, many organizations, including think tanks and non-governmental agencies, create their own predicted visions of the future. For instance, the Center for Strategic and International Studies produced Defense 2045: Assessing the Future Security Environment and Implications for Defense Policy Makers (2015), and the World Economic Forum produced Global Risks 2015, 10th Edition.

The evaluation team found that the DND/CAF maintain situational awareness on changes in the security environment in two ways. First, the potential issues in the FSE are refreshed annually in the CDS’s direction on force posture and readiness. Second, an FSE document is maintained and updated to support strategic planning.

CAF capability development is focussed on the capabilities that will be needed in the future, typically in Horizon 2 (5 to 10 years) or Horizon 3 (10 to 30 years). The predicted future capabilities requirements of the CAF must be based on some projection of what the GC will demand of it in the FSE. The DND/CAF uses the FSE document to provide a standardized baseline for CBP. The first FSE was produced in 2009 as The Future Security Environment 2008-2030, Part 1: Current and Emerging Trends. It was revised and re-issued in 2014 as The Future Security Environment.

There are benefits and limitations to the use of the FSE construct. The core benefits of the construct are that it serves as a baseline to allow ongoing monitoring of potentially adverse security trends and to allow long-term capability development planning.^[13] However, any assessment of the future does not prevent strategic surprise, indicate what potential opponents may arise, reveal where the next commitment of the CAF will occur or elaborate the concepts and means to counter a projected trend.

Indicator: Context documents and processes exist that support the identification of capability gaps.

Context documents and processes exist at most levels to support the identification of capability gaps. This includes the Unforecasted Operational Requirements (UOR) process for operations, the CBP as part of the broad departmental strategic planning process, the DND/CAF capability development process and the force generators with their own particular subordinate models of capability development.

The future is inherently unknowable, and the complexity of capability development decisions and plans can often leave capability gaps when operations are undertaken against a specific adversary or in response to an unforeseen security situation. For instance, few predicted that Canada would be involved in a lengthy campaign in Afghanistan. Once troops are deployed to a theatre, capability gaps become quite evident and are normally resolved through projects identified as UORs.

With the passing of the Soviet threat, most western militaries transitioned at the strategic level from threat-based planning to CBP, a process developed by RAND analyst Paul Davies in the early 2000s.^[14] By 2010 CBP was considered the gold standard of defence planning.^[15] The CAF has been using CBP for well over a decade. CBP is a key decision aid that assists senior leadership in making force development choices to ensure critical capabilities exist to assure our national security in the 5 to 30-year timeframe. It has allowed the DND/CAF to move beyond the sole use of professional judgement in making complex defence purchasing decisions.^[16] It has evolved considerably over this period and is now a relatively mature process.^[17] A CBP cycle takes three years to complete with two complete cycles having been competed since 2008: one in 2012, where eight scenarios were assessed, and one in 2016, where 10 scenarios were assessed. The final output of CBP is the FCG. The FCG sets priorities for capability development, which then informs the development of the IP and the Multi-Year Establishment Plan. The detailed philosophy, logic and mechanical processes that comprise the CBP process are outlined in detail in the CFD Capability Based Planning Handbook. The oversight mechanism for CBP is the DCB. The CBP is briefed to the DCB at the end of each phase of the process so that guidance can be provided as required. At the conclusion of the assessment phase, the DCB endorses and the CDS approves the future force design and CBP

Final Report. The CFD, who is responsible for the harmonization, synchronization and integration of the force development activities of the CAF, controls the standardization and execution of the CBP cycle.

The following are some general observations that can be made about the CBP process as it is currently practiced in the DND/CAF:

Canadian CBP is not based on a particular national security strategy.^[18] The ten CBP scenarios constitute a “most likely” projection of missions for the CAF based on historical evidence, with the assumption that similar missions will be assigned in the future.^[19] Our key allies have a more direct link between explicit national military strategies and their defined military capability requirements. For example, the United States (US) has the Quadrennial Defence Review, Australia has a formal review process, and the UK has recently enacted a four-year national military strategy review process.^[20] These nations can develop major long-term campaign plans for the delivery of core systems (e.g., the UK’s plans for the continuation of its aircraft carrier fleet) or for the delivery of specific platforms within the militaries (e.g., the US plan for the long-term viability of its UAS program). These long-term campaign plans benefit from being more directly linked to explicit national strategies.^[21]
Although capability development as a process was divested to the L1s in the late 1990s, CFD utilizes the CBP, the FCG, the IP and various “challenge function” boards to facilitate governance over force development in the DND/CAF. There are numerous benefits to this centralization. First, the role of CFD in capability development serves a crucial standardization function. This standardization has contributed to the credibility of projects submitted for approval. Second, while the FCG identifies critical capability gaps, it does not clearly constitute a vision for the future structure of the CAF as a fighting force. Additionally, the FCG will not have sufficient authority if not approved at the appropriate level.^[22] Third, centralized control ensures that joint requirements are heard and that reasonable consensus can be established over the future capability requirements of all CAF components.
The necessity to resolve UORs can cause three main problems to capability development. First, requirements for specific theatres of operation are not always consistent with the longer-term needs of the future CAF.^[23] Second, capabilities sought over the long term may be delayed or deferred due to the expertise and resources assigned to resolving immediate problems on operations. Third, processes and authorities for the completion of UOR-initiated projects have not always been clear.^[24]

Capability gaps are not addressed in a standardized fashion between force elements. The RCN, CA and RCAF processes are well documented, elaborate and transparent, and decisions made about capability development are easily tracked. For the Environments, many initial conceive and design issues are tasked out to their warfare centres. At a certain level of maturity, projects are passed to requirements staff, who then further develop them. While CFD prioritizes overall force development, the L1s develop the detailed individual projects that deliver the overall program. Newer L1s use a capability development approach, often modelled on the CA processes but aligned with their priorities and resources. CJOC is working on developing its own system. It can identify new and emerging capability requirements through after-action reports, LL and UORs but needs to address them through either CFD or the Environments’ force development staff.

Indicator: Concepts, doctrine, LL and warfare centres contribute to the identification of capability gaps.

Key Finding 1: The joint concepts, doctrine and LL are not fully exploited in support of the Capability Development Program.

The ability to properly identify the capabilities appropriate for the CAF to be successful in its missions is closely related to the integration of concepts, doctrine and LL, which establish and reinforce those needs. Clear concepts and doctrine help articulate why particular capabilities are needed, as well as their value to the defence of Canada. These issues are discussed in the following section.

Concepts

Nations employ their military capabilities in accordance with the perceived utility of their use. This perceived utility is underpinned by warfighting preferences based on theories and concepts. Similarly, the modern application of air power is focussed on precision. Concepts, proven or not, affect how operations are designed and executed. For example, peacekeeping has moved from the traditional state vs state model to one that emphasizes the protection of civilians. Additionally, operations in Afghanistan were influenced to a significant degree by the concept of the comprehensive approach.^[25] Conflicts, of course, do not wait for the concepts necessary for success to be worked out in detail. How to fight an insurgency was not well understood in the CAF, or in any other western military, prior to the conflicts in Iraq and Afghanistan.^[26]

Fundamental concepts, whether for current operations or future challenges, are needed to underpin capability development at all levels. The selection of a warfighting concept has a core importance to the doctrine developed, the equipment selected and the training conducted. Incorrect or undeveloped conceptual choices can be a great detriment to future combat effectiveness. Understandably, it is important that concepts be published to encourage debate and the dissemination of common perspectives. Some of this debate occurs in the professional journals of the CAF and, to an extent, among defence intellectuals in Canada. Stakeholders require an ability to develop concepts to ensure that capability gaps in current and future operations can be identified.^[27] Concepts underpin the conceive phase of capability development. The health of the conceptual undertakings of various stakeholders is as follows:

CA. Commencing a new cycle, the CA is fully invested in developing a concept of the future operating environment (2040 and beyond), and the “next” force employment concept for the Army of Tomorrow (circa 2035). For the current timeframe, the aspirational underpinnings are contained in Land Operations 2021: Adaptive Dispersed Operations – The Force Employment Concept for Canada’s Army of Tomorrow (2007). Preceding this work, yet published afterwards, was Toward Land Operations 2021: Studies in Support of the Army of Tomorrow’s Force Employment Concept (2009). These works were followed by Designing Canada’s Army of Tomorrow (2011) and most recently Waypoint 2018: The Canadian Army Advancing toward Land Operations 2021 (2015). This last document both checks and confirms that the CA is progressing toward its capability development goals for the Army of Tomorrow. These higher-level studies are used to inform most of the work on subordinate elements, such as light forces, in the capability development activities of the CA.
RCN. The RCN is also well invested in developing a vision for future naval forces. While a number of long-range planning documents appeared in the 1990s, the first substantial effort, Leadmark: The Naval Strategy for 2020, was published in June 2001 just before the September 11 attacks. This was updated in 2005 with Securing Canada’s Ocean Frontiers: Charting the Course from Leadmark. Most recently, the RCN has released Canada in a New Maritime World: Leadmark 2050 (2016) and The Future Maritime Operating Environment (Defence Research and Development Canada, 2016).
RCAF. The RCAF faced concept development challenges in the early 2000s.^[28] Since then, it has shown a marked improvement in conceptual work since the establishment of the Canadian Forces Aerospace Warfare Centre (CFAWC). This work is evident in the Projecting Power series Canada’s Air Force 2035 (2009), Alternative Futures for Canada’s Air Force in 2020 (2010) and Project Laminar Strike: Canada’s Air Force Post OP Athena (2011).
Joint. While the Environmental staffs are well engaged in developing concepts that underpin their visions of their future warfighting role, more work is required at the strategic/joint level to provide the conceptual underpinnings for future CAF operations. CFD formerly had a concept cell whose intent was to write the concept for each of the functional domains: command, sense, act, shield, sustain and generate. The main products of the cell were the Integrated Capstone Concept and the Arctic Integrating Concept, which were issued shortly before the cell was disbanded. Strategic concept work was then delegated to the CFWC, which mainly involved participation in the Multi-National Experiment series. Some strategic concepts have been completed by officers working on joint enablers, such as cyber and command, control, communications, computers, intelligence, surveillance and reconnaissance (C4ISR). Current plans are underway to revise and replace the Integrated Capstone Concept with a Joint Capstone Concept Outline. Generally, the DND/CAF has faced challenges developing higher-level operational concepts in areas such as targeting, information operations and use of UAS.^[29] The CAF could benefit from consolidating strategic-level concept development to prioritize and more fully explore critical emerging capabilities.^[30] In addition, more attention is required to ensure concepts that will inform future warfighting are validated before they are brought into use by the CAF.

Doctrine

The CAF definition of doctrine is “fundamental principles by which military forces guide their actions in support of objectives. It is authoritative but requires judgment in application.”^[31] More simply, doctrine explains the perceived “best way” to accomplish military objectives as framed by the cultural norms and perspectives of a particular military. It serves as an educational tool by providing a common basis for understanding operational thinking. This common frame of reference allows commanders greater flexibility and adaptability when confronted by a specific situation. Further, doctrine provides a structure for the integration of elements, units and larger forces, both within individual services and in joint and combined constructs, for the purposes of conducting operations and campaigns.^[32]

If capability development is the art of making military power, doctrine has a particularly important role. As I. B. Holley Jr. noted, “Unless the armed forces are guided by appropriate doctrines, greater numbers and superior weapons are no guarantee of victory.”^[33] Doctrine is important to capability development in three key ways. First, the pace at which a weapons system can be introduced is determined by the effectiveness of the procedures established to translate ideas into weapons and how to use them. Second, the introduction of a superior capability only gives a qualitative edge until effective counter-measures evolve. Therefore, doctrine must be developed and tested concurrently with the capability to maximize its benefit from its earliest introduction. Third, without effective doctrine, warfighting elements cannot be integrated into their greatest combat effectiveness. The status of doctrine development of various stakeholders is as follows:

CA. The CA has a comprehensive and detailed doctrine development program within the Canadian Army Doctrine and Training Centre. This is captured in the annual Army Doctrine Development Directive, which contains a CA Five-Year Doctrine Writing Plan. Most manuals are developed internally with the assistance of subject matter experts. Work often proceeds with the allied and partner armies within the American, British, Canadian, Australian and New Zealand Armies’ program construct.
RCN. The RCN utilizes a bottom-up approach to determining doctrine priorities. The Maritime Concepts Development and Experimentation Coordination Group is tasked to develop and review annually a rolling five-year plan.^[34] This organization then provides the recommended way ahead to the Maritime Capability Development Board. Its tactical doctrine is often, but not always, the same as NATO doctrine. To ensure its national acceptability and utility, the RCN works closely with other NATO nations in the development of these manuals through the Maritime Standardization Board. Common doctrine enhances interoperability amongst NATO maritime forces. Some of these manuals are available on the NATO Standardization Office website.^[35] Nonetheless, there are important RCN-specific tactical doctrine publications. RCN doctrine has to be flexible enough to work within NATO and when deployed in support of US forces (i.e., carrier strike groups or surface action groups).
RCAF. The RCAF operational-level doctrine was first written in accordance with the functional domains concept: sense, act, command, shield and sustain. Units, wings and task forces found it difficult to see themselves within this construct, so the RCAF operational-level doctrine is now being rewritten to reflect keystone core capabilities and keystone enabling capabilities. Manuals are being produced mostly by the CFAWC on control of the air, air attack, expeditionary air operations and other topics.^[36]
Joint. Doctrine is important at the joint level for a variety of reasons. First, major expeditionary operations typically involve joint and combined operations. That is, these campaigns are waged through the intermingling of capabilities, often from multiple nations, the individual Environments and the joint enablers to achieve greater strategic effect. Coordination is required not only because of the complexities of the situation encountered, but also because of the complexities of the coalition fielded. Joint doctrine is essential to clarify and describe how the assembled forces should operate together to the best advantage. Second, many aspects of warfare at the strategic and operational levels, such as targeting, require a detailed professional knowledge of joint capabilities. Third, properly prepared joint doctrine synchronizes and enhances environmental doctrine instead of being overwhelmed by it. Finally, many CAF enablers, such as the CFINTCOM and CMP, require the joint doctrine structure to host and deliver their manuals.
- Joint doctrine, as overseen by CFWC, has only been partially successful. There is an adequate governance structure in the Joint Doctrine Working Group, a detailed manual on how to write joint doctrine and elaborate rolling work plans.^[37] The main area of concern is that the work of the Joint Doctrine Branch (JDB) is process driven. Unfortunately, the JDB possesses little subject matter expertise on the content of the manuals it oversees, which was evident when it sought to publish Canadian Forces Joint Publication (CFJP) 4-0 – Support prior to Joint Ex-13, presuming it was ready. When an urgent requirement develops, the JDB has to seek outside experts to prepare a manual. This occurred for the Joint Targeting Doctrine manual, which could have been produced more rapidly if there had been resident expertise in CFWC. The staff can merely confirm whether the doctrine writing processes have been followed. This lack of internal expertise also affects the JDB’s ability to comment or contribute to NATO or allied doctrine debates and documents or ensure internal consistency across Canadian manuals. Lastly, the JDB has conducted a limited amount of international work with key allies on doctrine development in recent years.
- A doctrine manual should be reviewed every five years.^[38] A review of the joint doctrine hierarchy shows that a large number are well out-of-date—mostly the core operations manuals. For example, CFJP 3-3 Peace Support Operations has not been updated since 2002. Additionally, with the current intense interest in targeting, many manuals are in need of important revisions. Despite optimistic work plans, given the structure and capabilities of the JDB staff, it is unlikely that most revisions will be timely. When faced with a similar issue, Australia contracted out the revision and updating of several of its key publication.^[39] The JDB may have to consider similar action. These issues with the joint doctrine system were also detailed in the CRS March 2007 report, Evaluation of the Maintenance and Currency of Canadian Forces Doctrine.

Lessons Learned

There are always lessons to be drawn from operations and wars past and present. From trenches to tanks to targeting, the capabilities used to wage wars are constantly evolving. Battle alone could be considered the final criterion to determine the value of a weapon, a doctrine or an organization. Therefore, there needs to be a system; that is, a clear set of established procedures, to collect and analyze the facts and phenomena of the battlefield. Such a system would provide the detailed feedback required to make the sound adjustments and modifications necessary to capabilities that are not meeting operational needs, whether they are equipment, practices or structures. Capturing and then acting on LL is a key component to immediate adaptation in warfighting and long-term innovation within defence institutions. This is the purpose behind an LL system.

Most elements in the CAF have set up LL programs in response to the CDS direction in Defence Administrative Order and Directive 8010-1 for establishing the processes that add value to existing bodies of knowledge, and to correct deficiencies in areas of concept, policy, doctrine, training, equipment and organizations. While each program is unique to its environment or element, all their respective chains of command are required to steer implementation and execution of the LL. Most programs have established linkages with their allied counterparts, giving access to a much wider range of operational experiences. The CFWC provides the central management for the overall DND/CAF program. The status of various programs is described as follows:

CA. The CA has a mature LL process. The Canadian Army Lessons Learned Centre (ALLC) is part of the Canadian Army Doctrine and Training Centre. The ALLC is mandated to collect observations and actionable lessons from operations and training based on a command-driven critical topics list. Analysts use a variety of sources and procedures to collect observations and regularly consolidate the lessons identified for Army Command action under the doctrine, organization, training, material and policy lines of governance. LL teams are normally embedded with deployed CA elements. LL are communicated to the Army by the chain of command, supported by ALLC communication means, such as: Dispatches, a publication that allows experienced soldiers, usually experts in a specific subject to address LL on recent operations or training; Bulletin, a publication that provides soldiers the opportunity to share their experiences; and training posters that are thematic in nature and briefly summarize items of interest from Post Operation Reports, Post Exercise Reports and other related documents.^[40] While in Afghanistan, the CA had a very effective chain established where lessons identified in theatre were rapidly passed to training authorities in Canada to ensure training for operations was as up-to-date as possible.
RCN. The RCN has a mature LL process. The RCN formerly had a system of rolling up all LL within the fleet on an annual basis and then assigning the responsibility to solve key issues to designated officers. This process is being re-established. Details for the RCN’s LL program are well described on its website.^[41]
RCAF. The RCAF continues to invest in the development of its LL process. Post Op HESTIA (2010), the RCAF recognized the need for an LL program and published an implementation directive for the program in December 2011. The program is implemented under the guidance of the CFAWC, on behalf of the Commander RCAF, while the Deputy Commander RCAF and the division commanders are responsible for its execution. CFAWC has proven very effective in training individuals in the LL process, and other elements are course loading candidates on its training. The program is explained in great detail in the Air Force Lessons Learned Programme Manual (document B-GA-005-780/AG-001).
Joint. The Joint LL process is mature but undervalued. The CFWC Joint Lessons Learned Branch, on behalf of Commander CJOC, acts as the assigned LL authority. The Joint Lessons Learned Branch provides leadership for the LL community, harmonizes LL efforts across the DND/CAF and develops, delivers and manages common LL processes, procedures and tools, as well as Joint Lessons Learned training, education and communication programs. Overall harmonization of the program is controlled through the Lessons Learned Working Group (LLWG). LL are captured and shared through the CFWC sponsored Knowledge Management System (KMS) database (located on the DND intranet site). Although its importance is overstated, the details of the program are contained in CFJP A2 Lessons Learned (B-GL-025-0A2/FP-001, April 2015). The LL program at the joint level suffers from two main difficulties. First, the KMS database is dated and not as user friendly as it should be. This may dissuade individuals from exploiting the entries therein as the information may be more readily available elsewhere. Second, CJOC must continue to exploit the use of the reports produced. For example, little from the report from Joint EX 2013 was acted upon. A monitoring system has been introduced from the report from Joint Ex 2015 that may lead to better monitoring and implementation of results.

Clear processes have been established across the CAF to discover LL and then remedy the capability gaps identified. In general, the LL process could be improved by ensuring it includes the most salient points. More critically, the implementation of any lesson identified remains a command responsibility and requires due attention. The DND/CAF Organizational Learning Strategy’s goals to learn from our own experience, as well as from other organizations, will not be met if the implementation of LL is not monitored or liaison with other key organizations is not conducted.

Warfare Centres

The various warfare centres conduct baseline work that helps develop capability requirements. They normally contribute to and advise on the conceive and design phases. CFD conducted a review of the warfare centres in 2016, and some of the following comments repeat that study’s discoveries.^[42] Although they are not all structured similarly, warfare centres tend to host or have access to concept development, M&S, experimentation, doctrine and LL. Each of the environmental warfare centres are structured for the specific needs of their service. For instance, the Canadian Forces Maritime Warfare Centre (CFMWC) serves to assure the RCN is as tactically relevant and as capable as possible, whereas the CFAWC concentrates more on strategic, operational and educational issues. In general terms, the environmental warfare centres are the intellectual centres of development for future warfighting concepts for their services. However, environmental staffs and common educational facilities also have a role. In playing this key role, warfare centres must closely integrate the development of concepts and doctrine and monitor what is captured in the LL programs. Proper concepts, doctrine and LL are critical to the fulsome development of the necessary capabilities in the CAF. Improper or poorly developed concepts and doctrine can place the CAF at a serious disadvantage, particularly in the early stages of a campaign. This failing may be attributed to concept development that is linked to particular assumptions, to adversaries exploiting strategic surprise or to a lack of regard for an adversary’s historical warfighting norms. For instance, going into Afghanistan, the CAF had little familiarity with counter-insurgency or the broad parameters of mine warfare, but as the campaign progressed, operational performance improved. This highlights the importance of effective LL programs for specific operations.

The CFD study on the warfare centres also analyzed the assignment of CFWC to CJOC. The report noted that CFWC should be assigned to an appropriate Joint L1. However, in the absence of one, CJOC is the best alternative. CFWC has a world class virtual exercise capability, and this seems to be its prime mission under CJOC—to serve a force generation training role. It has also done good work over the last few years on targeting, once it was designated as the CAF Targeting Centre of Excellence. However, the CAF Targeting Center of Excellence could improve its functions by emulating those performed by the UK’s Development, Concept and Doctrine Centre.^[43] CJOC’s operational preferences and the internal CFWC’s focus on the battle labs lead to little emphasis placed on key joint disciplines. Targeting has long been a principle exploratory experiment at CFWC. CFWC experimentation may need to be prioritized and tailored to contribute to resolving key operational or joint capability gaps. While CFWC has performed well in its virtual exercise role, improvements are required in the area of joint warfare expertise.

ADM(RS) Recommendation

1. It is recommended that the conceptual components of joint warfighting, in particular, concepts, doctrine and joint LL, be properly organized, institutionalized and exploited to improve pan-CAF capability development.

OPI: VCDS

ADM(RS) Recommendation

2. It is recommended that the joint LL database be updated and improved to increase its functionality.

OPI: CJOC

OCI: VCDS, ADM(IM)

ADM(RS) Recommendation

3. It is recommended that VCDS and CJOC review the purpose, role, functions and responsibilities of the CFWC.

OPI: VCDS

OCI: CJOC

2.4.2 Immediate Outcome 2: Options are selected to resolve capability gaps

The evaluation used the following indicators to assess the outcome:

Tools are available, including business case analysis (BCA), to analyze the capabilities offered by various options.
Research and experimentation contribute to options analysis.
Proposals to address capability gaps are considered credible.

Indicator: Tools are available, including BCA, to analyze the capabilities offered by various options.

Finding 5: Sufficient tools, research and experimentation exist to support the examination of options to resolve capability gaps.

Good tools are available at various levels to support capability development in the conceive and design phases. The wide variety of tools used in support of CBP are fully described in the CFD Capability Based Planning Handbook (2014). The tools assure that the FCG, the final output of CBP, has been thoroughly analyzed and is credible. One of the fundamental tools is PRICIE. There is a wide variety of other tools that are used: a specific suite used in support of CBP; project delivery tools such as BCA, various forms of risk analysis, and costing methodologies; research support; and interaction with industry. Developing options is also supported by the challenge function that occurs. The application of these various tools and techniques is discussed in this section.

PRICIE is a widely used tool in capability development and is discussed in both the Capability Based Planning Handbook and the Project Approval Directive (PAD). Developed by the DND/CAF in the late 1990s, PRICIE encourages a more thorough systems analysis of the capability requirements, which better prepares the way for systems integration later in the capability development process. The elements in the CAF have adopted this methodology for baseline analysis. However, the details of options analysis must be carefully monitored. For instance, the 2005 CRS Audit of Construction in Support of Capital Equipment Projects stated that construction costs were frequently underestimated or ignored. Although PRICIE has led to a fuller consideration of complete system costs, project risk audits conducted between 2011 and 2016 indicated that proper construction estimates remain an outstanding issue.^[44]

In the design phase, the various options that could resolve a perceived capability gap are analyzed to determine which one will be further developed. Various techniques are available to aid in options analysis, including scientific research and modelling, as well as the TBS procedures for BCA.^[45] Options analysis has been improving because of the challenge function now used in capability development, but it can be constrained by four factors. First, project staffs must be sufficiently trained in options analysis techniques. Second, the breadth of research into potential options must be sufficient so that the solutions presented move beyond single platform/contractor solutions. Third, the high-level mandatory requirements and statements of operational requirements must remain as constant, clear and unchanging as possible.^[46] Fourth, cost trade-offs that may occur later in the capability development process must not affect the initial selection of the preferred option.^[47]

Risk analysis is conducted throughout the capability development process. The use of risk analysis in this process in the DND/CAF is extensive.^[48] Risk considerations determine who will be the appropriate project approval authorities. Risk analysis is conducted both during the CBP process and through various stages of project management.^[49] Risk analysis is included in the project management process to ensure senior decision makers can make informed decisions. Project risk, an assessment of the probability of failure and the consequences of failure, largely addresses technical, cost and schedule issues. Even given this extensive regulation, risk management practices are not always well founded, and continuous oversight of project risk management plans is required.^[50] Recent CRS audits have highlighted the need for revisions to the PAD and further training in risk management.

Perhaps the most difficult element in the early phases of concept development is preparation of cost estimates, even those that are just rough order of magnitude. The details of this issue have been identified in the 2013 CRS Internal Audit of Capital Project Cost Estimation. Cost estimates include the delivery price for the platforms themselves, plus the whole life costs (the cost to maintain them throughout their service lives), and, as a minimum, infrastructure and training costs. While the confidence levels for estimates are important in all phases of a project, early demands for a high level of accuracy are unrealistic. This is especially true where a platform is under development. Until the design is completed, prototypes tested, the production run set and maintenance procedure established, most costs can only be generally forecasted.^[51] This is the reality of the delivery of major defence projects. GC policy generally regards initial estimates as caps for the project vice a starting point of investment.^[52] This environment makes government decisions with respect to procurement difficult as the complete final cost may not be evident, but also tend to diminish the breadth of capabilities that the CAF will finally receive. This evaluation did not explore this issue more fully, but the UK policy of forming large capital contingencies may have potential to ameliorate this issue to a degree.^[53]

Indicator: Research and experimentation contribute to options analysis.

All stakeholders give direction to their associated defence scientists concerning their priorities for capability development. The CA has a tightly scripted plan with work filtered through the Army Science and Technology (S&T) Board where key thrusts are identified. The RCN in a similar fashion uses the Maritime S&T Coordination Group. Research and experiments in support of capability development vary in size and scope. They may be conducted as studies, make use of synthetic environments or involve actual units and personnel. Research and experimentation can give good evidence concerning potential force structures and potential system combinations for greater operational effectiveness that support capability development.

The quality of capability development achievable is not only related to internal subject matter expertise in the DND/CAF but also to the relationship of the institution with industry. Much of capability development involves maintaining a technological edge over potential adversaries. Though internal research and development in Canada may provide some general parameters of what is feasible, the pragmatic conversion into useable products will only occur with the participation of appropriate industrial partners. Industry has an effect on capability development choices in three main ways, as follows:

First, capability choices are affected by industrial capacities and in some instances availability of space on the existing production lines of larger platforms. Canada’s defence purchases are normally not large enough for a company to establish a separate production line just to satisfy DND/CAF requirements. Some of the major platforms acquired in recent years (e.g., C-130Js, C-17s and Chinook helicopters) have in part been successful because pre-existing production lines could be exploited. While the National Shipbuilding Procurement Strategy should deliver some industrial capacity to support RCN capability development over the long term, a similar strategy is not readily apparent for the RCAF or the CA.^[54] Where production lines are in the US, care must be taken to assure that the platform being built actually matches well-identified Canadian requirements. Costs may also increase on projects where production lines have been closed.^[55]
Second, given the long development times of most defence procurements, industry needs the best information possible and hard evidence of choice before they commit scarce resources to supporting Defence needs. Some information is shared through the DND/CAF’s participation in the annual Canadian Association of Defence and Security Industries conference. A larger step was taken through the development of the DPS in early 2014. As part of this strategy, the first iteration of the Defence Acquisition Guide, a rough first effort, was published in June of that year.^[56] This guide aligns with what the Australian and the UK government had already been doing for a number of years.^[57] The Defence Acquisition Guide has been improved since then, and further modifications are planned. These mechanisms promote greater and earlier dialogue with industry, but it has not been resolved yet as to how to engage continually with industry but not prejudice the government contracting process.
Third, a goal of the DPS is to enhance the development of the defence industries in Canada. The focus is on key industrial benefits immediately vice general regional benefits, which occurred under the previous policy. The near-100-percent industrial benefits offset policy, enables Canada to participate in the arms industry. In 2014, for instance, Canada exported approximately $676 million in military goods and technology.^[58]

In summary, a wide variety of good tools exist to identify gaps, analyze options and manage projects within the capability development process. None of these tools are necessarily easy to master. They therefore demand appropriate training and careful monitoring of their use. In addition, strong relationships need to be fostered with both defence scientists and industry to aid in the development of feasible capability solutions.

Indicator: Proposals to address capability gaps are considered credible.

It is essential that a sufficient challenge function exist to ensure that the DND/CAF proposals are credible. Mechanisms that carry out this challenge function are outlined in this section. They include internal bodies, external bodies, the public and national government audit agencies. Each force generator has a challenge function embedded within its respective capability development process. At the national level, the CFD staff serve as the first level of the challenge function. The CFD staff review all projects before they proceed to the DCB. Key aspects of the review include confirmation of strategic requirements and capability needs. CFD staff assess proposed projects against government policy, strategic direction and CBP outcomes including the final report and FCG. CFD staff also validate that projects have a confirmed source of funds by ensuring they are included in the IP or that they have been approved by the Investment and Resource Management Committee (IRMC) as key capability projects, which DCB will review pending a source of funds. CFD staff will also assist with the coordination of documentation for those projects that fall within the scope of the Independent Review Panel for Defence Acquisition (IRPDA). By assisting in the vetting of capability requests, the CFD staff ensure a standardized quality approach to higher approval authorities and thus minimize rework and revision. This centralized approach should not only improve submissions in general but ensure better information is available to make apportionment decisions.

The IRPDA is a body external to the DND/CAF that conducts a challenge function over defence procurements larger than $100 million and those that are considered prudent to monitor. The IRPDA conducts a detailed review of all major DND project proposals to ensure they are consistent and easily understandable.^[59] As a new organization, the IRPDA initially worked on developing its procedures and reviewed the first portion of departmental projects. Theoretically, once reviewed, projects would be more readily acceptable to the Treasury Board and would therefore move faster through the approval process.

The challenge function also occurs amongst academics and in newspapers as defence procurement, to a great measure, is done in the public eye. This debate can be highly useful for expanding upon and severely testing the options under consideration. At times, it can be supportive, but at others detrimental. A large part of the academic literature focuses on defence procurement overall and does contribute somewhat to the early phases of capability development. More platform-centric analysis, however, can be a good aid in further expanding concepts to fully explore capability gaps and enable the DND/CAF to be fully informed on the nuances of the capabilities they are planning to procure.

Capability development consists generally of four stages: conceive, design, build and manage. While various mechanism are in place to monitor capability submissions as they pass through the first three stages, a detailed review of all major projects across the various stages does not occur in Canada.^[60] In the US, Congress conducts an annual review, with the Government Accountability Office reporting on the overall progress of major projects collectively and individually.^[61] The National Audit Office in the UK and the Australian National Audit Office carry out a similar function.^[62] These annual reviews allow not only the status of individual projects to be monitored, but also allow for the capture of best practices across all phases of defence procurement. For instance, given the time to procure defence equipment, Australian practice is to purchase commercial off-the-shelf items, unless a business case can be presented to prove that more detailed development is required for a particular item. Broad reviews of Canadian defence procurement occur in academia but not within government.^[63] Annual reviews conducted by appropriate authorities could confirm that stated capabilities continue to meet strategic needs and that select projects provide the necessary capabilities required for the effective defence of Canada.

2.4.3 Immediate Outcome 3: Selected options are integrated into existing warfighting systems

The evaluation used the following indicators to assess the outcome:

Oversight is exercised over system integration of capabilities.
Duplication does not exist between extant or future capabilities.

Indicator: Oversight is exercised over system integration of capabilities.

Finding 6: Sufficient mechanisms exist to integrate capabilities and guard against duplication.

System integration is accomplished by most L1s by a separate directorate established to oversee the process. For instance, the CA has the Director Land Force Development, which reports to Chief of Staff – Army Strategy, the head of capability development in the CA. The Director Structure Integration (DSI) within Director General Capability and Structure Integration (DGCSI) plays a similar role in CFD.

Indicator: Duplication does not exist between extant or future capabilities.

The avoidance of duplication in capability development is important to ensure the maximum value is achieved from defence spending. That is, money is not spent on unnecessary redundancies or that capabilities are replicated across stakeholders. Duplication can be a serious problem. For instance, in the US, the Air Force, the Navy and the Marines all have aircraft. It is therefore very difficult in the US to ensure a particular aircraft is providing a unique and necessary service. In Canada, defence spending is much more modest, as is the number of platforms procured, so we can ill-afford duplication of any kind. A formal mechanism exists in the last steps of the CBP process—the investment, divestment or sustainment decision—to ensure that duplication does not occur or persist. In general, CFD ensures that duplicate capabilities, whether structural or platform based, do not occur or continue to exist. The likelihood of duplication occurring or persisting in the CAF is therefore minimal.

2.4.4 Immediate Outcome 4: The Capability Development Program is properly governed

The evaluation used the following indicators to assess the outcome:

Regulations, plans and orders exist to direct capability development.
Decisions to further develop projects are taken by the appropriate authorities.
The capability and capacity to conduct force development / capability development exist at all necessary levels.

Indicator: Regulations, plans and orders exist to direct capability development.

Finding 7: The Capability Development Program is governed effectively across the DND/CAF.

Capability development is an extensively regulated system. Orders, direction, manuals and handbooks exist for practically every aspect of it. Most force generators have detailed orders, direction and guidance to describe how capability development will be conducted, how doctrine will be written and how LL will be captured. The system is also regulated extensively at the strategic level. Control mechanisms include central reference documents, like the FSE, and numerous planning directions and guides. These guides include the Force Capability Guide, the IP and process standard operating procedures, such as the CFD Capability Based Planning Handbook (2014) and the PAD 2015.

Indicator: Decisions to further develop projects are taken by the appropriate authorities.

Capability development decisions are approved by appropriate authorities. The Environments, other L1s (CFINTCOM, CANSOFCOM, CMP) and joint enablers (Space, C4ISR, Cyber) have strategic governance boards to conduct internal review and approval of capability development proposals including the Army Capability Development Board, the Maritime Capability Development Board and the Air Force Development Committee. Boards also exist at CFINTCOM, CANSOFCOM and within CFD itself.

Once a capability proposal is approved by the responsible L1, the project office must seek departmental approval via the Project Approval Process. The first major milestone of that process is the DCB, which initially approves a project’s Strategic Context Document. This document identifies a project’s strategic context and capability requirements, and it includes the high-level mandatory requirements and viable options to be assessed. A subsequent engagement with DCB provides a detailed analysis of options via the BCA and identifies the preferred option with which to move forward. Broadly speaking, the DCB engagements serve as a challenge function to ensure the alignment of future capabilities, the comprehensiveness of options analysis examination and the transparency and detail of project costing figures. DCB also helps with the prioritization of capabilities in the 5 to 20-year demand line by providing strategic advice on future capability investments and divestments, risk and impact assessments on future capability planning resulting from Level 0 Plan decisions and direction and review and approval of the Strategic Context Document and BCA of projects brought forward to it. It is important to note that recent changes to the project approval process restrict DCB access, and therefore departmental approval, for projects with an identified source of funds or unfunded projects approved by IRMC as key capability projects. The standard process to identify a source of funds for a project will be IRMC approval via the CIPPR process or through a major defence review as occurred in 2016/17. In each of these cases, VCDS and the Chief Financial Officer prioritize investment options (all unfunded projects) against the available funding. An additional challenge function is provided by the IRPDA for projects within its scope. The IRPDA conducts reviews of projects immediately following approval at each of the DCB engagements.

Indicator: The capability and capacity to conduct force development / capability development exists at all necessary levels.

Key Finding 2: The CFD’s role in the standardization of the Capability Development Program and in the development of joint enablers needs to be revalidated and documented.

There is a clear necessity for joint capability oversight and sponsorship to ensure joint enablers do not become separated and ignored. One of the purposes behind the establishment of CFD was to institutionalize a capability development champion for joint capabilities. The attention and priority accorded to this function has evolved over the years as CFD, like many organizations, has adjusted to new missions and tasks. CFD has more often given away joint responsibilities than assumed new ones. This extends from disbanding the joint concept cell to reassigning Director General Space to the RCAF, to transferring CFWC to CJOC but not assuming the responsibility for strategic, logistic capability development transferred out of CJOC, which SJS assumed. More elements of CFD are slated to be reassigned including Cyber, components of C4ISR, and chemical, biological, radiological and nuclear defence. While this dispersion is occurring for valid reasons, it will make the overall monitoring and control of the development of joint capabilities, a residual responsibility of CFD, much more challenging.^[64] There may be a requirement to establish a joint capabilities development board to ensure that joint capability gaps are appropriately identified, resourced and secured. This function will have to be balanced with CFD’s intermediary and standard control role within the broader parameters of departmental project management.

There are other internal issues within CFD, primarily in DGCSI, which may need attention for the organization to work more effectively, as follows:

Director Capability and Structure Analysis Support is required to respond to short-timeline information requests from senior officials but is greatly understaffed to do so. During the evaluation, only one analyst was available to respond to such requests. Most recently, DSI has had a six-person Lean Headquarters cell established in its lines in lieu of focussing on the future structure of the CAF.
DSI is responsible for strategic-level human resource planning. This organization has been manned in the past by individuals on short-term postings with no particular expertise in human resources matters such as organizational design. This compromises the strategic-level human resources planning.
Planning in DSI has been problematic in some instances. For example, in the early days of this evaluation, DSI was working on an RCAF issue (i.e., the Air Force Expeditionary Concept Project), which was clearly the purview and responsibility of the Air Staff to perform.

ADM(RS) Recommendation

4. It is recommended that the role, functions and organization of CFD be revalidated, including identifying the necessary governance structures, to ensure the development of joint enablers.

OPI: VCDS

2.4.5 Intermediate Outcome 1: proposed future family of CAF capabilities are identified and validated

The evaluation used the following indicators to assess the outcome:

Plans and programs exist to maintain or improve CAF capabilities.
Capability deficiencies are prioritized (internal to functional elements and pan-CAF).
The requirements of the environments, joint enablers, support and sustainment are appropriately balanced.

Finding 8: The DND/CAF has a well identified proposed family of future capabilities.

A wide variety of plans and programs exist to maintain or improve CAF capabilities. These include baseline documents, such as the FCG and iterations of the IP, as well as the broad programs run by the Environments and other stakeholders. At the strategic level, capability deficiencies are prioritized within the FCG and the IP, and crucial advice is provided to senior decision makers. It is also within these documents that the requirements are balanced to ensure that the CAF remains operationally effective.

Back to Table of Contents

2.5 Performance—Achievement of Expected Outcomes (Efficiency and Economy)

The following section examines the extent to which the Capability Development Program provides value for money by being both efficient and economical in achieving the outcomes expected of it. The TBS Evaluation Policy (2009) defines efficiency as “maximizing the outputs produced with a fixed level of inputs.” That is, given a fixed set of resources, they are used efficiently (i.e., there is little waste or rework). Inefficiencies are usually evident in the processes that convert inputs to outputs. Economy is defined as “minimizing the use of resources to achieve expected outcomes.” The overall cost of the program to deliver the outputs is important. Economy is also related to what it costs the program to secure the resources necessary to carry out its tasks. In certain instances, more staff effort may be consumed in obtaining the resources to perform tasks than focussing on the core purpose of the program. To demonstrate the efficiency and economy of the Capability Development Program, the following evaluation questions were addressed:

Is the DND/CAF delivering capability development in an efficient manner?
Is the progress made toward expected outcomes adequate for the resources expended?
Are there alternative, and perhaps more economical, ways of delivering capability development?

The specific indicators examined were the following:

The IP remains affordable and changes to it are planned.
Program costs are reasonable.
Planning and scheduling occurs to maximize capability acquisition.
Staff selection and training are adequate.

Indicator: The IP remains affordable and changes to it are planned.

Finding 9: The DND/CAF’s ability to monitor overall program affordability and set project priorities through the IP promotes greater efficiency.

The IP remains affordable and changes to it are well planned. The first IP was based on the Canada First Defence Strategy (2008) and included those elements of CAF modernization and transformation that are to be implemented during the planning period. The purpose of the IP was to ensure long-term affordability and coherence of the overall Defence Services Programme, with the IP detailing annual spending levels and indicating which projects would receive funding. This ensures that project spending is not over-programmed and that the capability demand is not so excessive that it is completely unachievable. Unrealistic programming forced dramatic cuts in UK defence capabilities in the late 2010s, an outcome that would be best avoided in Canada. Projects are entered into the IP in a few ways, depending on the timing/circumstance. The IP is reviewed/updated every three years. Discussions that lead to the IP will lead to a prioritization of projects to be included. For IP 17, that discussion function was served by the Defence Policy Review, which, in conjunction with Budget 17, served to reset the projects identified for funding in the IP. The Investment Plan Change Proposal is a formal and detailed process that exists to make amendments to the IP.

The IP, as a longer-term process, is complemented by the shorter-term CIPPR. The CIPPR is a process by which projects are analyzed and validated in order to produce a consolidated, balanced and costed portfolio consisting of critical, viable and affordable capabilities representing best value for money. In between the IP review years, the Deputy Minister can identify approved departmental funding (in-year surpluses) for projects contained in the CIPPR via the IRMC. While CIPPR exploits funding space with in-year budgets, it also provides a periodic review of projects to prioritize those that will have a source of funds in the Departmental Investment Plan. Generally, these projects have been identified as pressures until the next iteration of the IP.

The IP and the CIPPR improve efficiency in capability development. The CIPPR can be regarded as a tool in support of the IP and defence procurement that ensures more effective cash flow for projects. Other tools also assist in the analysis and prioritization of the CIPPR. This allows staffs at all levels to focus only on those key departmental projects that have been prioritized and approved and that are therefore being actioned.

Indicator: Program costs are reasonable.

To assess the economy of the Capability Development Program, the evaluation analyzed trends in expenditures, staffing levels and program administration from FY 2010/11 to FY 2015/16. This assessment was based on information provided by data from the PAA and the Defence Resource Management Information System.

Finding 10: The costs required to conduct the Capability Development Program are reasonable.

The amount of resources dedicated to capability development has a significant impact on the overall defence budget. Capability development addresses not only the equipment with its attendant life-cycle costs of the future force but also how organizations will be designed and how personnel structures will be constructed. It therefore has an influence not only on the capital program, but also on the Operations and Maintenance program and on personnel costs. The costs of the Capability Development Program can be related to overall government expenditure, as well as to overall departmental expenditure. In FY 2014/15, the DND budget was $19.8 billion.^[65] DND expended $160.6 million on capability development,^[66] equating to 0.8 percent of the DND budget.

The year-to-year comparison of costs was limited by the attributions in the PAA. As Table 3 denotes, from FY 2010/11 to FY 2013/14, the attributions were consistent from year to year. The attribution percentages changed in FY 2014/15, and comparison to prior years was no longer possible. During the period FY 2007/08 to FY 2013/14, the budget for capability development was, on average, 35 percent of the total budget of approximately $475 million for PAA 5.0, with the remaining 65 percent being attributed to S&T (Defence Research and Development Canada). In FY 2014/15, the budget for capability development was approximately equal to that of S&T (about $230 million). In FY 2015/16, the budget for capability development was approximately $194.2 million.

Table 3. Capability Development Program Costs for 2011 to 2016. This table lists the PAA program costs in the four capability development PAA elements from 2011 to 2016.
PAA Elements	Costs for Capability Development ($ millions)^[67]
PAA Elements	2011	2012	2013	2014	2015	2016
5.1.1 – Capability Design and Management	46.0	50.8	58.4	47.1	72.4	88.6
5.1.2 – Concept, Doctrine Development and Warfare Experimentation	70.5	78.2	74.2	81.5	77.8	68.7
5.2.1 – Strategic Capability Planning Support	52.7	39.9	21.3	30.1	75.1	33.0
5.2.2 – Strategic Force Posture Planning Support	2.7	2.9	3.2	1.9	2.1	3.8
Total Capability Development	171.9	171.8	157.1	160.6	227.4	194.1
5.1.3 – Science and Systems Development and Integration	310.4	313.1	294.9	307.1	235.0	230.6
Total for PAA 5.0	482.3	484.9	452	467.7	462.4	424.7
Capability Development as a Percentage of PAA 5.0	35.6%	35.4%	34.8%	34.3%	49.2%	45.7%

Table 3 Details - Capability Development Program Costs 2011 – 2016

As a percentage of annual amounts spent on capital acquisition, the costs of capability development averaged 5.8 percent over the period of FY 2010/11 to FY 2013/14, and 6.6 percent over the period of FY 2014/15 to FY 2015/16, as seen in Table 4.

Table 4. Capability Development Cost vs Capital Acquisition Budget. This table lists the capability development costs as a percentage of the acquisition costs.
Acquisition	Costs ($ millions)
Acquisition	2010/11	2011/12	2012/13	2013/14	2014/15	2015/16
Capital Acquisition	3,734.0	3,207.9	3,025.7	2,831.9	3,250.4	3,075.5
Capability Development	171.9	171.7	157.2	160.6	227.4	194.2
Capability Development as a Percentage of Capital Acquisition	4.6%	5.4%	5.2%	5.7%	7.0%	6.3%

Table 4 Details - Capability Development Cost vs Capital Acquisition Budget

Table 5. FTEs by PAA Element for FY 2015/16. This table lists the personnel costs for the capability development PAAs.
PAA Elements	Civilian FTEs^[68]	Military FTEs	Total FTEs
5.1.1 – Capability Design and Management	14	250	264
5.1.2 – Concept, Doctrine Development and Warfare Experimentation	52	366	418
5.2.1 – Strategic Capability Planning Support	69	61	130
5.2.2 – Strategic Force Posture Planning Support	n/a	19	19
Total	135	696	831

Table 5 Details - FTEs by PAA Element for 2015/2016

The bulk of the costs within PAA element 5.0 are personnel costs as the bulk of capability development is intellectual work. Personnel numbers were available for FY 2015/16 only. In that year, 2,133 FTEs were attributed to PAA 5.0, 831 FTEs were used to deliver capability development (PAA items 5.1.1, 5.1.2, 5.2.1, and 5.2.2); that is, 135 civilian and 696 military. These 831 FTEs were attributed as per Table 5.

Due to the attribution rules of the PAA, it was not possible to determine exactly how the environments allocated their capability development staffs to particular projects. As a proxy, major capital projects demonstrate the emphasis of work in the Department. Sixteen major capital project in FY 2015/16 accounted for over 55 percent of the $4 billion capital budget, with a great share going to the Medium-to-Heavy Lift Helicopter Project (3 percent), the Force Mobility Enhancement Project (3 percent), the Maritime Helicopter Project (6 percent), the Light Armoured Vehicle III Upgrade (8 percent), the Halifax Class Modernization (9 percent) and the Arctic Offshore Patrol Ship (9 percent).

Overall the defence capability development and research activities represents approximately 2 percent of departmental spending by program dollars. Total FTEs (2,065) in this area (including S&T, which was reviewed separately) represent 2.3 percent of the Department’s human resources. If S&T numbers are removed, then the front-end of the Department’s planning for future capabilities, with an annual total in the billions, is performed by about 0.9 percent of the workforce. While capability development staffs exist at all levels, capability development remains a command responsibility. The resources dedicated to it are related to the importance attached to it and to the other priorities of individual commanders. It was not clear during this evaluation whether the size of staffs at the joint level was sufficient.Indicator: Planning and scheduling occurs to maximize capability acquisition.

Indicator: Planning and scheduling occurs to maximize capability acquisition.

Finding 11: Tighter control of changes and better fleet management techniques could make the Capability Development Program more efficient and economical.

Change occurs frequently in project management. Major change comes with specific economic costs. Change prompts re-work, which can often be regarded as extra or wasted effort to achieve the same outcome. More importantly, since GC projects have become accounted for in budget-year dollars, any significant delay to a project caused by change greatly reduces the amount of capabilities that can be purchased.

Some change is inevitable in any project after the conceive and design phases for a variety of reasons. For instance, the option presented to project staff for the build phase may not have been thoroughly developed, designed or costed.^[69] Also, there are always trade-offs between costs and capabilities as a project is more fully developed. But, delays caused by changes must be carefully managed due to their economic consequences. First, as noted previously, extended delays greatly reduce the purchasing power of the funds allocated (budget-year dollars).^[70] Second, technology in many areas of the defence enterprise evolves rapidly. Excessive delay can cause a platform to be near obsolete at delivery or require extensive rework and change orders prior to delivery. Additionally, waiting for technology to evolve, such as was the case on the JUSTAS project, can delay delivery.^[71] Third, relatively minor but seemingly interminable projects prevent staff from being reassigned to other priorities. Major procurement projects typically have life spans exceeding 10 years, but some projects (such as Fixed-Wing Search and Rescue or the Twin Otter Replacement) have life spans that nearly exceed major procurement initiatives.

When change occurs in a project during or after the design phase requiring further attention of the capability development staff, this can be considered a form of rework. Rework can occur during the design phase when projects are brought forward for review and are returned for further work, a normal part of project management. This can be magnified when additional steps are imposed into project approval processes. In the build phase, where there is a fuller consideration of all force integration costs (e.g., personnel, infrastructure, etc.), a re-examination of the option selected may be required. Previously, some DND/CAF projects underestimated life cycle costs, at times infrastructure costs were not sufficiently factored, and major changes were made to the statement of requirements late in the process, all of which caused problems and delays. Some of these causes of change have been mitigated through the wider use of PRICIE analysis and the challenge function provided by the CFD, DCB and IRPDA.

Capability development incurs an economic cost when an incomplete set of equipment is procured. Partial procurement may cause additional work when limited but necessary operational equipment must be transferred from unit to unit or platform to platform. But partial procurement may also mean that the capability gap has not been effectively resolved. In this instance, capability development staff must rework the issue to ensure that the extant, plus the partial capability obtained, can either perform the needed capability effectively or that an alternate solution may be required. Occasionally, capability development staffs and warfare centres are required to devote considerable effort to improvising system fixes that a complete procurement should have satisfied.

Fleet management planning and fleet consolidation opportunities play an important role in making capability development more efficient and economical. Almost immediately upon introduction into service, the plan for the sustainment of a capability must be put in motion as every piece of equipment has a limited life span (i.e., it wears out through normal usage). Sustaining a capability may require either a life extension project, estimated life expectancy extensions, which are common for airframes, or all-out replacement. The funds available for procurement and for maintenance are limited. Fleet management plans are required to allow decision makers to choose between replacement or continued maintenance on each major fleet. The RCAF has long used an annual flying hours model to this end and the CA has recently developed more sophisticated fleet management plans. Fleet consolidations can lead to savings in maintenance and training costs. For example, the UK, the US and Australian militaries have consolidated their helicopter fleets over the last decades for this purpose.^[72] Consolidation of fleets has previously occurred in Canada with the purchase of the CF-18s and the Griffon helicopters.^[73] The use of a common baseline platform can also be seen in the CA’s development of the family of combat vehicles fleet and in the RCN’s plan to use a common hull for the newest navy frigates.^[74] Consolidation yields sufficient benefits that it should be considered when replacing any capability.

Key Finding 3: Greater use of M&S could make the Capability Development Program more efficient and economical.

M&S contributes to both the efficiency and economy of capability development if properly used during the conceive and design phases. M&S can be used to test organizational design, the viability of employment concepts, equipment placement and structures, as well as other elements. Good M&S can reduce costs of final field trials, initial equipment mock-ups and aid in system integration both from an operational and a maintenance perspective. A five-year M&S campaign plan has recently been issued that may improve its overall use. Interviewees indicated that M&S is presently an under-utilized resource and some extant capabilities, such as those at the CFMWC, are at risk due to minor funding issues. More M&S capability is required within CFD to assist in the development and testing of concepts through a formal experimentation process and to aid in broader systems analysis to validate capability trade-offs. This capability may be equally important in selecting the elements of the US Third Offset Strategy in which to participate. Greater efficiency and economies could be obtained throughout the CAF in capability development by devoting greater efforts to the development and use of M&S.

ADM(RS) Recommendation

5. It is recommended that CFD develop a comprehensive experimentation and M&S campaign plan for its internal program, and that all L1s with capability development responsibilities exploit to the maximum extent possible experimentation and M&S in their conceive and design phases.

OPI: VCDS

Indicator: Staff selection and training are adequate.

Key Finding 4: Capability development staff should be more carefully selected, trained and supported.

Given the complexity, cost and risks associated with major projects, capability development staff must have the size and expertise necessary to deliver on their projects. The better trained and selected capability development staffs are, the more efficient and economical the capability development process will be. Staffs need training in the profession of arms, technical issues and project management disciplines. For better value for money, once capability development staffs are groomed and trained for their positions, their time in those appointments should be extended beyond the common 18 to 24-month limits.

Modern warfare involves integrating complex systems into a joint collective to achieve campaign objectives. Normal military career progression provides a good measure of the knowledge of the profession of arms to aid in the understanding of the capabilities that need to be integrated. However, the technical aspects of the capabilities must also be understood, and the CAF requires subject matter experts in areas ranging from airworthy nuts and bolts to indirect fire and cyber warfare. Only the CA has a specific course to train technical staff officers. In the RCN and the RCAF, officers from technical branches carry out aspects of capability development as part of their career progression but not necessarily with platform-specific training. Further, many capability development positions are filled by generalists who lack the detailed knowledge of concept development, doctrine and LL until they receive further training such as that conducted by CFAWC in its doctrine and LL cells. Training in the project management processes commonly used in the DND/CAF is also a key requirement. Basic project management training includes initiatives like the TBS project risk profile procedures and BCA process, and the High-Level Mandatory Requirements system.

For capability development subject matter experts to be most efficient, they must liaise closely with key allies, academics and industry representative to keep abreast of initiatives and developments. This activity represents good value added for Canada as the DND/CAF is able to contribute to and make use of the extensive work of our allies.

Training should be improved for capability development staff in functional areas (profession of arms, technical, human resource management, etc.), as well as in project management techniques. Sufficient specialists, who have acquired the necessary experience, training and education, must be developed to enhance the efficiency and effectiveness of the capability development process.

ADM(RS) Recommendation

6. It is recommended that VCDS conduct a training needs analysis for capability development staff, which should then influence assignment, training and core activities.

OPI: VCDS

OCIs: All L1s with capability development staff

Back to Table of Contents

Annex A—Management Action Plan

ADM(RS) Recommendation
1. It is recommended that the conceptual components of joint warfighting, in particular, concepts, doctrine and joint LL, be properly organized, institutionalized and exploited to improve pan-CAF capability development.

Management Action

CFD will develop and examine options (with L1 support) to organize concepts, doctrine and LL such that they are available pan-CAF to facilitate the development of concepts at all levels across all L1s. A new Directorate of Concepts and Experimentation has been established within CFD and will be the lead for this action item. They are working on the way ahead and will reach initial operational capability with a draft plan in 2018. The aim is to have the full plan implemented by 2020.

Supporting organizations will be: RCN, CA, RCAF, CANSOFCOM and CJOC (CFWC)

The Directorate of Concepts and Experimentation will have ahead plan draft at initial operational capability by December 2018.

OPI: VCDS
Target Date: December 2018

ADM(RS) Recommendation
2. It is recommended that the joint LL database be updated and improved to increase its functionality.

Management Action

Under the direction of Comd CJOC, the Departmental Lessons Learned Program provides leadership and management for the harmonization and promotion of LL efforts across the Department, including the coordination of issues that transcend DND’s internal and external organizational boundaries. As part of a management action plan for the Evaluation of the DND Contributions to Humanitarian Operations, Domestic Relief Operations and Non-combatant Evacuation Operations (October 2013), CJOC reviewed and updated (where appropriate) the departmental direction for LL. Departmental activities for LL are governed by DAOD 8010-0 and guided by the LL doctrine within CFJP A2.

A key enabler for the Departmental Lessons Learned Program is the KMS, which has been in service since 2004 as the pan-departmental LL software and has been periodically updated with Vote 1 funds. The KMS development has not been entirely successful in keeping pace with the evolving needs of the Departmental Lessons Learned Program and has been hampered by an inconsistent funding envelope. On June 29, 2017, the CJOC Resource Management Board allocated sufficient FY 2017/18 funds for the initial phase of the KMS update. On July 18, 2017, the Resource Management Board further confirmed that, through the Business Planning Process, funding would be earmarked for the following two years (FY 2018/19 and FY 2019/20) for the continued execution of the KMS update plan. Assured funding for the out-years is central to successful update completion. ADM(IM), through an existing In-Service Software Transfer Directive, has been engaged to facilitate the update. A 138-item LL user-community-approved Statement of Requirements has been produced to guide the updating work.

Milestone 1: March 31, 2018. Contractor resources secured, stakeholders engaged and roles established, project charter and scope statement produced, Statement of Requirements validated, existing KMS business processes examined and new business process model produced and validated. Work plan and user testing schedule for software update completed, allocation of FY 2018/19 funds is obtained prior to March 31, 2018 to allow contracted work to continue uninterrupted.

Milestone 2: March 31, 2019. Represents 44 percent of the software work. The following KMS software modules will be updated to support new business processes: “Managing Support Material,” “Managing Unsolicited Observations,” “Managing Activities,” “Managing Issues,” “Managing Lessons,” and “Managing Questionnaires.” Allocation of FY 2019/20 funds will be obtained prior to March 31, 2019 to allow contracted work to continue uninterrupted.

Milestone 3: March 31, 2020. Represents 56 percent of software work. The following KMS software modules will be updated to support new business processes: “Managing Observations and Comments,” “Managing Configuration,” “Managing Task List,” “Managing Analysis Projects,” “Managing Status Reports,” “Managing Organizations,” “Cross-Cutting,” and “Reports.”

OPI: CJOC/CFWC/LL Branch Head
OCI: VCDS, ADM(IM)
Target Date: Milestone 3 achieved by April 2020

ADM(RS) Recommendation
3. It is recommended that VCDS and CJOC review the purpose, role, functions and responsibilities of the CFWC.

Management Action

Review of CFWC has been completed, CAF Warfare Centre Study Report (1920-1(CFD/DSI)) published May 24, 2016 and actions implemented.

The study indicated that a strategic-level concept from which the service elements can obtain guidance and direction was lacking. The community of practice provides limited institutionalized interconnectivity, interoperability and capacity, and it has been the primary vehicle for integrating the Strategic and Operational Level Joint Doctrine and the Joint Concept to Capability Development. The study proposed the creation of a joint concept/capability development section within CFD/DGCSI, which would assume the legacy VCDS functions resident in CFWC. Reassigning duties such as chairmanship of the community of practice and full management of the Strategic Experimentation Account to CFD would provide the authority, means and mandate to develop and control the conceive and design pillars for joint matters. With CFD acting as the CAF’s joint steward at the strategic level, CJOC (and CFWC) would be able to focus purely on initiatives related to joint training and force employment. Assignment of these L0 type functions to CFD will also provide a mechanism to resolve deadlocks in the event that L1 collaboration fails and will ensure that alignment with L0 directives is promoted and maintained.

As a result, the Directorate of Concepts and Experimentation was established within CFD in 2017. It will centralize key aspects of concepts and experimentation as recommended in the report.

No additional action is intended.

OPI: VCDS
OCI: CJOC
Target Date: Complete

ADM(RS) Recommendation
4. It is recommended that the role, functions and organization of the CFD be revalidated, including identifying the necessary governance structures to ensure the development of joint enablers.

Management Action

The role, functions and organization of CFD have been reviewed in accordance with direction from the CDS. As a result Space, Cyber and C4ISR force development functions have been devolved to the services. Functional roles will continue to be assessed periodically to ensure the development of joint enablers. If assessed as appropriate, CFD will conduct the force development function to enable emerging joint capabilities until they are mature enough to be passed to the appropriate service.

OPI: VCDS
Target Date: Complete

ADM(RS) Recommendation
5. It is recommended that CFD develop a comprehensive experimentation and M&S campaign plan for its internal program, and that all L1s with capability development responsibilities exploit to the maximum extent possible experimentation and M&S in their conceive and design phases.

Management Action

CFD has established the Directorate of Concepts and Experimentation. The purpose of this new directorate will be to promote and facilitate development of joint concepts that will be tested in a strategic experimentation environment, the results of which will form the basis for and inform force development. This directorate will intentionally achieve initial operational capability in 2018 and full operational capability in 2019.

OPI: VCDS
Target Date: Initial Operational Capability of Directorate of Concepts and Experimentation by December 2018

ADM(RS) Recommendation
6. It is recommended that VCDS conduct a training needs analysis for capability development staff, which should then influence selection, assignment, training and core activities.

Management Action

DGCSI will codify a professional development regime for staff officers posted in, which may include the establishment of a special personnel qualification requirement that will assist in identifying the pool of qualified staff.

CFD will work with CMP to establish the criteria for specific job positions and to establish a vetting process to ensure staff with the right skill and experience base are posted in. CFD will also explore the possibility of sponsoring a post-graduate program to further professionalize the work force.

OPI: VCDS
OCIs: All L1s with capability development staff
Target Date: Professional development regime will be completed and documented by December 201

Back to Table of Contents

Annex B—Evaluation Methodology and Limitations

1.0 Methodology

1.1 Overview of Data Collection Methods

The evaluation of the Capability Development Program included the use of multiple lines of evidence and complementary research methods to strengthen the rigour and reliability of the assessment. The methodology used a consistent approach when collecting and analyzing data to help ensure the reliability of the evaluation findings and recommendations. Quantitative and qualitative data collection methods were used and included reviews of literature and program documents, access to financial data, requests for key informant interviews and site visits. Following data collection and analysis, preliminary evaluation findings were presented to the key stakeholders via the Capability Development Forum. Discussions from these presentations helped to further refine and clarify the findings and recommendations that are presented in this report.

1.2 Details on Data Collection Methods

1.2.1 Program Document and Literature Review

A review of program documents was conducted in the initial phase of the evaluation to establish an understanding of the background and context of the Capability Development Program. These documents included the following:

federal/departmental accountability documents;
strategic and operational program documents (i.e., orders, directives and briefing notes);
website content;
guidance documents, process and procedure manuals;
previous internal and external assessment reports; and
program products/outputs.

The document review was integral to the assessment of relevance of the Program, as well as to support performance findings from other lines of evidence.

An extensive review of relevant academic literature and publications was conducted. This was necessary as considerable data/information on capability development within militaries occurs in the open academic literature. This was particularly useful when examining our program’s comparability with those of key allies. In addition, this methodology was important for examining large (non-Canadian) procurement programs and for the evaluation of the particular use of capabilities during specific campaigns.

1.2.2 Access to Financial Data

Access to financial data was a useful method in scoping the program. The following sources were used in the analysis:

PAA expenditures
Departmental Performance Reports / Reports on Plans and Priorities
GC documents and academic studies

Having access to the financial repositories allowed for an in-depth analysis with regards to financial attributions.

1.2.3 Key Informant Interviews

Consultations were conducted at different phases of the evaluation primarily through interviews with key stakeholders. Capability development is carried out by multiple L1s, so activities and practices are widely dispersed and varied. Interviews were conducted across stakeholders including VCDS, SJS, CJOC, CA, RCN, RCAF, CANSOFCOM, CMP and CFINTCOM. Interviews were conducted during the scoping and then the data gathering phases of the evaluation. Information gathered by ADM(RS) from the interviews was cross-reference against documentation to assess performance.

1.2.4 Site Visits

A site visit was conducted to each of the warfare centres.

2.0 Limitations

Table B-1 shows the evaluation limitations and the corresponding mitigation strategies.

Table B-1. Evaluation Limitations and Mitigation Strategies. This table lists the limitations of the evaluation and the corresponding mitigation strategies.
Limitation	Mitigation Strategy
Inaccurate PAA attributions. The inability to accurately identify resources expended for capability development inhibits the assessment of economy and efficiency.	Inaccurate PAA attributions. The inability to accurately identify resources expended for capability development inhibits the assessment of economy and efficiency.
Research bias. Individual references can give a subjective impression of stakeholders and, as such, can lead to narrow, very wide or potentially biased views.	Insights derived from individual references required corroboration from at least one other source.

Table B-1 Details - Evaluation Limitations and Mitigation Strategies

Back to Table of Contents

Annex C—Logic Model

Figure C-1 Details - Logic Model for the Capability Development Program

Figure C-1 Summary: This flow chart shows the relationship between the Program’s main activities, outputs and expected outcomes.

Starting Point A

Inputs: knowledge
Activities: conceiving
Outputs: context; concepts
Immediate Outcome: Capability gaps identified
Intermediate Outcome: Proposed family of new or sustaining CAF capabilities
Ultimate Outcome: Selected options to sustain or improve CAF capabilities are approved for implementation

Starting Point B

Inputs: SMEs
Activities: conceiving
Outputs: context; concepts
Immediate Outcome: Capability gaps identified
Intermediate Outcome: Proposed family of new or sustaining CAF capabilities
Ultimate Outcome: Selected options to sustain or improve CAF capabilities are approved for implementation

Starting Point C

Inputs: funding
Activities: designing
Outputs: options
Immediate Outcome: Options analyzed
Intermediate Outcome: Proposed family of new or sustaining CAF capabilities
Ultimate Outcome: Selected options to sustain or improve CAF capabilities are approved for implementation

Starting Point D

Inputs: information
Activities: designing
Outputs: options
Immediate Outcome: Options analyzed
Intermediate Outcome: Proposed family of new or sustaining CAF capabilities
Ultimate Outcome: Selected options to sustain or improve CAF capabilities are approved for implementation

Starting Point E

Inputs: other resources
Activities: building
Outputs: Integration of selected option
Immediate Outcome: System integrated\
Intermediate Outcome: Proposed family of new or sustaining CAF capabilities
Ultimate Outcome: Selected options to sustain or improve CAF capabilities are approved for implementation

Starting Point F

Inputs: models
Activities: governing
Outputs: authorization
Immediate Outcome: Program coordinated
Intermediate Outcome: Proposed family of new or sustaining CAF capabilities
Ultimate Outcome: Selected options to sustain or improve CAF capabilities are approved for implementation

Back to Table of Contents

Annex D—Evaluation Matrix

Table D-1. Evaluation Matrix—Relevance. This table indicates the data collection methods used to assess the evaluation issues/questions for determining the Capability Development Program’s relevance.
Evaluation Issues/Questions	Indicators	Program Data	Document Review	Key Informant Interviews
1.1 Is there a need for the Capability Development Program?	1.1.1 The Capability Development Program addresses the future needs of the CAF.	Yes	Yes	No
1.2 How does the Capability Development Program align with current federal roles and responsibilities?	1.2.1 Alignment to GC roles and responsibilities	Yes	Yes	No
1.3 To what extent does the Capability Development Program align with current government policies and priorities?	1.3.1 Alignment with GC priorities 1.3.2 Alignment with DND/CAF priorities	Yes	Yes	No

Table D-1 Details - Evaluation Matrix—Relevance

Table D-2. Evaluation Matrix—Performance (Effectiveness). This table indicates the data collection methods used to assess the evaluation issues/questions for determining the Capability Development Program’s performance in terms of achievement of outcomes (effectiveness).
Evaluation Issues/ Questions	Indicators	Program Data	Document Review	Literature Review	Key Informant Interviews
2.1 Capability gaps are identified.	2.1.1 The CAF maintains situational awareness on the changes and evolution of the security environment, which may create capability gaps.	Yes	Yes	Yes	Yes
	2.1.2 Context documents and processes exist that support the identification of capability gaps.	Yes	Yes	No	Yes
	2.1.3 Concepts, doctrine, LL and warfare centres contribute to the identification of capability gaps.	Yes	Yes	Yes	Yes
2.2 Options are selected to resolve capability gaps.	2.2.1 Tools are available, including BCA, to analyze the capabilities offered by various options.	Yes	Yes	Yes	Yes
	2.2.2 Research and experimentation contribute to options analysis.	Yes	Yes	Yes	Yes
	2.2.3 Proposals to address capability gaps are considered credible.	Yes	Yes	No	Yes
2.3 Selected options are integrated into existing warfighting systems.	2.3.1 Oversight is exercised over system integration of capabilities.	Yes	Yes	No	Yes
	2.3.2 Duplication does not exist between extant or future capabilities.	Yes	Yes	No	Yes
2.4 The Capability Development Program is properly governed.	2.4.1 Regulations, plans and orders exist to direct capability development.	Yes	Yes	No	Yes
	2.4.2 Decisions to further develop projects are taken by the appropriate authorities.	Yes	Yes	No	Yes
	2.4.3 The capability and capacity to conduct force development / capability development exist at all necessary levels.	Yes	Yes	No	Yes
2.5 Proposed future family of CAF capabilities are identified and validated.	2.5.1 Plans and programs exist to maintain or improve CAF capabilities.	Yes	Yes	No	Yes
	2.5.2 Capability deficiencies are prioritized (internal to functional elements and pan-CAF).	Yes	Yes	No	Yes
	2.5.3 The requirements of the environments, joint enablers, support and sustainment are appropriately balanced.	Yes	Yes	No	Yes

Table D-2 Details - Evaluation Matrix—Performance (Effectiveness)

Table D-2 Summary: This table indicates the data collection methods used to assess the evaluation issues/questions for determining the Information Capability Development Program’s performance in terms of achievement of outcomes (effectiveness). It has six columns. The left-hand column lists the evaluation issues/questions; the second column lists the performance indicators applicable to the issues/questions; and the last four columns indicate the data collection methods used in the evaluation. For each evaluation issue/question, read across the row to know the corresponding performance indicators, as well as the data collection methods used (program data, document review, literature review or key informant interviews), which are indicated with the word “yes” in the appropriate cell.

Table D-3. Evaluation Matrix—Performance (Efficiency and Economy). This table indicates the data collection methods used to assess the evaluation issues/questions for determining the Capability Development Program’s performance in terms of efficiency and economy.
Evaluation Issues/ Questions	Indicators	Program Administrative and Finance Data	Document Review / Benchmarking	Key Informant Interviews
3.1 Is the DND/CAF delivering capability development in an efficient manner?	3.1.1 The IP remains affordable and changes to it are planned.	Yes	Yes	Yes
	3.1.2 Program costs are reasonable.	Yes	Yes	No
3.2 Is the progress made toward expected outcomes adequate for the resources expended?	3.2.1 Planning and scheduling occurs to maximize capability acquisition.	Yes	No	No
3.3 Are there alternative, perhaps more economical, ways of delivering capability development?	3.3.1 Staff selection and training are adequate.	Yes	Yes	Yes

Table D-3 Details - Evaluation Matrix—Performance (Efficiency and Economy)

Table D-3 Summary: This table indicates the data collection methods used to assess the evaluation issues/questions for determining the Capability Development Programs performance in terms of efficiency and economy. It has five columns. The left-hand column lists the evaluation issues/questions; the second column lists the performance indicators applicable to the issues/questions; and the last three columns indicate the data collection methods used in the evaluation. For each evaluation issue/question, read across the row to know the corresponding performance indicators, as well as the data collection methods used (program administrative and finance data, document review / benchmarking, or key informant interviews), which are indicated with the word “yes” in the appropriate cell.

Back to Table of Contents

Footnotes

Footnote 1 As defined in the Defence Terminology Bank.

Footnote 2 Ibid.

Footnote 3 CFD. Capability Based Planning Handbook, 2014.

Footnote 4 Each letter stands for a word or phrase as follows: P = Personnel; R = research and development / operations research; I = Infrastructure and organization; C= Concepts, doctrine and collective training; I = Infrastructure; E = Equipment and material.

Footnote 5 From Director General Capability and Structure Integration (DGCSI) Overview, CFD Orientation Day, September 12, 2016.

Footnote 6 CFD Mandate.

Footnote 7 DND 2017-18 Departmental Plan.

Footnote 8 Totals may not match due to rounding.

Footnote 9 TBS Directive on the Evaluation Function, April 1, 2009, http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15681&section=text. Last consulted on July 4, 2014.

Footnote 10 Minister of Defence Mandate Letter http://pm.gc.ca/eng/minister-national-defence-mandate-letter. Last consulted on February 1, 2017.

Footnote 11 Future Security Environment 2013-2040.

Footnote 12 Australian Future Land Warfare Report, 2014; and the US Quadrennial Defense Review, 2014.

Footnote 13 Future Security Environment 2012-2040.

Footnote 14 Paul K. Davis. Analytic Architecture for Capabilities-Based Planning Mission-System Analysis and Transformation, RAND Corporation, 2002.

Footnote 15 See Stephan De Spiegeleire, “Ten Trends in Capability Planning for Defence and Security,” The RUSI Journal 156.5, October/November 2011.

Footnote 16 Auditor General’s Report, April 1998.

Footnote 17 D. Blakenet et al. “Operational Research Tools Supporting the Force Development Process for the Canadian Forces,” Information & Security 23.1, 2009; and M. Lizotte et al. Capability Engineering Process Version 2008, Defence Research and Development Canada, 2009.

Footnote 18 Robert Hartfield. “Planning Without Guidance: Canadian Defence Policy and Planning, 1993-2004,” Canadian Public Administration 53.3, September 2010.

Footnote 19 Lloyd Campbell. “Replacing the Canadian Manned Fighter Capability,” Canadian Military Journal 11.3, Summer 2011.

Footnote 20 Tobias Ellwood, “Leveraging UK Carrier Capability: A Study into the Preparation for and Use of the Queen Elizabeth-Class Carriers,” RUSI, 2013. For UAS see United States Air Force. RPA Vector: Vision and Enabling Concepts 2013-2038, February 2014.

Footnote 21 The benefit is that it enables DND to match the force structure to requirements other than capability.

Footnote 22 Paul Cornish and Andrew M. Dorman. Complex security and strategic latency: the UK Strategic Defence and Security Review 2015,” International Affairs 91.2, March 2015.

Footnote 23 The UK found that, without a proper strategic framework, spending on military operations in Iraq was partially driven by low-level decisions on equipping the Armed Forces. See UK Government. The Report of the Iraq Inquiry, Section 13.2. https://www.gov.uk/government/publications/the-report-of-the-iraq-inquiry. Last consulted on February 1, 2017.

Footnote 24 CRS. Audit of Unforecasted Operational Requirement Process, September 2012. (CRS was the former designation of ADM(RS), which changed in May 2015).

Footnote 25 Christian Tripodi. “Grand Strategy and the Graveyard of Assumptions: Britain and Afghanistan, 1839-1919,” Journal of Strategic Studies 33.5. Also, David S. McDonough. “The Paradox of Afghanistan,” International Journal, Summer 2007.

Footnote 26 Amitai Etzioni.“COIN: A Study of Strategic Illusion,” Small Wars & Insurgencies 26.3, 2015.

Footnote 27 The CA has successfully pursued this practice of communicating its developing concepts. Michael Rostek et al., Toward Army 2040: Exploring Key Dimensions of the Global Environment, 2011.

Footnote 28 Al English. Command & Control of Aerospace Forces: Conceptual Foundations, 2008.

Footnote 29 John Gellner, “Strategic Analysis in Canada,” International Journal 33.3, summer 1978.

Footnote 30 La Franchi, “Mapping a Future,” Flight International, March 2005.

Footnote 31 See CFJP 01 Canadian Military Doctrine, 2011.

Footnote 32 Harald Hoiback, Understanding Military Doctrine: A Multidisciplinary Approach, 2013.

Footnote 33 I. B. Holley, Jr., Technology and Military Doctrine: Essays on a Changing Relationship, 2004.

Footnote 34 Naval Order 3771-13.

Footnote 35 See http://nso.nato.int/nso/. Last consulted on February 1, 2017.

Footnote 36 The transition of operational-level doctrine is described at http://www.rcaf-arc.forces.gc.ca/en/cf-aerospace-warfare-centre/aerospace-doctrine.page. Last consulted on February 1, 2017.

Footnote 37 CFJP A1 Doctrine Development Manual.

Footnote 38 NATO Allied Joint Publication-01.

Footnote 39 Aaron Jackson, Doctrine, Strategy and Military Culture: Military-Strategic Doctrine Development in Australia, Canada and New Zealand, 1987 – 2007, CFAWC, 2013.

Footnote 40 CA LL process. http://www.army-armee.forces.gc.ca/en/lessons-learned-centre/lessons-learned-process.page. Last consulted on February 1, 2017.

Footnote 41 CFMWS intranet page.

Footnote 42 CFD warfare centre study, May 2016.

Footnote 43 Development, Concepts and Doctrine Centre. https://www.gov.uk/government/groups/development-concepts-and-doctrine-centre. Last consulted on February 1, 2017.

Footnote 44 See CRS Internal Audit: Tactical Armoured Patrol Vehicle, 2011; CRS Internal Audit of Medium Support Vehicle System Project, March 2014; and CRS Internal Audit of Capital Project Cost Estimation, May 2013.

Footnote 45 PAD, Chapter 9 – Options Analysis and Business Case gives general direction on the process. For the Treasury Board model, see https://www.tbs-sct.gc.ca/emf-cag/business-rentabilisation/bcg-gar/bcg-gar-eng.pdf. Last consulted on February 1, 2017.

Footnote 46 CRS. Internal Audit of Joint Unmanned Surveillance and Target Acquisition System Project, March 2014.

Footnote 47 CRS. Internal Audit of Capital Project Cost Estimation, May 2013.

Footnote 48 DND/CAF Integrated Risk Management process.

Footnote 49 CFD Capability Based Planning Handbook, Chapter 5 and PAD 2015, Chapter 10 – Project Risk Analysis and the Project Risk Management Plan.

Footnote 50 For projects with risk issues, see CRS Follow-up on Internal Audit: Fixed-Wing Search and Rescue Project, Project, March 2014. For an improvement in risk management practices see ADM(RS) Follow-up on Internal Audit Tactical Armoured Patrol Vehicle, May 2016.

Footnote 51 Michael Byers. The Plane that Ate the Canadian Military, Canadian Centre for Policy Alternatives, April 2014.

Footnote 52 Elinor Sloan. “Canadian Defence Commitments: Overview and Status of Selected Acquisitions and Initiatives,” School of Public Policy Research Paper, Volume 6, Issue 36, December 2013.

Footnote 53 UK Ministry of Defence. The Defence Equipment Plan 2015.

Footnote 54 Martin Auger. “The National Shipbuilding Procurement Strategy: A Five-Year Assessment,” Library of Parliament, June 2015.

Footnote 55 Dan Gettinger. Drone Spending in the Fiscal Year 2017 Defense Budget, Center for the Study of the Drone, February 2016.

Footnote 56 J. Craig Stone, Implementing the Defence Procurement Strategy: Is It Working, Canadian Global Affairs Institute, 2016. Charles Davies, Canada’s Defence Procurement Strategy: An End or a Beginning?, CDA Institute, 2014.

Footnote 57 Australia has published annually a capability investment guide and, along with this year’s defence white paper, published the 2016 Integrated Investment Program. The UK Ministry of Defence has issued a Defence Equipment Plan since 2012.

Footnote 58 Report on Exports of Military Goods from Canada – 2015. http://www.international.gc.ca/controls-controles/report-rapports/mil-2015.aspx?lang=eng. Last consulted on February 1, 2017.

Footnote 59 IRPDA. http://www.forces.gc.ca/en/business-how-to-do/irpda.page. Last consulted on February 1, 2017.

Footnote 60 Annual reviews have been occasionally prepared by academics, but these, of course, lack the authority of internal DND/CAF or GC reviews. See David Perry, 2015 Status Report on Major Defence Equipment Procurements, Canadian Global Affairs Institute, 2015.

Footnote 61 US Government Accountability Office. Defense Acquisition: Assessment of Selected Weapon Programs, March 2016. http://www.gao.gov/assets/680/676281.pdf. Last consulted on February 1, 2017.

Footnote 62 For the UK see, for instance, the National Audit Office. Major Projects Report 2015 and the Equipment Plan 2015 to 2025. https://www.nao.org.uk/report/major-projects-report-2015-and-the-equipment-plan-2015-to-2025/. Last consulted on February 1, 2017. For Australia see https://www.anao.gov.au/pubs?f[0]=field_publication_type%3A9. Last consulted on February 1, 2017.

Footnote 63 David Perry, 2015 Status Report on Major Defence Equipment Procurements, Canadian Global Affairs Institute, December 2015.

Footnote 64 This dispersion aligns with the Hybrid Model suggested in the Report on Transformation 2011.

Footnote 65 Andre Leonard, Federal Government Finances: Questions and Answers, Library of Parliament, 2016. DND, Making Sense out of Dollars, January 2015.

Footnote 66 See Table 2.

Footnote 67 Totals may not match due to rounding.

Footnote 68 FTE = Full-Time Equivalent.

Footnote 69 Elinor Sloan. “Canadian Defence Commitments.”

Footnote 70 David Perry. Putting the “Armed” Back into the Canadian Armed Forces, CDA Institute, January 2015.

Footnote 71 D. Garrett-Rempel, “Will JUSTAS Prevail? Procuring a UAS Capability for Canada,” RCAF Journal 4.1, Winter 2015.

Footnote 72 Rationalization of Rotary Wing Aircraft in the [Australian Defence Force] http://www.mcmanus.ca/97RAAFSC/reynolds.htm. Last consulted on February 1, 2017. Geoff Slocombe. “Progress of Defence’s Helicopter Strategic Master Plan,” Asia Pacific Defence Reporter, February 2016. Marc Seleinger. “Navy Chief Approves Helicopter Consolidation Plan,” Aerospace Daily, April 10, 2002.

Footnote 73 Regarding the F-18 consolidation: Kim Nossal, “Late Learners,” International Journal, Winter 2012-13. Regarding consolidation of RCAF helicopter fleets: Auditor General’s Report of April 1998.

Footnote 74 Kim Nossal, “Military Procurement and the 2015 Election,” Policy Brief – Fall 2015.

Page details

Date modified:: 2018-11-16