Evaluation of Land Force Development

November 2021

1258-3-043

Reviewed by ADM(RS) in accordance with the Access to Information Act.  Information UNCLASSIFIED.

Acronyms

ACDB
Army Capability Development Board
ACDC
Canadian Army Capability Development Continuum
ACIMS
Army Collaborative Information Management Services
ADM(DIA)
Assistant Deputy Minister (Data, Innovation and Analytics)
ADM(IE)
Assistant Deputy Minister (Infrastructure and Environment)
ADM(IM)
Assistant Deputy Minister (Information Management)
ADM(Mat)
Assistant Deputy Minister (Materiel)
ADM(RS)
Assistant Deputy Minister (Review Services)
ARA
Accountability, Responsibility and Authority
CA
Canadian Army
CAF
Canadian Armed Forces
COS A Strat
Chief of Staff Army Strategy
DND
Department of National Defence
DRMIS
Defence Resource Management Information System
DSPP
Defence Services Program Portal
EAC
Evaluation Advisory Committee
FD
Force Development
FTE
Full-time Equivalent
FY
Fiscal Year
GBA Plus
Gender-based Analysis Plus
IDEaS
Innovation for Defence Excellence and Security
ISA
Integrated Strategic Analysis
IT
Information Technology
L1
Level 1
LFD
Land Force Development
OEM
Original Equipment Manufacturer
OPI
Office of Primary Interest
PAD
Project Approval Directive
PRICIEG
Personnel, Research/Development, Infrastructure, Concepts, IM/IT, Equipment, Gender-based Analysis Plus
SSE
Canada’s defence policy: Strong, Secure, Engaged
TES
Trained Effective Strength
UK
United Kingdom
VCDS
Vice Chief of the Defence Staff


Back to Table of Contents

Executive Summary

This report presents the results of the evaluation of the Land Force Development (LFD) Program conducted during Fiscal Year (FY) 2020/21 by the Assistant Deputy Minister (Review Services) (ADM(RS)) in compliance with the 2016 Treasury Board Policy on Results. The evaluation examines the relevance and performance of the LFD Program over a five-year period FY 2015/16 to FY 2019/20. 

Program Description

The LFD Program consists of Defence Program Inventory Program 4.3 Land Force Development. The LFD Program develops and manages the execution of activities that introduces new or modified capabilities and oversees the replacement of capabilities in order to achieve desired effects during Defence operations, activities or services. 

Overall Conclusions

Overall, the LFD is meeting a need within the Canadian Army (CA) and for the Canadian Armed Forces (CAF) as a whole. However, it is faced with numerous challenges that impact its ability to provide timely, agile and flexible approaches to capability development. Some of the issues are within the control of Chief of Staff Army Strategy (COS A Strat) to influence, while others require a more holistic departmental approach. 

Scope

The scope of the evaluation focused on the relevance and performance of the LFD Program, including effectiveness and efficiency. While the evaluation examined key processes, sufficiency of resources, governance structures and results related to the four pillars of the Canadian Army Capability Development Continuum (ACDC) – Conceive, Design, Build and Manage, it also focused on the interconnectivity and management of the pillars as they relate to the capability development process.

Results

Findings were identified in a number of areas across the LFD Program. Although some of these findings remain directly connected to the LFD Program itself, some are larger issues that were found to impact other Force Development (FD) Programs across the department, as highlighted in the Integrated Strategic Analysis (ISA). Key Findings of this evaluation centred around the following areas: 

LFD

  • Continued need and value added of the Program;
  • Governance and accountabilities; and
  • Replacement of capabilities.
  • ISA

    All findings referring to the ISA will be associated with this icon.

ISA Related

  • Innovation;
  • Timeliness and value of produced capabilities;
  • Processes and standards; 
  • Collaboration; 
  • Information management and integration; and 
  • Resources and training.


Back to Table of Contents

Evaluation Scope

Coverage and Responsibilities

The evaluation examined Program 4.3 Land Force Development during the period of FY 2015/16 to FY 2019/20. The scope was determined in consultation with the Evaluation Advisory Committee (EAC). The evaluation provides an assessment of the relevance, effectiveness (achievement of immediate outcomes) and efficiency (design and delivery) of the LFD Program. The evaluation examined key processes, sufficiency of resources, governance structures and results related to the four pillars of the ACDC – Conceive, Design, Build and Manage. It also focused on the interconnectivity and management of the pillars as they relate to the capability development process. Progress in meeting Canada’s defence policy: Strong, Secure, Engaged (SSE) initiatives and other government commitments were also examined.

This evaluation is one of three concurrent FD evaluations conducted in FY 2020/21. Findings from this evaluation containing elements outside the control of the CA, along with findings from the Air & Space FD evaluation and Navy FD evaluation, are further discussed in the ISA: FD.

Out of Scope

Despite being led by the CA, Joint Capabilities were not included in this evaluation. Rather, the activities of this unit will be examined as part of a future Joint Force Development evaluation.

The departmental procurement/acquisition process was also excluded, aside from references to its impact on LFD activities.

Methodology

The issues and recommendations within this report were informed by multiple lines of evidence collected throughout the conduct phase of the evaluations. The evaluation conducted a document review, case studies (n=5), interviews (n= 30), surveys (n=18, response rate of 41%), administrative data collection and international benchmarking (United Kingdom (UK) and Australia).

Key findings fall under two groups:

  • Findings within the control of the CA (LFD)
  • Findings outside the control of the CA (ISA related)


Back to Table of Contents

Program Profile

Program Objectives

The LFD program develops and manages the execution of activities that introduces new or modified capabilities and oversees the replacement of capabilities in order to achieve desired effects during Defence operations, activities or services. It includes capability development managed and overseen by COS A Strat with acquisition being conducted by Assistant Deputy Minister (Materiel) (ADM(Mat)).

The overarching goal of the LFD program is to ensure that the CA has the capabilities it needs to be successful in operations today and in the future. In striving towards this goal, there are three overarching objectives of the Program.

  • Build readiness by closing capability gaps;
  • Enable modernization to assure the CA competitive advantage; and
  • Strive for interoperability with its key allies and within the CAF.

Program Stakeholders

Internal to CA

  • Canadian Army Land Warfare Centre
  • Land Force Development
  • Land Requirements
  • Land Command and Information
  • Land Infrastructure and Land Environment
  • Director General Army Reserve, Field Force, Canadian Army Doctrine and Training Centre, Canadian Army G3
  • Chief Of Staff Army Operations

Internal to Department of National Defence (DND)/CAF

  • Assistant Deputy Minister (Human Resources – Civilian), Assistant Deputy Minister (Information Management) (ADM(IM)), Assistant Deputy Minister (Infrastructure and Environment) (ADM(IE))
  • ADM(Mat), Assistant Deputy Minister (Policy), Assistant Deputy Minister (Defence Research and Development Canada), Chief of Military Personal
  • The Royal Canadian Air Force, Royal Canadian Navy, Special forces, Canadian Joint Operations Command

External to DND/CAF

  • External (Public Services and Procurement Canada, Defense Industry)
  • International (Five Eyes, North Atlantic Treaty Organization, etc.)

Program Resources

Figure 1
Figure 1. LFD Program Expenditures FY 15/16 – FY 19/20 ($millions)1.
Figure 1 Summary

Program expenditures for the period include the restated figures for FY 2015/16 to FY 2018/19 and actual amount in expenditures for FY 2019/20. The increase in dollars in FY 2017/18 were due to an in-year transfer related to SSE.

In FY 2019/20, the LFD Program was supported by 170 military FD positions and 42 full-time equivalent (FTE) civilian positions.2

Source:

  1. Defence Resources Management Information System
  2. Data provided by Program


Back to Table of Contents

Key Findings: Land Force Development

The LFD Program is relevant to the CAF's operational needs.

FINDING 1: The LFD program adds value to the CAF and remains connected to government and departmental priorities and objectives.

Lines of evidence indicate that there is a continued need for the LFD program in order to meet the needs of the CA as identified in SSE, the Future Security Environment 2013-2040 report, and the goals set out in the Minister of National Defence Mandate Letter. Interview responses described the LFD program as value added by providing the army with the capabilities it needs for operations today and into the future. This is corroborated with survey results which identified a continued need for the LFD Program.

Analysis of current LFD projects suggests the Army capability development priorities reflect the anticipated strategic needs of the SSE, Future Security Environment, Army of Today, Army of Tomorrow documents and Future Army strategic documents.

Figure 2
Figure 2. Survey results indicate there is a continued need for the LFD Program.
Figure 2 Summary

Ongoing relevance was influenced by a number of challenges

Evolving technology

Challenges in keeping pace with constantly evolving technology pose a high risk of obsolescence, especially related to technology-based capabilities (to be further discussed in Finding 6).

Length of capability development process

There is room for improvement in enabling modernization to assure the CA’s competitive advantage. Issues identified included the continuum process being too long compounded by delays within project files (to be further discussed in Finding 8).

Governance structures are in place.

FINDING 2: Governance and oversight structures are generally clear and adhered to, especially under the Build pillar.

There are several committees involved in the governance and oversight of LFD projects, including the Army Program Board, Army Science and Technology Board, Senior Review Board and the Army Capability Development Board (ACDB). The ACDB is intended to be the main committee for project approval.

Some interview respondents raised concerns about the frequency and focus of ACDB meetings, believing that these meetings do not occur as often as needed and that, more recently, the focus has moved from endorsement and decision making to more information items. Based on a review of ACDB agenda items, frequency of the meetings remained stable during the evaluation period, and there was a tendency to have more discussion/information items than decision/endorsement items.

Due to the large number of LFD projects in progress at any given point, the urgency tied to some projects, and the frequency of ACDB meetings, the evaluation found that some projects did not go through the ACDB itself but were approved secretarially. Although this flexibility improves efficiency by not holding up project progression, collaboration and engagement across stakeholders is reduced.

Although this evaluation did not include a comprehensive examination of policies and procedures guiding the LFD program nor an assessment of whether they are being adhered to, there is general evidence to support the fact that policies and processes are being adhered to.

100% of survey respondents agreed that policies relevant to LFD activities are being adhered to.

         

80% of survey respondents agree processes relevant to LFD are being adhered to.

  • According to the Defence Services Program Portal (DSPP), four out of the five case study projects moved through at least one level of approval by going before one of the oversight boards or committees for the LFD Program.
  • Only one out of five of the case study projects had been presented to the ACDB.
  • According to case studies, most projects completed the mandatory documentation under the Project Approval Directive (PAD); however, two projects did not include the Project Complexity and Risk Analysis or the Project Management Plan.
  • Although four out of five case study projects included some sort of PRICIE analysis, the extent this was done varied by project (see Finding 7).
  • Gender-based Analysis Plus (GBA Plus) was not a mandated requirement at the time of any of the case study projects.

Figure 3

Figure 3. Survey results indicate that most respondents believe there is sufficient oversight of Canadian Army Force Development.
Figure 3 Summary

The current process to replace capabilities faces challenges.

FINDING 3: Delays and challenges in the process to replace capabilities can cause inefficiencies and potential duplication

All capabilities have a shelf life and are replaced as new capabilities are produced. Replacement of capabilities is an important consideration in the ACDC to ensure replacement capabilities are planned for, and there is a smooth transition to the implementation and training of new capabilities. Although the CA is a stakeholder in this process, it is owned by ADM(Mat).

Results from the evaluation indicate room for improvement in the planning for and process of replacing capabilities. Although 67% of survey respondents agreed that the process for sunsetting/completing existing capabilities was well managed, nearly one third of interviews conducted with LFD staff expressed concerns related to this process in the CA.

The CA maintains a 10-year divestment plan that is updated at least once a year. This plan is focused on the divestment of equipment found within capabilities (i.e., to provide updated equipment for the capability).

Observation: A Supply Administration Manual has been in place since 2016 and was recently updated in Spring 2021. It includes an overview of all processes related to divestment planning and execution.

"In a resource-constrained asset management program, the first process cut is divestment because decision makers don’t have to deal with its ramifications."

Table 1. There are numerous challenges to the replacement of capabilities than can lead to inefficiencies.

There are numerous challenges to the replacement of capabilities that can lead to inefficiencies.

Decommissioning storage

Issues with the actual process of decommissioning equipment being left unprocessed impacts the capabilities of the facility storage (e.g., Light Armoured Vehicles/Coyotes vehicles). 

Timing

The start date for equipment replacement sometimes occurs when we no longer support the platform, impacting the effectiveness of the capability such as interoperability with allies and capability gaps. Original equipment can no longer be purchased for replacement, leading to multiple versions of equipment being purchased to fill the specific capability (e.g., multiple versions of hazmat suits due to product changes over time). Additionally, some equipment is utilized right until its end and then gets too old to fix and/or is too expensive to replace.

Process

Although it was reported that capabilities nearing the end of their life-cycle are monitored and tracked, there are factors outside of DND's control that impact the timing of new capabilities, including challenges with the capability development process itself (see Finding 5), resources and approval processes.

Capability development delays

Delays in new capability development can impact replacement planning and implementation as bases are reluctant to replace old capabilities prior to obtaining new ones. With capability tradeoffs frequently occurring, including the production of fewer number of capabilities available for distribution, this increases hesitancy to get rid of older capabilities.

Table 1 Summary


Back to Table of Contents

Key Findings: Integrated Strategic Analysis Related

Innovation is occurring but could be improved.

FINDING 4: Innovation is present but could be improved, especially in how industry is engaged and capabilities are produced. ISA

Innovation is an important element in capability development. Evidence gathered through the evaluation identified that innovation is being considered and supported in the LFD Program but, in some areas, it is still in its infancy. Additionally, challenges with the transition from innovative ideas to implementation were noted related to policies and procedures, risk acceptance levels and lack of dedicated resources.

Interviews highlighted a part-time position that had been created in the LFD Program that was a dedicated Science & Technology scientist who would look at Army emerging and disruptive technology. Interviewees believe that extension of this position would greatly benefit the LFD Program as without it, there is a gap in the connection between LFD and emerging science and technology.

The Innovation for Defence Excellence and Security (IDEaS) program was identified as a key innovation initiative referenced by respondents in the evaluation. According to internal program documents, calls have resulted in support for 40 proposals addressing 9 major capability challenges, representing a $35 million investment in Canadian industry and academia over the last two fiscal years. The impact of these investments is not yet known.

Figure 4
Figure 4. 91% of survey respondents agree the LFD is supporting innovative capability development.
Figure 4 Summary

Innovative technologies currently being integrated by the Army:

  • Silent Watch Battery Pack, which allows for full functionality of all vehicle mission essential systems and sub-systems for a minimum of 8 hours.
  • Short-Wave Infrared, a leading- edge technology that can be used both during the day and at night, making it more practical than night vision alone.
  • Greening initiatives focused on achieving better emissions standards than previous fleets of vehicles.

Industry plays a key role in innovation and in supporting the LFD program in keeping pace with change, especially related to technological advancements. In addition to sufficient resources and updating policies, it was felt that in order to be successful, the LFD must establish a way to contract with industry that allows for flexibility and adaptability. Although there has been some reported success related to this, it is still an area that requires improvement.

Example: the Tactical Armoured Patrol Vehicle project is the first fleet in the CA to negotiate a performance-based support contract with its original equipment manufacturer (OEM). Under the 25-year contract, the OEM will provide the long- term support for the fleet for five years, following the last vehicle delivery and uses a performance-based payment system to incentivize timely and effective OEM support. This has been a more efficient approach to contracting.

Interviews with land-based capability development program allied counterparts in Australia and the UK also emphasized the importance of innovation. The Australian Army holds an Innovation Day event where calls go out to industry for innovative approaches to address key issues. If bids are successful, they receive funding for three years. This is similar in nature to the IDEaS program. Also, the UK has made use of innovation funding in order to purchase some technologies “off the shelf,” rather than mandating that government own that technology. This allows updates to be added more easily.

Evolving technology poses a challenge for our current Capability Development process.

FINDING 5: Current processes and standards do not allow the LFD program to keep pace with changing technology and software development as part of capability development. ISA

Current capability development processes and standards impact the LFD Program's ability to keep pace with changing technology. Processes tend to be rigid and not allow for the flexibility necessary to adapt to constantly evolving technology. This leads to increased potential for capabilities to be obsolete by the time they are developed. Updates to the PAD and delegation authorities announced in SSE should allow for quicker capability development for certain projects. However, interviewees noted that these different options were not always being used.

Traditionally, the government approach to procurement has been a rigid contractual process with industry for a specific deliverable with payment being issued based on deliverables and/or set timeframes. Allocating a certain dollar figure at the beginning of the contract and then waiting 15 years for it to be produced does not take into account market value changes, meaning fewer units of the capability. Further, once produced, the government then takes ownership of that deliverable. Any necessary changes or updates to that deliverable would then require a separate contract. Although this process works for more traditional equipment, it creates challenges for capabilities that involve technology and/or require software updates. The example of an iPhone was routinely highlighted. Under the current process, the government would develop a contract to produce a phone. Once received, that phone is owned by the government and any software updates would not be included.

Technology reliability levels are another challenge identified through the evaluation. When proceeding with the development of a larger capability, the technology might not currently be of the desired quality or stage, which then impacts capability development timelines. As a result, LFD projects have been forced to make difficult decisions with available funding, by investing in more limited number of long-term capabilities in the immediate term and reserving some resources for later when the shorter-term, technology-based capability is ready, or has been updated.

Figure 5
Figure 5. Survey results indicate there is room for improvement in how the LFD is enabling modernization to assure the Canadian Army’s competitive advantage.
Figure 5 Summary

Interview responses identified the need for a more agile procurement process and a need to differentiate projects by short/long-term capability procurement cycles. In some cases, capability projects would require both long-term and short-term procurement cycles. For example, the development and procurement of armoured vehicles would fall under a long-term capability procurement cycle, while its integrated Information Technology (IT) systems would require multiple upgrades over the vehicle’s lifespan. These IT systems would need a short-term capability procurement cycle (sometimes referred to as a fast procurement cycle). Without upgraded software procured over a shorter timeline, it may impact the armored vehicle in question with a lack of communications with other capabilities.

Example: The Logistics Vehicle Modernization project was originally mandated to be equipped with a communications system that would become obsolete by the late 2020s. Instead, the project team decided only to equip 20-30% of the fleet with the current system and discussions are ongoing regarding the replacement communications system for the remaining vehicles.

Timelines are a challenge with Capability Development.

FINDING 6: Capability Development Timelines do not always meet the needs of the CA. ISA

The capability development process is hampered by lengthy procedures, challenges and shifting priorities, which ultimately cause significant delays in the timely production of capabilities.

Figure 6
Figure 6. 54% of survey respondents disagree capability development timelines are acceptable.
Figure 6 Summary
Figure 7
Figure 7. 49% of funded capital projects on DRMIS are experiencing risks to its schedule.
Figure 7 Summary
Figure 8
Figure 8. Survey results indicate the ACDB enables effective prioritization of capabilities; however, there is room for improvement in how the LFD delivers new capabilities.
Figure 8 Summary

Although it was reported that 100% of priority capability projects were on schedule in FY 2018/19, in FY 2019/20, only 60% of Canadian Army Capability Development initiatives were on schedule.

Prioritization of capability projects is a complicated process in LFD, with Program representatives feeling that prioritization criteria is unclear and often dependent on who is in charge at the time, causing priorities to change over time. However, SSE sets the overarching priorities for the LFD Program, particularly for major capital equipment projects since dollars are already profiled for those projects. The LFD program may have to prioritize within these projects as they move through the ACDC. Prioritization is not a linear process and often involves the balancing and rebalancing of resources to meet the most urgent requirements.

An analysis of the 24 active projects identified it took on average 9.7 years to move through the Definition stage only. With capability development timelines being well over ten years, the likelihood of producing obsolete capabilities is very high. Keeping pace with constantly evolving technology poses a high risk of obsolescence, especially related to technology-based capabilities (see Finding 5).

PRICIEG* and GBA Plus are being conducted in the Land Force Development Program.

FINDING 7: PRICIEG is generally being integrated but there are inconsistencies in its understanding and application. GBA Plus was shown to be successfully incorporated and accepted into the PRICIEG framework. ISA

Example: The Tank Replacement Project was undertaken to replace the aging Leopard 1 tank fleet with the Leopard 2, providing the Army with a more modern direct fire capability. As the project was intended to be a 1-for-1 replacement, the need for a comprehensive PRICIEG analysis was waived due to the assumption that the requirements would be similar. However, it became apparent that the Leopard 2 was a more complex vehicle than the one it had replaced, and that increased support requirements for the tank had been overlooked. The result has been reduced serviceability of the replacement fleet, which poses a threat to the Army's operational capabilities.

PRICIEG analysis is meant to facilitate the effective delivery and integration of new land capabilities. It examines and plans for the capability’s impacts on institutional factors such as personnel and training requirements; infrastructure and the environment; information technology; and support for equipment.

The ACDC identifies that initial PRICIEG considerations should be done in Conceive and then Indicative PRICIEG considerations are done in Design. The substantive PRICIEG analysis is done in Build. However, in practice, there is disagreement about when PRICIEG should be conducted. 

Nevertheless, some evaluation participants have asserted that PRICIEG may only be undertaken in the Build pillar, after Conceive and Design work has been completed.

A lack of consensus on when and/or how to properly conduct PRICIEG can hinder its effectiveness and, in turn, threaten a capability's successful integration into existing infrastructure. Improper conduct of PRICIEG may lead to complications, such as higher costs for capability maintenance. Additionally, conducting PRICIEG prematurely may lead to inaccurate results, as many capabilities undergo modifications between the Conceive and Build pillars due to trade-offs.

Figure 9
Figure 9. 64.3% of survey respondents indicated that PRICIEG should begin in the Design pillar.
Figure 9 Summary
Figure 10
Figure 10. Individual components of PRICIEG.
Figure 10 Summary

*Personnel, Research/Development, Infrastructure, Concepts, IM/IT, Equipment, Gender-based Analysis Plus.

In 2019, GBA Plus was officially incorporated into the PRICIEG framework for the CA, replacing "Generate." This resulted in the expectation that a thorough GBA Plus analysis be conducted for every capability being developed.

The evaluation team heard that prior to the formal roll-out of GBA Plus, the LFD program had sought to ensure that equipment design should accommodate users of a range of physical statures, according to data collected on CAF personnel by the DND Canadian Anthropometric surveys.

Due to its recent addition into the PRICIEG framework, the extent to which GBA Plus is integrated into capability design cannot yet be fully determined.

"They are only now beginning to start incorporating GBA Plus into Design, by having someone from GBA Plus in the room for working groups to ask questions and challenge the way they think about things."

Interviews with LFD stakeholders suggest that, in recent years, GBA Plus considerations are being more and more incorporated into capability projects, particularly in the Conceive and Design pillars. This has taken the form of training for program staff as well as consultation with gender specialists from the Directorate for Gender Equality and Intersectional Analysis, who have been increasingly represented at LFD meetings and working groups.

Figure 11
Figure 11. Individual components of GBA Plus.
Figure 11 Summary
Figure 12
Figure 12. Survey results indicate LFD program stakeholders overwhelmingly agree that GBA Plus is being incorporated into capability development, with the most consideration given during the Build pillar.
Figure 12 Summary

Trade-offs between capabilities are common challenges.

FINDING 8: Although the CA typically produces the capabilities it needs, there is often a resource/capability trade-off with some capability gaps left unfunded/unaddressed. ISA

There is often a need to reconcile the stated capability requirements of the Army and the affordability of achieving those requirements. It is a process of matching policy commitments to actual levels of investment. Combined with long procurement timelines, this can result in trade-offs to the final capabilities delivered by the LFD program, in the form of less equipment, modified capabilities or other compromises. These trade-offs can lead to increased depreciation and a shorter life-span for produced capabilities, higher maintenance costs and reduced protection and/or capacity for the CAF. Despite this, most program stakeholders agreed that the LFD program meets the CA's core capability requirements.

5 out of 5 case studies have shown capability development projects delivered the core capability requirement with capability trade-offs or compromises.

Example: Due to budgetary constraints, the Logistics Vehicle Modernization project was forced to reduce the number of vehicles originally intended to be purchased by approximately half. Despite this difference in expected vs. actual number of vehicles, the project will still meet what the project team has identified as its "core" capability requirement.

Figure 13
Figure 13. 50% of survey respondents agreed final produced capabilities align with originally identified gaps.
Figure 13 Summary
Figure 14
Figure 14. 70% of survey respondents agreed LFD delivers new capabilities that meet core capability requirements.
Figure 14 Summary
Figure 15
Figure 15. Survey results indicate there is room for improvement in how the LFD delivers capabilities that align with operational land requirements and in addressing capability deficiencies.
Figure 15 Summary

"Final capabilities align fairly closely to their original proposals, but in going from proposed capability to actual capability implementation, there are many factors taking place along that continuum."

FD enablers are key to the successful development of LFD capabilities.

FINDING 9: Areas of responsibility and collaboration across the LFD Program and with key enablers is not formalized. ISA

Force Development activities for the Army are spread out across multiple groups and Level 1s (L1), and often operate quite independently from each other. Overall, 90% of internal stakeholders responding to the survey felt the LFD has mechanisms or processes in place to capture their organization’s land requirements.

The lines between pillars and stakeholders are becoming increasingly blurred, and there is a need for areas to work together to maintain momentum and keep up with project changes and decisions. Although there is an overarching governance document for the LFD Program, it has not been updated since 2012 or re-issued since 2015. There are broader departmental FD guides and handbooks, including the Capability-based Planning Guide and the PAD, but these are higher level documents that do not outline the roles and responsibilities specifically for the Army.

Figure 16
Figure 16. Survey results indicate that most respondents believe collaboration is sufficient; however, room for improvement exists, especially in collaboration between stakeholders.
Figure 16 Summary

“All stakeholders are invited to have a say, from the moment of Conceive, even if the weight of their effort won’t matter till later in process.”

The majority of groups, especially in the Conceive, Design and Build pillars of the ACDC, fall under the responsibility of COS A Strat, which has implemented a number of processes that has helped build collaboration within that area and has helped clarify Accountabilities, Responsibilities and Authorities (ARA) internally. These have included informal working groups established at the conceive, design and build pillars of the ACDC, and orientation days for employees with tabletop exercises outlining how each group works together within the continuum. When implemented, they are perceived to produce positive returns for building collaboration and clarifying ARAs, thus improving efficiency. Despite their benefit, they are not a formal requirement but are put in place based on the discretion of project leads and availability of resources both within the LFD Program and with FD enablers including ADM(IM), ADM(Mat) and ADM(IE). Although, Human Resources capacity issues across FD enablers reduces availability to engage in LFD activities, integrating a broad collaboration effort within the FD model was suggested as a way to mitigate this challenge.

Timing and level of engagement and communication with key enablers could be improved. Involvement of these groups early in the process was reported to facilitate hand over and file transfers as well as increase understanding of the capability gap that needs to be addressed. Having a central repository to track progress, even when projects move outside the responsibility of COS A Strat, could help the Army maintain awareness of project status.

Information management is a challenge for the LFD Program.

FINDING 10: There is not a centralized information management platform being used by the LFD Program to support LFD projects, leading to challenges in oversight, knowledge transfer and technical integration. ISA

Multiple information management platforms are currently involved in storing project data and documentation for the LFD Program. The Capability Investment Database previously served as the department’s primary source for project information; however, this has been transferred to the Defence Resource Management Information System (DRMIS) and the DSPP.

While feedback from program stakeholders and representatives from ADM(IM) on the DSPP has generally been positive, perceptions of DRMIS have been mixed.

Accessibility issues were a recurring concern of DRMIS users. Interviews of LFD program representatives highlighted that obtaining consolidated data through DRMIS is challenging. Inconsistencies in project coding can create challenges for extracting data to track projects. Furthermore, it has been reported that project directors have faced restrictions on their ability to modify project information on DRMIS.

“The DSPP is a great step forward but is only one year in its implementation. Continued work, experience and training will help this become the IM solution for capital projects.”

Figure 17
Figure 17. Survey results indicate there is room for improvement with interoperability.
Figure 17 Summary

Among directorates within LFD, the Army Collaborative Information Management Services (ACIMS) is used as the system of record for capital equipment projects. However, the lack of a centralized method to track and transfer this information limits its effectiveness and overall Program efficiency. Interview comments noted that some project managers have resorted to creating their own information repositories on ACIMS but would prefer a centralized system that is automated and open to collaboration.

The LFD program currently does not have access to a dashboard that can pull together a comprehensive overview of the capability development process. This results in challenges in performance measurement tracking due to the lack of a methodology to track how many gaps have been identified and how many of those are being addressed through capability development projects.

ADM(RS) Recommendation

Despite a slight increase in FTEs, the LFD Program is challenged by vacancy rates, stability of staff and lack of training.

FINDING 11: Perceived limited resources and training in the LFD program creates challenges for the program. ISA

Interview and survey results reported a perceived shortage of resources facing the LFD Program, which impacts the overall effectiveness of the program. Some areas in the LFD program, including the Canadian Army Land Warfare Centre, have consistently documented a request for additional funding as part of annual business plans for the entire duration of the evaluation period. Despite this need, no additional funding was received. LFD expenditures represented 11% of total CA expenditures in FY 2018/19 and 9% in FY 2019/20.

  • Ad hoc training/on-boarding and challenges in retention and posting cycles posed difficulties for the LFD Program.
  • Interview comments reported an average of six months or longer of on-the-job experience to gain the necessary skills needed for the position, leading to a decrease in capacity of employees to do FD work. In addition, a high percentage of military staff suggests the LFD is heavily impacted by frequent posting cycles, with the perception that employees are provided with training and then posted out to new positions once they gain the skills and experience needed for the job.
  • Although training programs such as the Army Technical Staff Officer Program or the Army Technical Warrant Officer Program exist, there is a lack of an overall FD training program available to all LFD employees managing capability development.
  • The CAF was expecting an approximate shortfall of 5,000 personnel in Trained Effective Strength (TES) (2017). This did not take into account the fact that personnel within the TES did not all have the right qualification, occupation and rank to fill positions that were within our Trained Effective Establishment nor personnel on Leave Without Pay or the Service Personnel Holding List.
  • It was noted that although SSE covers significant investments, it does not include funding for many of the activities occurring during the first pillars of the ACDC.
  • Based on February 2020 LFD program documentation, there were 203 FTEs.
    • 167 were military and 36 were civilian
    • Vacancy rate: 14%
  • Based on March 2021 LFD program documentation, there were 212 FTEs.
    • 170 were military and 42 were civilians
    • Vacancy rate: 16%

Note: Data on FTEs obtained from Program for the other years within the scope of this evaluation were not available.

Table 2. LFD Spending.

-

Spending

2018/19

$90,644,170.67

2019/20

$ 91,055,749.80

Sommaire du tableau 2

This table has 3 rows including the title row which says “Spending”. The first column lists the fiscal year, and the second column lists the dollar amount of spending that year.

Figure 18
Figure 18. 40% of funded capital projects on DRMIS are listed as experiencing risks to budgets.
Figure 18 Summary

100% of survey respondents disagree the resources available to the LFD are adequate.

89% of survey respondents disagree LFD resources are redundant, duplicative, under- utilized or otherwise unnecessary.


Back to Table of Contents

Conclusions

Overall, the LFD is meeting a need within the CA and for the CAF as a whole. However, it is faced with numerous challenges that impact its ability to provide timely, agile and flexible approaches to threat-based capability development. Some of the issues are within the control of COS A Strat, while others require a more holistic departmental approach.

There is a perceived lack of resources (both financial and FTE) dedicated to the LFD Program, particularly in the early stages of the ACDC. Due to the large number of capability gaps that need to be addressed by the CA and available resources, the program is faced with constant prioritization issues and the need to reallocate resources. Posting schedules and lack of formal FD training and development intensify resource issues facing the Program. Although effort is being made across the department to address FD training gaps, this remains an issue for FD in the CA.

Innovation and engagement of industry are key to moving capability development forward. The LFD Program has made use of the various innovation groups across the department, such as the IDEaS program and should continue these efforts.

Although ARAs and governance are becoming clearer within COS A Strat, there is room for clarification and communication across stakeholders. It was generally felt that processes and standards are being well-adhered to within the LFD Program, especially within the Build pillar. PRICIEG remains a beneficial tool to support capability development, but timelines and approach could be clarified to ensure consistent application.

Although it was generally thought that the CA gets the capabilities it needs to address core capability gaps, the time it takes to develop these capabilities can result in them being obsolete by the time the capability is produced. Furthermore, resource/capability trade-offs were felt by LFD representatives to result in less equipment, fewer capabilities and/or other compromises being made. However, these trade-offs are sometimes necessary to ensure the core capability is delivered within available resources.

It is important that the LFD Program develops technologically relevant capabilities and has mechanisms to track capability development gaps and progress. The program must ensure that internal information management practices allow for readily accessible project and program data to inform evidence-based decision making as well as effective knowledge transfer.


Back to Table of Contents

Annex A – Management Action Plan

ADM(RS) Recommendation

Management Action

The CA acknowledges this issue. It also recognizes that it is part of a Departmental challenge where a need exists to align CA use of LFD information with that of other departmental stakeholders such as ADM(Mat), Chief of Force Development and ADM(IM).

Action 1.1: The CA recognizes its own deficits in the management of knowledge, information, and data. In order to address those immediately, the Army Collaboration Environment is currently undergoing the first phase implementation of the CA electronic Central Registry (CAeCR implementation order) which aims to capture all Information Resources of Business Value (IRBV) and Information Resources of Enduring Value (IREV). The streamlining of data management in LFD will be addressed by the CAeCR plan already in place which will realize solutions to the ADM(RS) identified issue NLT Q1 FY 22/23. This approach will improve LFD by ensuring proper life cycling of corporate records thereby mitigating the risk to the department as it relates to information loss, undue retention, and time required to produce information on demand.

OPI: CA – Command Information Management Officer
Target date: May 2022

Action 1.2: The CA Headquarters is also working to improve information workflows with expected rollout not before Q3 2021, to better connect the staffing of documents to their final retention within the CAeCR.

OPI: CA – Headquarters Information Management Officer
Target date: December 2022

Action 1.3. With respect to how data is managed, Director Army Staff has recently been assigned as the Army Chief Data Officer. Director Army Staff, in close collaboration with Assistant Deputy Minister (Data, Innovation and Analytics) (ADM(DIA)), are developing army level governance and policy in line with the proposed Defence Administrative Orders and Directives policies.

OPI: CA – Director Army Staff
Target date: February 2022

Action 1.4. In addition to tackling the recommendation, the CA is also embarking on its own digital transformation journey. The broader issue related to the Army’s use of data will be addressed through ongoing digitalization efforts across the CA which are aligned with those of DND/CAF. We are leveraging the work undertaken through the Vice Chief of the Defence Staff (VCDS) with ADM(DIA) and other L1 stakeholders. The CA recognizes the importance of the data fabric enabled by robust communication and information systems and, where it can make progress from within, will endeavor to do so partnering with stakeholders. While remaining aligned with departmental guidelines and nested within the VCDS-led effort, the CA will develop its own digitalization strategy and supporting documents; the details of which will be made available when ready. The CA also intends to use this specific LFD Program data management Management Action Plan as a use case to inform its broader digital transformation strategy which will see work commence as early as Fall 2021.

OPI: CA – COS A Strat
Target date: December 2022


Back to Table of Contents

Annex B – Evaluation Context

This report presents the results of the evaluation of the LFD Program conducted by ADM(RS) as a component of the DND Five-Year Evaluation Plan (FY 2017/18 to FY 2022/23). It was approved by the Performance Measurement and Evaluation Committee in March 2020, and an EAC comprised of program representatives and members of the evaluation team was established to help guide the evaluation.

The evaluation examines the relevance and performance of the LFD Program during the period of FY 2015/16 to 2019/20 and was conducted in accordance with the Treasury Board Policy on Evaluation. Although a 2016 evaluation was conducted on Capability Development in DND, this is the first evaluation of the LFD Program as defined in the Departmental Results Framework, established in 2017.

The LFD Evaluation is one of three concurrent FD evaluations conducted in FY 2020/21. Findings from this evaluation, alongside findings from the Naval Force Development evaluation and the Air and Space Force Development Evaluation are presented in the ISA.


Back to Table of Contents

Annex C – Findings and Recommendations

Table C-1. Findings and Recommendations

Findings Recommendations

1. The LFD program adds value to the CAF and remains connected to government and departmental priorities and objectives.

No recommendation.

2. Governance and oversight structures are generally clear and adhered to, especially under the Build pillar.

No recommendation.

3. Delays and challenges in the process to replace capabilities can cause inefficiencies and potential duplication.

No recommendation.

4. Innovation is present but could be improved, especially in how industry is engaged and capabilities are produced.

Please refer to the ISA for a related recommendation. ISA

5. Current processes and standards do not allow the LFD program to keep pace with changing technology and software development as part of capability development.

Please refer to the ISA for a related recommendation. ISA

6. Capability development timelines do not always meet the needs of the CA.

Please refer to the ISA for a related recommendation. ISA

7. PRICIEG is generally being integrated, but there are inconsistencies in its understanding and application. GBA Plus was shown to be successfully incorporated and accepted into the PRICIEG framework.

Please refer to the ISA for a related recommendation. ISA

8. Although the CA typically produces the capabilities it needs, there is often a resource/capability trade-off with some capability gaps left unfunded/unaddressed.

Please refer to the ISA for a related recommendation. ISA

9. Areas of responsibility and collaboration across the LFD Program and with key enablers is not formalized.

Please refer to the ISA for a related recommendation. ISA

10. There is not a centralized information management platform being used by the LFD Program to support LFD projects, leading to challenges in oversight and knowledge transfer.

In order to streamline data management, the LFD Program should review its systems/databases currently being used by the Program to store information related to capability development projects as well as processes related to using these systems. ISA

11. Perceived limited resources and training/onboarding in the LFD program, inhibits the effectiveness and efficiency of the program.

Please refer to the ISA for a related recommendation. ISA

Sommaire du tableau C-1

This table has 2 columns. The first column outlines the finding, the second column contains the associated recommendation, if any.

Page details

Date modified: