Evaluation of CAF Bases and Wings Sustainment Programs

Executive Summary

This report presents the findings and recommendations of the Evaluation of Bases and Wings Sustainment Programs within the Canadian Armed Forces (CAF). The evaluation was conducted by Assistant Deputy Minister (Review Services) (ADM(RS)) as a component of the Department of National Defence (DND) Five-Year Evaluation Plan, in accordance with the Treasury Board (TB) Policy on Results (2016). The evaluation examined the relevance, effectiveness and efficiency of support provided by Bases/Wings focusing on Fiscal Years (FY) 2013/14 to 2017/18. Evaluation methods included interviews, site visits, surveys, document and literature reviews, and financial data analysis. As the expression “Wing” reflects the Royal Canadian Air Force (RCAF) Formation role, the use of the term “Base” in the report includes the RCAF “Wings” that perform base functions.

Program Description

CAF bases provide a variety of support services within an assigned geographical area. Depending on the specific base, services may be provided to Royal Canadian Navy (RCN), Canadian Army (CA), RCAF, Military Personnel Command (MILPERSCOM) and Canadian Special Operations Forces Command (CANSOFCOM) regular and reserve formations and units in support of the conduct of welfare, training, readiness and operational activities.^{Footnote 1} The evaluation of the sustainment programs includes Bases/Wings grouped by element: Naval Bases (RCN); Land Bases/Garrisons (CA); Air and Space Wings (RCAF); and Joint and Common and International Bases (Vice Chief of the Defence Staff (VCDS) and MILPERSCOM).

Relevance

This evaluation supports the continued need of the CAF Bases/Wings Sustainment programs. Bases are necessary to provide services to CAF members to ensure their operational readiness, a contributing factor towards the CAF’s operational effectiveness. CAF Bases/Wings Sustainment programs are aligned with federal roles and responsibilities, as per the National Defence Act. CAF Bases/Wings Sustainment programs are aligned with departmental priorities including Canada’s defence policy: Strong, Secure, Engaged (SSE).^{Footnote 2}

Effectiveness

CAF bases have provided the required support to the operational units and personnel. The creation of base sustainment Departmental Results Framework (DRF) programs will elevate the importance of bases and provide the opportunity to coordinate base activities, results and indicators. Base sustainment challenges have included:

• Personnel staffing shortages due to the staffing priority and exchange of healthy base military personnel with medically restricted personnel from higher priority units;
• Multiple transformations such as DND infrastructure and IM/IT governance; and
• Out-of-date governance documentation.

Efficiency and Economy

As new programs constructed out of components of 20 different Program Alignment Architecture (PAA) elements, an analysis of historical program expenditures was not feasible. The individual base sustainment expenditures have also fluctuated significantly during the evaluation period due to multiple CAF transformations. The expenditure changes between FY 2016/17 and 2017/18 ranged from -11 percent (Canadian Forces Base (CFB) Borden) to 24 percent (CFB Gagetown). The creation of a Sustainment Working Group and alignment of sustainment activities across the programs will provide the information required to measure and improve efficiency and economy.

Key Findings and Recommendations

Back to Table of Contents

1.0 Introduction

1.1 Context for the Evaluation

This report presents the results of the ADM(RS) evaluation of the CAF Bases/Wings Sustainment programs which was conducted in compliance with the TB Policy on Results between January 2018 and November 2018. As per the TB policy, the evaluation examines the relevance and performance of the program over a five year period, FYs 2013/14 through 2017/18. The evaluation may be used to inform future senior management discussions regarding the sustainment of CAF Bases. As “Wing” reflects the RCAF Formation role more than the base function, the use of the term “Base” in the report includes the RCAF “Wings” that perform base function. No previous evaluations were conducted; however, with base sustainment being embedded in 20 PAA elements, CAF base sustainment has been evaluated as a component of multiple evaluations.

The evaluation was supported by representatives from Level 1 organizations and specific divisions, including VCDS, RCN, CA, RCAF and MILPERSCOM providing advice and support to ADM(RS). Consultations were held at key intervals throughout the evaluation, specifically when defining the project scope, developing the logic model, identifying key performance indicators, and reviewing preliminary findings.

1.2 Program Profile

1.2.1 Program Description

CAF Bases provide a variety of support services within assigned geographical areas. Depending on the specific base, services may be provided to Navy, Army, Air Force, MILPERSCOM and CANSOFCOM regular and reserve formations and units in support of the conduct of welfare, training, readiness and operational activities.^{Footnote 3} The evaluation of CAF Bases/Wings sustainment programs includes Bases grouped by element: Naval Bases (RCN); Land Bases (CA); Air and Space Wings (RCAF); and Joint and Common and International Bases (VCDS and MILPERSCOM).

The Naval, Land and Joint Common and International base sustainment programs are structured similarly with five main activity groups. The program activities include: base operations and emergency services (e.g., ranges, snow and ice control, fire department, harbour services, armouries, training areas); non-operational information technology services; logistics services (e.g., warehousing, transportation, equipment maintenance and procurement); administration services (e.g., food services, finances, accommodation, support to Military Family Resource Centres (MFRC), community relations); and safety and environment services. The grouping of services allows for economies of scale and efficiencies in the delivery of numerous services for military members and their families, and civilian members of the defence team.^{Footnote 4} The Air and Space base program is structured differently with three activity groups: operations support; mission support; and specialist support.^{Footnote 5} Regardless of the variation in the RCAF program structure, all programs provide a similar set of services with minor deviations attributable to the environment.

1.2.2 Resources

The 2018/19 planned expenditures for the Base/Wing Sustainment programs^{Footnote 6} are as follows:

• Program 6.8: Naval Bases - $245.6 million
• Program 6.9: Land Bases – $466.1 million
• Program 6.10: Air and Space Wings – $224.3 million
• Program 6.11: Joint and Common and International Bases - $106.0 million

Due to CAF Bases/Wings sustainment programs being both a new program and created as components of 20 elements from the former PAA, an analysis at the program level was not feasible for the evaluation period. Two large bases were selected to represent each program. This resulted in the 100 percent coverage of RCN bases, 50 percent coverage of CA bases and 20 percent RCAF bases, and coverage of the majority of the Joint Common and International bases. Table 1 denotes the CAF Bases/Wings Sustainment programs expenditures of the eight bases selected for the evaluation for the FYs of the evaluation period.

The fluctuations and overall decrease in expenditures are mostly attributable to the transfer of infrastructure expenditures to Assistant Deputy Minister (Infrastructure and Environment) (ADM(IE)) and to a lesser extent IM/IT expenditures to Assistant Deputy Minister (Information Management) (ADM(IM)) that occurred during the evaluation period. CFB Borden’s fluctuations were also affected by a lack of baseline funding; this is elaborated later in the report.

Personnel

The Bases/Wings Sustainment programs employ military and civilian personnel to accomplish their mission. Table 2 indicates the number of personnel by DRF category of the eight bases/wings selected for the evaluation in 2018.^{Footnote 8}

1.2.3 Stakeholders

The key DND stakeholders for the evaluation are the VCDS, the three Environmental Command Staffs, MILPERSCOM and their respective Base Commanders (B Comd). Other stakeholders include Canadian Special Operations Forces Command (CANSOFCOM), ADM(IE), ADM(IM) and SSC.

1.3 Evaluation Scope

1.3.1 Coverage

This evaluation was linked to the CAF Bases/Wings Sustainment programs under the DRF program 6.0 and has as the expected result that bases enable military operations and defence activities.^{Footnote 10} Specifically, this evaluation includes the following DRF programs:

• Program 6.8: Naval Bases;
• Program 6.9: Land Bases;
• Program 6.10: Air and Space Wings; and
• Program 6.11: Joint and Common and International Bases.

The CAF Bases/Wings sustainment programs are new programs constructed of components of 20 different PAA elements. The former PAA elements are as follows:

• 3.1.1 Maritime Roles – Readiness Sustainment
• 3.1.2 Land Roles – Readiness Sustainment
• 3.1.3 Aerospace Roles – Readiness Sustainment
• 3.1.5 Joint and Common Roles – Readiness Sustainment
• 3.2.3 Aerospace Environment – Integration Training
• 3.3.1 Maritime Environment – Force Element Production
• 3.3.2 Land Environment – Force Element Production
• 3.3.3 Aerospace Environment – Force Element Production
• 3.3.5 Joint and Common – Force Element Production
• 3.4.1 Maritime Environment – Force Element Production, Coordination, Command and Control
• 3.4.2 Land Environment – Force Element Production, Coordination and Command and Control
• 3.4.3 Aerospace Environment – Force Element Production, Coordination, Command and Control
• 3.4.5 Joint and Common – Force Element Production, Coordination, Command and Control
• 4.1.6 Military Personnel – Occupation Training
• 4.1.7 Military Personnel – Morale and Well Being
• 4.1.8 Military Personnel – Health Care
• 4.1.9 Organization – Security, Protection, Justice and Safety
• 4.2.6 Materiel – Inventory Management and Distribution
• 4.3.4 Real Property – Operations, Maintenance and Repair
• 4.4.3 Info Systems – System Management and User Support.^{Footnote 11}

The evaluation excluded issues and areas that have been previously evaluated or audited by ADM(RS) or by the Office of the Auditor General of Canada. Additionally, activities influencing the delivery of support by CAF Bases/Wings that will be the subject of a future evaluation are not covered in this evaluation. The evaluation scope excluded the following components of CAF Bases/Wings Sustainment programs:

• Military Policing services;^{Footnote 12}
• DND Infrastructure Strategic Management;^{Footnote 13}
• DND IM/IT management;^{Footnote 14} and
• Non-Bases/Wings funded MFRC services and activities.^{Footnote 15}

1.3.2 Issues and Questions

In accordance with the TB Directive on Results (2016),^{Footnote 16} the evaluation addresses issues related to relevance and performance. The methodology used to gather evidence in support of the evaluation questions can be found in Annex B. A logic model providing a theory of change for this program is included in Annex C. An evaluation matrix, listing each of the evaluation questions with associated indicators and data sources is provided in Annex D.

Back to Table of Contents

2.0 Findings and Recommendations

2.1 Relevance

There is an ongoing and demonstrable need for CAF bases. Spread throughout Canada, bases are also the “home” for many units and formations within the CAF. They directly house formations, brigade groups, squadrons, regiments, ships and units. The home base support includes the provision of a variety of critical services to on-base and remote military personnel such as medical care, financial support, food accommodations and socio-cultural events.^{Footnote 17} Depending on the specific base, services may be provided to Navy, Army, Air Force, MILPERSCOM and CANSOFCOM regular and reserve formations and units in support of the conduct of welfare, training, readiness and operational activities.

The services provided by bases support CAF operations and training of soldiers, sailors, airmen and airwomen, providing the support required to make them prepared and successful on operations.^{Footnote 18} Services typically provided include: base operations and emergency services (e.g., ranges, training area, snow and ice control, fire department, harbour services, military police, air traffic control); non-operational information technology services; logistics services (e.g., warehousing, vehicle and equipment maintenance, transportation and procurement); administration services (e.g., food services, finances, accommodation, support to MFRCs, community relations, HR, padres); and safety and environment services.^{Footnote 19} Acting as a hub for services and support to all operational units linked to domestic and international operations, bases substantiate their existence on a continuous basis.

Defence is a core federal government responsibility as articulated in the Constitution Act,^{Footnote 20} which defines and outlines the responsibilities and duties of the federal government, including the CAF and DND. Furthermore, the National Defence Act states the CAF shall consist of commands, formations, units, and other elements organized under the authority of the Minister.^{Footnote 21} Departmentally, the Organization and Establishment Policy (CFP 219) describes a CFB as a unit of the CAF, which has been designated as a base by or under the authority of the Minister. The role of a base is to provide personnel and materiel support for assigned units as directed by the Chief of the Defence Staff.^{Footnote 22} Lastly, the DND/CAF departmental plan highlights the requirement to “develop and manage modern, operational and sustainable bases and infrastructure.”^{Footnote 23}

CAF Bases/Wings Sustainment programs are aligned with priorities and initiatives contained in SSE. SSE identifies the aim of improving the support and services offered for military family members by increasing support to MFRCs that are partially supported and funded by the Bases/Wings Sustainment programs. CAF Bases/Wings also support the SSE aim to ensure DND supports the low-carbon government targets outlined in the Federal Sustainable Development Strategy. Thirdly, SSE has an initiative to improve infrastructure on Bases/Wings, including housing for CAF personnel.^{Footnote 24}

The delivery of CAF Bases/Wings Sustainment programs is aligned with the priorities of DND. This alignment is denoted in the following Departmental Results:

• 6.1 – Naval and Army Bases, and Air Force Wings enable military operations and defence activities (DRF 6.8, 6.9, 6.10, 6.11)
• 6.2 – Defence infrastructure is well-managed throughout its lifecycle (DRF 6.10)
• 6.3 – Defence activities are carried out in a safe and environmentally responsible manner (DRF 6.8 and 6.9).^{Footnote 25}

DND denoted the vital roles fulfilled by Bases/Wings during the generation of the DRF. Under the previous PAA, base/wing sustainment activities were accounted for in 20 different PAA sub-programs. Under the DRF, base sustainment activities and results are combined under specific base sustainment programs.

2.2 Effectiveness

Governance documentation is required to regulate the conduct of base sustainment. The guidance would include a clear definition of what constitutes a base, its Authorities, Responsibilities and Accountabilities (ARA) and, to the maximum extent possible, standardize the base operations. It is a challenge to identify the actual number of bases as they are defined in different ways across the CAF. When referring to the Canadian Forces Organizational Orders (CFOO) database,^{Footnote 26} 20 CFOOs use the expression “Base” to identify the support/sustainment organizations (CA for seven, RCN for two, and RCAF for eleven).

Following the CA transition to a divisional structure, the concept of “base” was modified into a Canadian Division Support Base.^{Footnote 27} As part of a Canadian Division Support Group construct, (there are four in Canada), they encompass a larger geographical area and provide support to units from different commands (e.g., RCN and RCAF). Following this transformation, it was unclear on how to identify the entities providing support/sustainment to the CA by a common name (bases, garrisons, or detachments). Organizational delineation of ARA are not well defined and do not adequately support the sustainment mission given by the program. According to staff interviewed, Service Level Agreements do not exist or are not updated to reflect the involved service levels. Service standards in the various components of support are established according to the Policies and Directives originating from National Defence Headquarters (NDHQ), without making the necessary adaptation to local contexts and practices.

The RCAF superimposed the formation and base roles in 2009^{Footnote 28} creating a unique construct. Many wing commanders are double-hatted with a mix of base and formation responsibilities; it results in having the same person commanding and controlling the execution of operations and the support to be provided to the formation and to the attached units. Eliminating one position may appear to be an efficient model; however this structure requires clear delineation of responsibilities to prevent a potential conflict of interest in the prioritization of support resources among all lodger units. According to the existing CFOOs, some wings only have a purely formation role (e.g., conduct training, provide air ops capability) and rely on other regional organizations to support them (e.g., CFB Halifax supporting 12 Wing, CFB Borden supporting 16 Wing, and CFB Kingston supporting 1 Wing).

Canadian Forces Publication (CFP) 219^{Footnote 29} is an overarching governance document that represents the authority for Organization and Establishment Change in DND/CAF. Chapter 7 covers bases, providing definitions and details on the relations between bases and supported units, base organization and structure. This document was last updated in June 2012 and does not take into consideration the recent changes in base structure, or IM/IT and IE management transformations affecting the manner bases use to conduct their sustainment/operations.

The majority of sustainment activities and outputs are the same for all bases. This supported the development of similar logic models for DRF programs 6.08 (Naval), 6.09 (Land), and 6.11 (Joint, Common and International). For example, fire and emergency services are under the same activity group (Provide Base Operations and Emergency Services) for all three programs. These commonalities should be leveraged across all four programs.^{Footnote 30}

Program 6.10’s (Air Force) logic model has a different grouping of base activities. The RCAF program has three main activity groupings: operations support; mission support; and specialist support. This difference from the other programs will prove challenging for the coordination of the Bases/Wings Sustainment programs by hindering direct activity linkages. A method will be required by the RCAF to link the 6.10 activities to the other sustainment programs. Additionally, attention will also have to be given to ensure clear separation between RCAF operations and base activities.

The military staffing priority structure was changed during the evaluation period. In December 2017, the six level staffing priority system (1 to 6) was replaced with a three level category system (A to C).^{Footnote 31} The intent was to staff category A from 95 percent to 100 percent and category B from 90 percent to 95 percent. Once category A and B requirements were met the staffing of category C would be filled at the prerogative of the Command element.^{Footnote 32} The Bases/Wings are assigned the staffing priority of C “remaining functions and capabilities.”

The base staffing priority resulted in shortages of CAF members in identified positions. The focus on staffing high category (A and B) positions with operational personnel results in personnel shortages being absorbed by lower category units such as bases. DND reported the percentage of positions filled as 98 percent for 2015/16 and 97.5 percent for 2016/17.^{Footnote 33} As Table 3 denotes, in 2018 the percentage of base regular force positions filled ranged from between 77 percent and 100 percent with an overall average for the eight bases of 89 percent. The two bases (CFB Borden and CFB Gagetown) with the highest percentage of positions filled are military training bases. However, this number does not reflect the actual personnel as training bases have a number of their positions filled by personnel completing on-the-job training (17 positions in CFB Borden)^{Footnote 34} before getting posted to other units.

Bases face the additional challenge of being utilized as a personnel pool for higher category units. When a person in a higher category unit is no longer operational (e.g., injury, maternity or paternity leave), the higher category unit will “swap” this person with an operationally ready one from the bases. The result is that even though a base shows “on paper” that they have a certain percentage of positions filled, there are actually less personnel fit for work without restrictions. As Table 3 denotes, with an average of 18 percent base personnel on restrictions, the military personnel without employment restrictions averaged 72 percent, well below the 90 percent threshold. When the restricted personnel data is included, CFB Halifax moves from being 10 percent more (92 percent vs 82 percent) to 10 percent less (62 percent vs 72 percent) than CFB Esquimalt.

The Bases’ staffing challenges have a negative effect on personnel quality of life for military and civilian personnel. DND has identified a healthy work environment as a key enabler in growing the CAF.^{Footnote 35} In order to maintain personnel in the military, there is a requirement to support a proper work-life balance. For example, the RCN does not keep sailors at sea for 20 years straight, and the CA tries to alternate between operational and non-operational postings for its members. Traditionally, the CAF utilizes base postings to provide an opportunity for a less stressful and more stable work environment enabling CAF members to reconnect with their families. Interviewees stated that the base personnel shortage and resulting increase in workload and stress have negative effects on the quality of life.

Similar staffing challenges were experienced with the base civilian personnel. Although no formal civilian staffing threshold exists, the percentage of positions filled by unrestricted personnel is below the 90 percent military threshold across all bases. Interviews revealed that the shortage of military staffed positions and the incremental and constant roll out of new programs result in workloads that exceed the staffing capacity. Over the long term, this situation causes the coping capacity of the existing employees to deteriorate as more personnel require special medical/stress leaves, leading to a cycle in work fatigue. Hiring more civilian staff to make up for the number of personnel on restricted employment is considered but not without its own obstacles, such as drawn out staffing processes, unique locations and competition with the private sector.

The time required to process civilian staffing documentation has been noted as a challenge in meeting the personnel requirements. CFB Gagetown conducted a trial where the optimum number of HR-Civ active files for processing was identified. A base-wide priority staffing list was then generated and maintained. The highest priority files were sent to HR-Civ for processing. As a file was completed, the next one on the priority list was then submitted for processing. This was found to greatly increase the HR-Civ throughput. Other bases have adopted this process and interviewees noted improved staffing performance as well. A challenge noted in adopting this process is in coordinating a number of units to produce a single prioritized civilian hiring list.^{Footnote 37} While this process is not a panacea for the staffing challenges, for example delays in classification, it is a practice that is worth consideration in other areas in DND.

The bases’ staffing challenges have a negative effect on delivery and performance. Bases prioritize their efforts on meeting client requirements before sustainment activities. Interviewees stated that the level of effort necessary to meet client requirements has led to delay or elimination of capability maintenance activities such as inventory taking. While this condition can be sustained for short surge times, over a protracted period of time the performance will be affected negatively. Base staffing shortages are to the point that actual service delivery is being adversely affected. Interviewees stated that Orderly Room manning was a particular example of this challenge. CFB Greenwood limits Orderly Room client access to mornings to enable the staff to complete necessary maintenance work in the afternoons and Canadian Forces Support Unit (Ottawa) (CFSU (Ottawa)) regularly closes their Orderly Room to walk-in service on Fridays.^{Footnote 38}

B Comds have influence but no control over base infrastructure decisions. In 2013, ADM(IE) was delegated full ARA for the centralized management of DND’s real property portfolio and began a transformation process to consolidate the number of real property custodians from nine^{Footnote 39} to a single ADM(IE) custodian model. This change took effect on April 1, 2016.^{Footnote 40} The ADM(IE) Real Property Operations Group (RP Ops) is responsible for infrastructure maintenance and recapitalization.^{Footnote 41} The strategic level infrastructure governance structure was covered in the 2018 Evaluation of Infrastructure Management – Real Property.

The governance structure between ADM(IE) and B Comds was noted as a recurring challenge in surveys and interviews. B Comds have been delegated the responsibility for environment under the Canadian Forces Organization Orders (CFOO)^{Footnote 42} and have authority to issue direction to personnel on CFB properties; however with centralization of real property, authorities for building and property management rest with ADM(IE). Due to differing priorities, this division of authorities can lead to challenges in resolving safety and environmental issues involving infrastructure. This has resulted in employees raising concerns and grievances through the chains of command about inadequate facility maintenance (e.g., asbestos management) and a lack of consultation with the civilian workforce.^{Footnote 43}

Site visits to various bases during the course of the evaluation noted a trend of issues regarding ageing infrastructure. Bases listed the long wait times and lack of clarity as an issue to deliver on their mandate for supporting base infrastructure. RP Ops’ delay to address some environmental concerns in a timely fashion exposes the B Comd to increased risk, particularly because the B Comd is “the face of the base.” For example, the gym at CFB Halifax was condemned with neither alternatives nor any plans for a new one resulting in a public affairs issue for the B Comd. The lack of RP Ops integration into the base communication plan resulted in the base users perceiving the B Comd as responsible. With the transfer of infrastructure to ADM(IE), the B Comd must remain responsive to infrastructure issues being experienced by the work force, but has no control over the outcome of when or how those issues are resolved.

Some bases have been able to mitigate this situation by including RP Ops in their senior staff meetings. This allows them to identify RP Ops prioritization of projects while simultaneously informing RP Ops of the bases’ pressing needs.

With the reorganization of IT activities in August 2011,^{Footnote 44} SSC became a major IT service delivery organization to all federal business units. Unfortunately, Bases’ experience with SSC reveals that there are no delivery thresholds for SSC activities. Service standards provide the client and service provider with the information required to effectively deliver the required results. The management of delays in the provision of SSC services has been challenging as there is no definition of the expected time to resolve an issue. The EITSM system defines an incident as something that does not behave as intended or that is affecting productivity with examples such as servers crashing, broken phones or applications running slower than usual.^{Footnote 45} Table 4 denotes an average of 24.4 percent of the incidents assigned to SSC which have been open for longer than 180 days. The lack of thresholds prevents senior management from identifying and rectifying challenges in the delivery of services to CAF Base clients.

Base IT sections are the frontline interface with clients for the resolution of IM/IT issues. Client and base interviewees stated that having no insight into the delivery of SSC services prohibits planning for resolution of client requests. The lack of information available to clients magnifies the effects to clients on delays in service. These issues are components of the Evaluation of Defence IM/IT being conducted by ADM(RS).

The hazardous nature of the diverse activities conducted by DND are supported by multiple functional areas of safety management. DND has 20 functional safety areas under the management of nine different organizations.^{Footnote 47} The DRF contains Safety Program 6.13 that ensures unity of effort and purpose to enhance the health, safety and well-being of the Defence Team.^{Footnote 48} This matter will be considered in the scoping of the upcoming evaluation of the DND Safety Program.

Bases have safety organizations and conduct activities to support the B Comd safety responsibility for personnel on base. The resources expended and results are managed as activities in their respective Base’s sustainment programs. The accurate reporting of DND Safety results and resources requires a method to track and compile information from the sustainment programs 6.08 to 6.11. The future evaluation of the safety program will examine this overlay from a strategic safety perspective.

Back to Table of Contents

2.3 Efficiency and Economy

The creation of the DRF and the development of specific CAF Bases/Wings Sustainment programs provide the opportunity to align CAF Base and Wing sustainment performance and results frameworks. The CAF Bases/Wings sustainment programs amalgamated base sustainment activities that were previously located in multiple PAA programs. Previously, base activities were not delineated from readiness as they are now clearly identified as specific programs in the DRF.

The VCDS Chief of Programme (C Prog), as the DND Head of Performance Measurement, stated the intention to create a working group with the aim of producing a comprehensive methodology and generating common service standards in the main logic model service areas.^{Footnote 49} During the course of the evaluation a formal working group was not in place and there was only sporadic coordination between respective Level 1 organizations.

The coordination of the program activities is a fundamental requirement to generating consistent results and performance measures and indicators. The maturity level of the results frameworks varies widely across the four programs. For example, while some programs have not yet identified sustainment activities, the RCN has generated a detailed matrix of sustainment sub-activities for each program activity group.^{Footnote 50} A CAF Bases/Wings Sustainment working group would identify this effort and support the results being leveraged by other Level 1 organizations. The alignment of the programs would also enable analysis of activities across the programs to identify best practices and areas for improvement.

Key Performance Indicators are neither fully developed nor communicated to program stakeholders, leading to confusion amongst program enablers. This lack can result in stovepipe structures at the operational level. For example, at a recent Army Comptrollers meeting, attempts were made to align activities to expenditures without the DRF Key Performance Indicators. The DRF will have lasting effects on reporting and accountability. A formal working group would create a common structure with a wide reaching communication plan.

The management of the DRF as it pertains to Bases and Wings will require ongoing support, consistent allocation of resources to the appropriate program and activity groups as well as adaptation to future transformations. Proper alignment of resources will also support a clear linkage between activities, outputs, indicators and results enabling the DRF work of one organization to be leveraged by other base sustainment stakeholders.

Multiple DND/CAF transformations and evolutions have resulted in changes to base sustainment activities. The number and scope of transformations show the need for a base-level task to resource analysis to be conducted. While the transformations have been managed as silos at the tactical level, the operational effect on overall base sustainment has not been assessed. The effects of the following changes have not been integrated with the holistic view of the CAF:

• Centralization of DND/CAF IT/IM under ADM(IM);
• Reorganization of IT support and Creation of SSC;
• CA reorganization into the Division Support Group construct;
• RCAF Wing as Formation and Base as sustainment organization construct;
• Centralization of Infrastructure and Environment under ADM(IE); and
• Resource Management Support (RMS) Occupation split into Human Resources Administrator (HRA) and Financial Services Administrator (FSA).

The processes of delivering HR and Financial support provided by base administration centres (Orderly Rooms) changed during the evaluation period. In 1998, the Administrative Clerk Finance Clerk, and Construction Engineer Clerk occupations were amalgamated into the RMS clerk occupation. An RMS clerk provided both HR and Financial services to clients. In October 2016, the RMS occupation was split back into the HRA and the FSA occupations.^{Footnote 51}

The RMS split affected the performance of the administrative service delivery. Tasks that would normally be completed by a single RMS clerk now require actions by both an HR and an FSA. For example, travel to a service member’s next of kin (involving home leave travel allowance), requires an HRA to first validate the location and leave and then an FSA to conclude a travel claim. When questioned to identify methods to mitigate this situation, base level personnel stated the challenge is that HRA and FSA personnel do not have the required access to each other’s systems. In interviews with senior HRA and FSA personnel, it was stated that system accesses were possible and that communication and processes would improve when the split reaches its final operating state.

An analysis of the Orderly Room processes would improve performance and redundancy. An example of this would be in the claims process. There is no requirement for an FSA to conduct all of the activities in the claims process. An HRA clerk is capable of arranging the travel and is trained in initiating travel claims.^{Footnote 52}

The split has also caused redundancy challenges, especially for small Orderly Rooms. A small Orderly Room that had one RMS clerk will now require one HRA and one FSA position. The loss of one person will now result in the loss of the HRA or FSA capability.

The FSA and HRA Military Occupational Structure Identifications (MOS ID) are short personnel. CAF identifies a MOS ID as “Critical” when the Trained Effective Strength is less than 90 percent of the Preferred Manning List number. As of the end of August 2018, the FSA status was 88 percent and the HRA status was 86.7 percent.^{Footnote 53} Interviewees stated that FSA and HRA individual training was not a factor in the personnel shortage.

CFB Borden Business Plans highlighted the lack of baseline funding analysis. This was also noted in interviews with stakeholders at CFB Borden and CFSU (Ottawa). A task to resource analysis would consolidate and coordinate the bases’ tasks and identify the resources required for base sustainment. A task analysis would aid in standardizing the tasks across the bases and could leverage and be leveraged by the common DRF activity structure noted earlier in the report. The resource component could be utilized to create baseline funding for CFSU (Ottawa) and CFB Borden, and update the baseline funding for other bases. The CAF Bases/Wings Sustainment Working Group identified above could fulfill the coordination function.

Back to Table of Contents

Annex A — Management Action Plan

Key Finding 4: Governance documentation on bases’ structure is outdated

Management Action

C Prog will update relevant sections of CFP 219 pertaining to Base/Wing governance to reflect major changes and processes to align with current policies and regulations and provide adaptability and flexibility for changing realities and missions of DND/CAF.

OPI: VCDS
Target Date: Update to be completed by April 1, 2020

Key Finding 6: The combination of a low staffing priority and the exchange of healthy base military personnel with medically restricted personnel from higher priority units has negatively affected sustainment deliverables.

Management Action

Director Defence Force Planning will conduct an analysis and submit a report to the VCDS on any recommended revision to the staffing categories for bases/wings once the FMSD study has determined the right mix of personnel requirements to adequately support base services and is assessed against the Defence Team Establishment Plan.

OPI: VCDS
Target Date: Report submitted by November 30, 2020

Key Finding 8: Coordination in the development and management of the four CAF Bases/Wings Sustainment programs will improve performance measurement.

Management Action

The VCDS Group has already initiated the establishment of a sub-Working Group. The sub-Working Group will be formalized with Terms of Reference under the existing Corporate Performance Management Working Group. The terms of reference will be scripted to remain in synch with FMSD Phase 2 work that is specifically addressing right-sized force generation and Institutional structures required to support the force employment structure, along with other important force structure drivers and considerations. The Corporate Performance Management sub-Working Group on bases/wings will be co-chaired by C Prog and ADM(RS) staff with representatives from the RCN, CA, RCAF, MILPERSCOM, and VCDS. As appropriate, other Level 1 organizations will be invited as subject matter experts relevant to the topics and/or issues being discussed (e.g., ADM (Finance); ADM(IE); ADM(IM)). The focus of the sub-Working Group will include the following objectives:

• Synchronize program logic models, as appropriate, to inform the identification of common program activities, outputs and outcomes and relevant performance indicators;
• Identify, within the program logic models, and in conjunction with FMSD Phase 2 work (2019/20), sustainment services activities common to the programs. Once established program segments are validated for FY 2020/21 and the framework is given a period of one year to stabilize, move to trial identified program activities in FY 2021/22 Level 1 Business Plans to permit a more detailed level of tracking of actual versus planned resources and results within the enterprise financial system along validated activity lines in FY 2022/23 and beyond; and
• Develop standard performance outcomes and indicators across the programs, where feasible, beginning in 2019 with the aim to have established outcomes and indicators in place for FY 2021/22. Begin tracking and monitoring progress towards achieving the stated outcomes and report FY 2021/22 results to senior leadership to inform decisions and/or to Canadians and Parliamentarians through the Departmental Results Report.

OPI: VCDS
OCI: RCN, CA, RCAF, MILPERSCOM and ADM(RS)
Target Dates:

• A formal sub-Working Group will be established with approved Terms of Reference and membership by June 30, 2019.
• Program logic models will be synchronized, as appropriate, by November 30, 2019;
• Program activities will be aligned, as appropriate, initially by November 30, 2020 for trial in the FY 2021/22 Level 1 business planning process and formally by February 2022 for FY 2022/23 tracking in the enterprise financial system.
• Draft standard program outcomes and indicators will be established by February 2020 with final approved versions in place by October 1, 2020 to begin tracking and reporting FY 2021/22 results.

Key Finding 9: Base support responsibilities have changed without conducting a task to resource analysis which has negatively affected base sustainment capabilities.

Management Action

VCDS: CFSU (Ottawa) will commence a task to resource analysis in parallel with, informed by, and subordinate to the FMSD Phase 2 work slated for 2019/20. The task will follow a four stage process. CFSU (Ottawa)’s ARA will be identified. The specific tasks required to support the ARA will then be identified. An analysis will be conducted on each task to inform FMSD Phase 2 work and lead to identifying the appropriate CFSU (Ottawa) resource requirements. The task to resource analysis report will be generated and utilized to support the CFSU (Ottawa) baseline requirements.

Informed by the FMSD Phase 2 work, the Defence Common Administrative Services Initiative will see the final personnel transferred from 18 Level 1s via Inter-Component Capability Transfer Implementation Plans by March 31, 2020. Once all personnel and/or corresponding salary have been transferred, CFSU (Ottawa) will be in a position to provide the full range of Wave 1 common administrative services to units in the National Capital Region.

OPI: VCDS
OCI: RCN, CA, RCAF, ADM(IE),MILPERSCOM, and ADM(IM)
Target Date: Report completed byMay 31, 2021

MILPERSCOM: MILPERSCOM will conduct a CFB Borden task to resource analysis in a four stage process using the Departmental Result Framework.

• Stage 1 will identify CFB Borden’s ARA.
• Stage 2 will identify the specific tasks required to support the ARA.
• Stage 3 will identify the resource requirements for each of the tasks.
• Stage 4 will generate the report required to support CFB Borden baseline requirements.

OPI: MILPERSCOM
OCI: VCDS, RCN, CA, RCAF, ADM(IE), and ADM(IM)
Target Date: Report completed byDecember 31, 2019

Back to Table of Contents

Annex B—Evaluation Methodology and Limitations

1.0 Methodology

The evaluation team used multiple lines of evidence and complementary qualitative and quantitative research methods to help ensure the reliability of information and data to support evaluation findings. In order to ensure the validity of data captured through different methodologies, a data triangulation approach was used. The methodology established a consistent approach in the collection and analysis of data to support findings, conclusions and recommendations. Based on the evidence from available sources, the evaluation reviewed the achievement of expected outcomes and the efficiency of the Base Sustainment activities within the DRF, to develop a proper baseline for the relevance and performance of the programs. Information and data were correlated to each evaluation question and corresponding indicators.

1.1 Overview of Data Collection Methods

Comparison of both qualitative and quantitative assessments was used to validate the overall analysis and to develop the evaluation’s findings and recommendations.

The following data collection methods were used to gather qualitative and quantitative data for the evaluation:

• Literature and document review;
• Key informant interviews;
• Base Commander Survey;
• Comparative research analysis with allies;
• Program and financial data reviews; and
• Site visits

1.2 Details on Data Collection Methods

1.2.1 Literature and Document Review

A preliminary document review was conducted as part of the planning phase of the evaluation to gain a foundational understanding of the Base Sustainment activities in the context of the new programs listed in the DRF. A comprehensive document review was undertaken as part of the conduct phase of the evaluation, focusing on the relevance and performance of the programs.

The following documents were reviewed during the planning and conduct phases of the evaluation:

• Government Internet and Intranet websites;
• Government documents: 2015 Speech from the Throne; 2016 Federal Budget; 2017 defence policy: Strong, Secure, Engaged; 2016 TB Policy on Results; 2016 TB Directive on Results; Initial Minister of National Defence Mandate letter;
• DND documents: Reports on Plans and Priorities; Departmental Results Report (previously Departmental Performance Reports); CAF Force Posture and Readiness;
• Program documents: Program Inventory Profiles applicable to the four programs; RCN, CA and RCAF working document; and
  
• Legal documents: National Defence Act; CFP 219 (Organization and Structure of National Defence).

1.2.2 Key Informant Interviews

Interviews were conducted in person and over the phone. Interviewees were provided with an interview guide in advance. During interviews, clarifying questions were asked. Notes were taken by the evaluators during interviews. These were later transcribed and compared for clarity and accuracy using an evidence matrix tool, designed to capture reoccurring themes in a common record.

Over the course of the evaluation study, over 75 interviews were conducted with key program stakeholders: NDHQ/RCN OPI; DRF 6.1 OPIs; ADM(IE) – RP Ops Gp; Naval Bases: CFB Halifax – Base Commander and senior section heads, CFB Esquimalt – Base Commander and senior section heads; Army Bases: CFB Gagetown – Base Commander and senior section heads, CFB Edmonton – Base Commander and senior section heads, CFB Kingston – B Comd and HQ staff, CFB Petawawa – Base Commander; Airforce Bases: CFB Trenton – Base Commander and senior section heads, CFB Greenwood – Base Commander and senior section heads; Joint Bases: CFB Borden – Base Commander and senior section heads, CFSU (Ottawa) – Base Commander and senior section heads. Some respondents were interviewed more than once for clarity.

1.2.3 Base Commanders’ Survey

B Comds are considered the main sponsors of the Base Sustainment programs. Hence, in May 2018, a bilingual (English and French) questionnaire was administered to gain insights on Base Commanders’ activities, on their assessment of the deliverables, and on the risks and challenges they meet in their day to day work. The questionnaire was distributed to all 42 B Comds contributing to the four programs. Response rates were high at 73 percent. Questionnaire results were compiled on spreadsheets for better interpretation and analysis.

1.2.4 Comparative Research with Allies

Research for information on international allies was conducted to compare various aspects of the programs such as global organization and concepts behind sustainment. Information was gathered through foreign government websites. As the present evaluation consists in establishing a baseline for new DRF programs, information captured was not used to feed the findings and recommendations in this report.

1.2.5 Program and Financial Data Reviews

Program data relative to military and civilian personnel were sourced from each participating base. IT data relative to the number of tickets being raised have been extracted from the ADM(IM) Assyst application. Financial data covering the four programs and the eight bases part of the study were reviewed to determine the expenditures of program activities; data covering FY 2013/14 to FY 2017/18 were obtained from the Base Comptrollers.

1.2.6 Site Visits

In order to develop and capture information pertaining to the context in which the programs are executed, the evaluation team visited the following locations hosting CAF bases: Borden, Halifax, Greenwood, Gagetown, Trenton, Kingston, Ottawa and Edmonton. The evaluation team was able to observe first-hand the quality of base services such as accommodations, food services, logistics, etc.

2.0 Limitations

Back to Table of Contents

Annex C—Logic Model

Back to Table of Contents

Annex D—Evaluation Matrix

Back to Table of Contents