Evaluation of Aerospace Equipment Maintenance

February 2013

1258-189 (CRS)

Reviewed by CRS in accordance with the Access to Information Act (AIA). Information UNCLASSIFIED.

Acronyms and Abbreviations

1 CAD

1 Canadian Air Division

2 CAD

2 Canadian Air Division

A4 Maint

Air Maintenance Group, Winnipeg

ADM(Mat)

Assistant Deputy Minister (Materiel)

AEM

Aerospace Equipment Maintenance

AEO

Aircraft Engineering Officer

AEPM

Aerospace Equipment Program Maintenance

AERE

Aerospace Engineer

AMC

Aerospace Maintenance Committee

AMO

Air Maintenance Organization

AMS

Air Maintenance Squadron

AVN

Aviation Technician

AVS

Avionics Technician

AWR

Additional Work Request

C Air Force

Chief of the Air Force Staff

CAS

Chief of the Air Force Staff

CDS

Chief of the Defence Staff

CF

Canadian Forces

CFB

Canadian Forces Base

CFDS

Canada First Defence Strategy

CFSD

Canadian Forces Supply Depot

Comd

Commander

CRS

Chief Review Services

DAOD

Defence Administrative Orders and Directives

DGAEPM

Director General Aerospace Equipment Program Management

DGMSSC

Director General Material Systems and Supply Chain

DM

Deputy Minister

DMPP

Directorate Materiel Policy and Procedures

DND

Department of National Defence

DPR

Departmental Performance Report

DRMIS

Defence Resource Management Information System

DRP

Distribution Resource Planning

DTAES

Directorate of Technical Airworthiness and Engineering Support

ELE

Estimated Life Expectancy

FE

Force Employment

FMS

Foreign Military Sales

FWSAR

Fixed Wing Search and Rescue

FY

Fiscal Year

GC

Government of Canada

HPR

High-Priority Request

IP

Intellectual Property

ISS

In-Service Support

ISSCF

In-Service Support Contracting Framework

LCMM

Life Cycle Materiel Manager

MAP

Management Action Plan

NATO

North Atlantic Treaty Organisation

NCO

Non-Commissioned Officer

NORAD

North American Aerospace Defence Command

NP

National Procurement

NPOC

National Procurement Oversight Committee

OEM

Original Equipment Manufacturer

Op

Operation

OWSM

Optimized Weapon System Management

OWSS

Optimized Weapon System Support

PAA

Program Alignment Architecture

PAV

Primary Air Vehicle

PBL

Performance-Based Logistics

PfM

Performance Measure (Metric)

PWGSC

Public Works and Government Services Canada

RCAF

Royal Canadian Air Force

RPP

Report on Plans and Priorities

SAMA

Senior Air Maintenance Authority

SAMEO

Senior Air Maintenance Engineering Officer

SAR

Search and Rescue

SWE

Salary Wage Envelope

SWR

Supplementary Work Request

TAT

Turn-Around Time

TLIR

Third-Line Inspection and Repair

USAF

United States Air Force

USGAO

United States Government Audit Office

VECP

Value Engineering Change Proposal

WSM

Weapon System Manager

YFR

Yearly Flying Rate

Overall Assessment

Aerospace maintenance is effective.
A lack of performance measures makes it difficult to measure the efficiency and economy of the Program.
In-Service Support Contracting Framework (ISSCF) contracts may not provide Canada with value for money in the long term.
Management of spare parts needs improvement.
Best practices should be implemented across all fleets.

Results in Brief

This report presents the results of the evaluation of the Aerospace Equipment Maintenance Program (AEMP) of the Department of National Defence (DND). The aim of this evaluation was to assess the relevance, performance, effectiveness and efficiency of the Program.

The DND spends approximately $2.05 billion annually[1] on the AEMP and is reported under Program Alignment Architecture (PAA) Section 1 (resources are acquired), Section 2 (readiness), and Section 3 (defence operations improve peace, stability and security).

The principal stakeholders for this evaluation are the Deputy Minister (DM), the Chief of the Defence Staff (CDS), the Chief of the Air Force Staff (C Air Force), the commanders (Comd) of 1 Canadian Air Division (Comd 1 CAD) and 2 Canadian Air Division (Comd 2 CAD), and the Assistant Deputy Minister (Materiel) (ADM(Mat)).

The evaluation was undertaken between June 2011 and April 2012, and examined program relevance and performance for the period April 2008 to March 2012.

Background

Royal Canadian Air Force (RCAF) aerospace equipment maintenance (AEM) was last evaluated in 1992. That evaluation resulted in the creation of the RCAF Airworthiness Program. No evaluation has occurred since; however, Chief Review Services (CRS) has conducted audits of major aircraft upgrade and modernization projects in 2007 (CP140, CC150, CH149, CF188), 2008 (CC150), 2009 (Fixed-Wing Search and Rescue (FWSAR), CP140, CF188) and 2011 (CP140).

AEM includes the activities conducted by DND to ensure that the 15 operational fleets and approximately 325 aircraft of the RCAF are ready to conduct operations, meet safety regulations, and that equipment life is maximized. Three major activities support these objectives: the supply and management of parts and services; the management of the program, personnel and the weapons systems; and the conduct of maintenance, repair and overhaul of aircraft.

Methodology

The evaluation approach used multiple lines of evidence (questions, literature reviews, document and data reviews, interviews, site visits, analysis of best practices, a review of management tools, case studies, and an administrative and financial data review) to ensure the reliability of reported results.

Findings and Recommendations

The federal government is responsible for the defence and security of Canada and assigns the principal role in this to the DND/Canadian Forces (CF). One aspect of “defence and security” is airpower, and the DND/CF aerospace equipment maintenance program is in alignment with this broad security mandate. Airpower is also in accordance with the federal treaty obligations found within North American Aerospace Defence Command (NORAD), and the federal obligation to provide a domestic air search and rescue (SAR) capability. Aerospace maintenance supports this federal responsibility to all Canadians.

Finding 1: There is a continuing need for an AEM function within the DND/CF.

The RCAF aircraft fly over 146,000 hours annually. Without an AEM program, technical air worthiness concerns and operational air worthiness risk considerations would preclude aircraft from being flown. Thus, to ensure that sufficient aircraft are available to support force employment (FE) missions, there is a continuing need for this program.

Finding 2: DND AEM is aligned with both departmental policies and priorities and with federal government roles and responsibilities.

The 2011 Departmental Report on Plans and Priorities (RPP) and Departmental Performance Report (DPR) highlight the importance of air power with respect to the ability of DND to “conduct daily continental operations and ensuring control of our airspace through ...NORAD.”[2] In addition, with respect to the Canada First Defence Strategy, each of the six core missions requires RCAF aircraft. The application of these resources is dependent upon AEM.

With respect to federal roles and responsibilities, the Government of Canada (GC) is responsible for the defence and security of Canada and assigns the majority of this role to DND. The AEM Program supports commitments to this broad security mandate, including treaty obligations in NORAD and the North Atlantic Treaty Organisation (NATO). In addition, the program supports other federal roles such as SAR within Canada and aligns with the federal responsibility of ensuring the safety of Canadian airspace (airworthiness).

Finding 3: The AEM Program operates in an effective manner.

The evaluation team found that the program is meeting its key objectives of supporting the force generation requirements of the RCAF, ensuring airworthiness and safety concerns, and maximizing the lifespan of the aircraft. However, some areas of concern were noted.

Spare parts. The provision and inventory control of spare parts could be improved. High levels of “rob rates” (taking parts from one aircraft to service another) reduce efficiency by essentially doubling the work on maintenance organizations.

Performance Contracting. The use of performance-based logistics (PBL) and associated information management is not applied uniformly. PBL could be improved.

Contract frameworks. ISSCF contracts may limit the ability of ADM(Mat) to effectively manage contractors over the long term.

Finding 4: While assessment of three fleets has demonstrated that efficiency is likely improving, a lack of performance measures and inconsistent application of best practices make it difficult to measure overall efficiency and economy of the program.

Analysis of costs associated with three fleets (CF188, CC130, CP140) demonstrates that efficiency has steadily been improving over the past five years. Maintenance costs for these three fleets have been consistent with the number of aircraft and flying hours, notwithstanding that the CF tends to keep its aircraft long past their optimum retirement dates and, as a result, costs rise substantially for declining yearly flying rates (YFR) at the end of the fleet's life.

Further, ADM(Mat) has undertaken several initiatives over the past decade to drive efficiencies and economy into the aerospace maintenance program. For instance, through participation in common maintenance programs with other operators of the same aircraft (i.e., the Global Sustainment Partnership for military users of the CC177 transport aircraft) the RCAF is applying trends from the civil aviation community. This has proven to be good value for the DND/CF through reduced inventory costs for spare parts.

ADM(Mat) has also established various forms of outsourcing of maintenance to the private sector. However, there is limited data available to confirm whether or not this is providing true value for money. Many contracts identified for incorporation into broader collective contracts remain as individual stand-alone contracts. Projected savings may not be realized in these types of programs (Optimized Weapon System Management (OWSM) or ISSCF) due to potential loss of the ability of ADM(Mat) to challenge the supplier on additional costs, particularly in issues involving intellectual property (IP).

Recommendations

1. In order to maximize value and options for maintenance approaches, ADM(Mat) should review procurement practices with respect to obtaining sufficient technical data and IP rights for new platforms and ensuring that maintenance support contracts such as in-service support (ISS) are negotiated at the time of the capital procurement. The review should give consideration to non-disclosure clauses, foreground IP, or transfers of IP after a set period of time, in order to maximize options for maintenance approaches.

OPI: ADM(Mat)

2. Performance metrics must be better understood, used more consistently, standardized where possible, and included in all major contracts. Staff must be formally trained in their use.

OPI: ADM(Mat)

3. A review of the ISSCF contract mechanisms should be conducted to ensure the ability to manage costs and performance are maximized over the long term. The review should give consideration towards splitting the ISSCF into smaller contracts (i.e., for airframe, avionics, and engine systems), rights to conduct cost audits, and consistent approaches towards value engineering, incentives, penalties, etc. This is closely linked with Recommendation 1.

OPI: ADM(Mat)

4. Best practices noted at Canadian Forces Base (CFB) Greenwood regarding periodic inspections should be examined for opportunities to apply to other fleets.

OPI: CAS

Note: Please refer to Annex A—Management Action Plan (MAP) for the management response to the CRS recommendations.

1.0 Introduction

This report presents the results of the evaluation of AEM as it is done by ADM(Mat), the Director General Aerospace Equipment Maintenance (DGAEPM) and the RCAF. The evaluation was conducted by CRS between June 2011 and April 2012, to examine program relevance and performance for the period April 2008 to March 2012, and to inform future management decisions related to AEM.

The AEM Program is found in the following areas of the PAA:

  • Section 1 – Resources are acquired to meet Government Defence Expectations
    • s.1.2.2.2 – Initial individual occupation training
    • s.1.3.3 – Aerospace equipment acquisition and disposal
    • s.1.4.1.3 – Aerospace real property acquisition and disposal
  • Section 2 – National Defence is ready to meet Government Defence Expectations
    • s.2.3.6.1 – Aerospace training
    • s.2.3.6.2 – Aerospace infrastructure maintenance and wing support
    • s.2.3.6.3 – AEM

The principal stakeholders for this evaluation were the DM, the CDS, C Air Force, the commanders of 1 CAD and 2 CAD and their staffs and subordinate wings and squadrons, the ADM(Mat), and DGAEPM.

The results of this evaluation will be used to inform the DND and CF decision-makers on the continued need for this program; its alignment with Government priorities, roles and responsibilities; and its ability to achieve expected outcomes in an effective and efficient manner.

1.1 Profile of Aerospace Equipment Maintenance

1.1.1 Background

AEM includes the activities conducted by DND to ensure that the 15 operational fleets and approximately 325 aircraft of the RCAF are ready to conduct operations, that they meet safety regulations, and that their lifespan is maximized. Three major components of aerospace maintenance support these objectives: the management and utilization of a supply chain for materiel and services; the development of the doctrine and approach taken to maintenance and the organization or personnel and resources; and the maintenance, repair and overhaul of aircraft. The program logic model at Annex B provides more detail about the activities, outputs and outcomes.

1.1.2 Program Objectives

The objective of the AEM program is to provide sufficient aircraft to aid in the generation, employment and sustainment of the air power necessary to fulfill Canada’s defence needs.

To achieve this objective there are two critical areas for consideration: readiness and airworthiness.

  • Readiness. For the RCAF, “readiness” implies that an aircraft is capable of meeting defined FE capabilities and is mission-ready (serviceable, properly configured).
  • Airworthiness. “Airworthiness” is comprised of two components: operational airworthiness (the standards of safety for air operations and aeronautical products related to flying operations) and technical airworthiness (the standards of safety relating to product design, manufacture, maintenance and materiel support).

The Aeronautics Act is the legal foundation for all Canadian aviation, with the objective of achieving safety for all civilian and military aviation activities. Under the Act, the Minister of National Defence and, under the direction of the Minister, the CDS, are responsible for any matter relating to military aviation, including those relating to personnel, aircraft, military aerodromes, or military facilities. As a statute of Canada, the Act places upon the Minister and the CDS the responsibility for the development and regulation of military aeronautics and the supervision of all matters related to military aeronautics.

1.1.3 Delivery Approach

The AEM program is managed through a partnership between ADM(Mat) and the RCAF. Within ADM(Mat) the key organization that delivers this program at the strategic level is the DGAEPM. This organization works closely with the RCAF, primarily with 1 CAD. Within 1 CAD, it maintains close contact with the A4 Maintenance (A4 Maint) staff, fleet senior air maintenance authorities (SAMA) and maintenance organizations at wings and squadrons. Other elements also play a role. This includes the Director of Flight Safety and the Director of Air Requirement in the RCAF headquarters staff in Ottawa.

Program Design – The Weapon System Construct

Within DGAEPM, each RCAF fleet has a designated weapon systems manager (WSM). The WSM designs and manages the overall maintenance program for any assigned fleets.

These fleets include the following:

  • jet fighters – the CF188 Hornet;
  • trainers – CT114 Tutor;
  • helicopters – CH124 Sea King, CH146 Griffon, CH147D Chinook and CH149 Cormorant (and the future CH148 Cyclone);
  • fixed-wing patrol and transport planes – CC115 Buffalo, CC130 Hercules (E,H, and J models), CC138 Twin Otter, CC144 Challenger, CC150 Polaris (Airbus), CC177 Globemaster, CP140 Aurora, CT142 Dash 8; and
  • Air Cadets – L19A Cessna, 182P Cessna, 8GCBC Bellanca Scout, gliders.

Aerospace maintenance is organized into three “lines” and three “levels” (see Table 1). First- and second-level maintenance is often performed by RCAF technicians, known as “blue suiters.” There is a military requirement for enough blue suiters to be able to respond to all defence requirements, including deployments overseas and in response to national emergencies. These “blue suit” technicians must be properly trained, developed, and certified.

Table 1. Definition of Aerospace Maintenance Lines and Levels. The table shows where maintenance is performed (referred to as “lines”), by whom, and the nature or “level” of maintenance work being done.

Table Summary:

This table summarizes the three lines and levels of aerospace maintenance. The left-hand column describes the lines, which are sub-divided into three stages of maintenance: first line (snags), second line (periodics) and third line (depot). For each stage, read across the row to determine who performed the repair work and what level of work was done.

Lines[3]

Performed By

Levels[4]

First Line (Snags)

Includes servicing, configuration and snag recovery of aircraft. This line is closely linked to the day-to-day flying operation and the scheduling of aircraft to specific missions.

Squadron Air Maintenance Organization (AMO), or Air Maintenance Squadron (AMS) if so configured

Level One (1st Level). Includes all servicing and corrective/preventive maintenance that can be accomplished without major disassembly of the aircraft.

Second Line (Periodics)

Includes deeper maintenance on the aircraft or specific components, but still at the tactical level (for example, periodic inspections, specialized shop support and component maintenance).

AMS, or Squadron AMO if so configured

Level Two (2nd Level). Primarily addresses aircraft or component maintenance activities that must be carried out under controlled conditions often with access to specific test equipment or facilities (shops, hangars, environmental controls). Aircraft activities that fall into this category include major preventive maintenance inspections, structural repairs and modification embodiment.

Third Line (Depot)

Where detailed, infrequent maintenance, and major structural modifications occur to CF aircraft. Third line generally refers to contractor-level activity, performed at a contractor’s facilities; hence the terms third-line inspection and repair (TLIR) or depot-level inspection and repair.

Contractor

Level Three (3rd Level). Encompasses more extensive activities such as replacement or restoration of major parts, assemblies or components, rebuilding and overhaul of equipment, mid-life improvements, life extension programs and lengthier activities that require specialized facilities beyond those normally available at a wing.

Contractors also play a significant role in the delivery of the AEM program. This includes the provision of the spare parts for blue suiters to use, or in conducting the work themselves through the provision of depot-level maintenance. A variety of other service contracts are normally let for repair and overhaul, engineering services, and testing. Contractors are now being given broader responsibilities under three different constructs: OWSM, ISSCF, and a fully contracted approach. In these approaches, contractors are assigned responsibilities across the levels of maintenance, depending on the scale and scope of the services demanded.

OWSM. Developed in the late 1990s and early 2000s, the OWSM concept sought to address the manpower shortages in DGAEPM caused by the force reduction program[5] and place RCAF contracting on a performance-based approach. The intent was to let contracts for the major sub-components of an airframe—the primary air vehicle (PAV), the engines (propulsion), and the avionics (AVS)—with DGAEPM still acting as the systems integrator. The key attributes of the OWSM program were that contracts were performance-based, outcome-focused, and incentivized. Under OWSM the contractors and the WSM must work closely together for the process to be effective.

ISSCF. The ISSF is similar to OWSM contracting and has been used recently on new acquisitions (CC177, CC130J, CH148, FWSAR). In this construct, the original equipment manufacturer (OEM) is selected to manage all aspects of ISS. The OEM then sub-contracts to other companies for the execution of the ISS. In both OWSM and ISSCF structures, CF members may still perform a substantial portion of the required first- and second-level maintenance.

Fully Contracted-out. A final approach is to contract all of the fleet ISS. This option is only presently used for the CH149. There is no involvement by DGAEPM or the RCAF in the maintenance of the airframe other than the DGAEPM and Public Works and Government Services Canada (PWGSC) staff involved in contract administration and performance management.

1.1.4 Program Spending

As shown in Figure 1, the DND/CF spends in total about $2.05 billion annually[6] on aerospace maintenance. Including RCAF maintenance units, other organizations and private sector contractors, approximately 7,800 people are directly involved in the maintenance of aircraft for the RCAF.[7]

Figure 1. RCAF Fleet Maintenance Costs. The pie chart shows the annual amount of money for each of the six major areas where the RCAF spends money on aircraft maintenance. The data is shown in Table 2.

Table 2. Fleet Maintenance Cost by Major Item. This table lists the cost in dollars of the six major items which comprise aircraft equipment maintenance in the RCAF.

Table Summary:

Rows list the six major items which comprise aircraft equipment maintenance in the RCAF. The second column lists the cost for each item.

Item Cost

Spares and R&O

$1,000,000,000

Betterments

$294,000,000

Real Property

$140,000,000

2 Division Training

$11,000,000

Pay and SWE

$452,000,000

Force Generation

$155,000,000

National Procurement, Spares, and Repair and Overhaul

Through the national procurement (NP) budget, DGAEPM receives varying amounts, from $800 million to more than $1 billion annually. The fiscal year (FY) 2012/13 NP allocation letter gives the following allocations to DGAEPM:[8]

  • FY 2012/13 – $844 million
  • FY 2013/14 – $988 million
  • FY 2014/15 – $1,117 million

Military Pay and Civilian SWE

For FY 2011/12, DND spent $452 million for military pay and benefits and civilian SWE on aircraft maintainers. DGAEPM has 1,059 civilian and military positions, and 210 on-site contractors, all providing corporate, managerial, technical, administrative and operational support[9] to aerospace maintenance NP. The RCAF has 4,751 personnel dedicated to AEM.[10]

Betterments

Betterment, defined as projects that extended the estimated life expectancy of an aircraft or which deliver a new or enhanced operational capability.

Force Generation

1 CAD spent $155 million in aerospace maintenance during FY 2011/12 for force generation.[11]

Training

2 CAD spent $11 million in FY 2011/12 for the training of aerospace engineers (AERE), aviation technicians (AVN), and AVS military occupations.

Real Property

The Assistant Deputy Minister (Infrastructure and Environment) spent $281 million in FY 2011/12 on the maintenance and repair, and new construction of real property for the aerospace maintenance function of the RCAF. This amount also covered the costs of construction in support of equipment. Approximately $140 million of that amount is attributable to real property in support of aerospace maintenance.

1.2 Methodology

1.2.1 Evaluation Objective and Scope

The evaluation followed the scope and methodology set out in an evaluation work plan during the planning phase completed prior to the commencement of the evaluation. The evaluation work plan was designed to align with the Treasury Board Policy on Evaluation (April 2009).

Preparation of the evaluation questions and identification of relevant indicators was performed in conjunction with DGAEPM and the RCAF to ensure technical accuracy and that the needs and concerns of all stakeholders were met. The evaluation team provided an outline of the evaluation approach to DGAEPM and the RCAF for confirmation and approval.

This evaluation focused on the relevance, performance, effectiveness and efficiency of AEM as it is conducted by the RCAF, and on behalf of the RCAF by private-sector contractors.

In particular, the evaluation considered the following:

  • The strategic management of the aerospace maintenance portion of national procurement funding. This included, but was not limited to, issues such as requirements definition, demand estimation, planning committees, allocation, management, expenditure budgeting and financial management, spares, airworthiness, performance measures and performance monitoring, contract oversight, training, and human resources.
  • The selection of the maintenance approach to be used for an airframe once in service.
  • The maintenance of the CF188 and CC130H fleets under the OWSM approach.
  • The maintenance of the CC177 and CC130J fleets under the ISSCF approach.
  • The maintenance of the CH149 fleet under a completely contracted-out approach.

Although aircraft maintenance is performed by both the military and civilian sectors, no comparison was made with the cost of aerospace maintenance in Canada’s civilian sector as the operating conditions, composition of fleets, and the required frequency and intensity of maintenance on a civilian aircraft is far different from that on a military aircraft.

1.2.2 Evaluation Issues and Questions

The evaluation of AEM examined issues related to relevance and performance (see Annex C for the complete evaluation matrix, which also includes specific indicators and methodologies for each evaluation question).

Evaluation Questions

Relevance

  • Does AEPM continue to address a demonstrable need best filled by the RCAF?
  • Is AEPM aligned with federal roles and responsibilities?
  • Is AEPM aligned with departmental roles and responsibilities?

Effectiveness

  • Is quality aircraft maintenance being performed?
  • Is there a sufficient, professional and sustainable workforce?
  • Are services and materiel available to support maintenance activities?
  • Is there an effective program structure?
  • Are there incentives for industry to improve its contribution?
  • Are appropriate performance measures in place?
  • Is planned YFR achieved?
  • Are aircraft maintained to achieve maximum service life?
  • Are strategic industrial capabilities sustained?
  • Is the Canada First Defence Strategy (CFDS) mission being met?
  • Is there evidence of unintended outcomes (positive/negative)?

Performance (Efficiency and Economy)

  • Are material and services acquired in a manner that delivers best value for the program?
  • Does the supply chain function in an efficient manner?
  • Is there an efficient use of maintenance personnel?
  • Does the management structure and governance drive best practices in efficiency?

To evaluate relevance and performance properly requires the collection of both qualitative and quantitative data. To meet this need, the evaluation was guided by an evaluation matrix and considered data collected during a literature review, a document and data review, the conduct of interviews, visits to DND/CF units and private sector contractors, an examination of best practices, and an analysis of contracting approaches.

1.2.3 Data Collection Methods

Although the evaluation relied on more than one line of evidence, surveys and focus groups were not used since the qualitative aspects of the evaluation were more than satisfied through the use of interviews. Quantitative evidence in the form of reports, analysis and RCAF documentation was used to support any interview perspectives.

The sample size of the interviews in the DND/CF was adequate and represented a very broad point of view from the WSMs, AERE officers, and technical trades. The interviews with the private sector covered a broad base of technical competence and expertise, as well as a representative geographical sampling.

1.2.3.1 Literature Review

Literature reviews were conducted to capture best practices for various practices that support the management of AEM. This included various types of maintenance theory, performance-based contracting, organizational management practices and theories, and applicable world-wide aviation practices. Comparable organizations in the United States, Australia and Britain were studied to enable benchmarking and identification of best practices.

1.2.3.2 Document and Data Review

Documents were reviewed to establish the parameters of what the program delivers and to determine underlying data that supported findings of economy and efficiency. For scoping this evaluation the following documents were part of the preliminary review: Air Force strategic documents and doctrine manuals, the “P” series of manuals, some MAPs, the annual Total Air Resource Management documents, proceedings of the National Procurement Oversight Committee (NPOC), proceedings of the AMC and proceedings of the DGAEPM quarterly performance review. As the evaluation progressed, the scope of documents that needed to be reviewed became more focused and the following were reviewed: all applicable MAPs, appropriate aircraft technical manuals, Air Force 9000+ documentation, maintenance records, scheduling practices, maintenance contracts for the CC130H Hercules, the CF188 Hornet, the CC177, the CC130J, and the CH149 (and their deliverables), air worthiness instructions and audit reports.

1.2.3.3 On-site Examinations

Visits were required for data collection and review. The evaluation team conducted interviews with staff from DGAEPM. As well, a visit to 1 CAD Headquarters was made to examine how aerospace maintenance is coordinated there and to assess the goals and objectives of their audit function.[12] Staff at 2 CAD Headquarters were interviewed for their views on aerospace maintenance training. AMSs at 8 Wing (Trenton), 4 Wing (Cold Lake) and 14 Wing (Greenwood) and a number of squadron maintenance flights were visited. During these visits, interviews were conducted with key staffs and contractors, MAPs and organizational structures were reviewed, and observations were made on local procedures. Contractor sites were visited to examine how their procedures match those of in-service organizations. The team visited 25 Canadian Forces Supply Depot (CFSD) in Montreal to examine the warehousing of aerospace parts.

1.2.3.4 Best Practices

Best practices were captured from wherever they were discovered: internal to the DND/CF, academia, industry or allies (United States, United Kingdom, Australia). They were examined to determine their potential for application in the DND/CF AEM program.

1.2.3.5 Management Tools

This evaluation examined the management practices used in the delivery of AEM. This included the use of the balanced score card methodology, business plans, performance metrics, the strategic processes used to secure national procurement funding, the standardized practices and deviation in the management of individual weapon systems, and the expected utility of the three forms of contracting: OWSM, ISSCF, and totally outsourced.

1.2.4 Limitations

The use of interviews may sometimes lead to bias in the results. However, for this evaluation, interviews from many different functional areas of the DND/CF, as well as from a broad geographical area, were conducted. Consequently, the aspect of bias which may arise from interviews was minimized.

The evaluation was also limited as follows:

  • As each airframe has unique parts, avionics and structure requiring a specific maintenance program that combines the necessary management, organizations and resources, it was not possible in the time available to examine in detail the fleet practices and policies for all fleets in the RCAF’s inventory. Therefore, emphasis was placed on key fleets that illustrate the various contracting approaches.
  • The bulk and scope of technical manuals within the Air Force precluded any detailed review of their individual quality or status.
  • Sustainment aspects for deployed operations exterior to Canada were not considered, other than how these operations affected the need for in-service maintainers as stated in the scope.
  • The evaluation did not consider the maintenance of ground support equipment nor ammunition issues.

2.0 Evaluation Findings

2.1 Relevance

The evaluation asked three questions to determine if the AEM is relevant:

  • Does AEPM continue to address a demonstrable need best filled and managed by the RCAF?
  • Is AEPM aligned with departmental roles and responsibilities?
  • Is AEPM aligned with federal roles and responsibilities?

2.1.1 Demonstrable Need for Program

Finding #1. There is a continuing need for an AEM function within the DND/CF.

Aerospace maintenance is conducted to ensure that the CF has sufficient aircraft available for force generation. This includes ensuring that aircraft are safe to fly (airworthy), reliable, and available both at present and well into the future by maximizing their lifespan. Further there is a legislative requirement (Aeronautics Act)[13] to provide air maintenance on all aircraft based upon set schedules determined by both calendar time and hours flown.

Maintenance needs are demonstrated by the number of hours flown each year by CF aircraft. On average, over the past five years, over 325 aircraft have flown 145,987 hours annually.[14] For FY 2012/13 this is projected to continue, ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ As new aircraft fleets have recently arrived (C177, C130J, CH148) and with more to follow over the coming five years, the operational hours flown will likely rise.[15]

2.1.2 Alignment with Departmental Priorities

Finding #2. The DND/CF role in securing and providing AEM is consistent with departmental policies and priorities.

The (2012) RPP states that the DND/CF will conduct “NORAD operations to ensure the continued security of our sea and airspace.”[16] The (2011) DPR stated that air power was “conducting daily continental operations and ensuring control of our airspace through NORAD.”[17]

The CFDS released by the government in 2008 represents the centerpiece of DND defence policy and priorities. Within CFDS, the GC assigned six core missions to the CF. Each of the core missions requires a different mixture of RCAF resources to aid in their accomplishment.

For example, conducting daily domestic and continental operations requires as a minimum the CF188, the CH124, the CH149 and the CP140 whereas deploying forces in response to crises elsewhere in the world requires the CC177 and the CC130. Responding to the terrorist attack of 9/11 and the mission in Libya required the CF188. Support to Canada’s mission in Haiti required the CC177 and CC130. Executing any of these missions requires available aircraft. Thus, this program is consistent with and supports the execution of DND priorities.[18]

2.1.3 Alignment with Federal Roles and Responsibilities

Finding #3. The manner in which DND acquires and delivers AEM is in alignment with federal government roles and responsibilities.

As per the National Defence Act, the federal government is responsible for the defence and security of Canada and assigns the principal role in this to the DND/CF. The DND/CF uses airpower dependent upon the AEM program in alignment with this broad security mandate and in accordance with the federal treaty obligations found within NORAD.[19]

The National SAR Program guides the collective activities of both the federal government in the federally mandated areas of Canada's SAR system, and the activities of provinces, territories and municipalities having either primary or secondary roles and responsibilities with respect to the provision of SAR services.[20] In addition to the entire landmass of Canada, the federal government has accepted the responsibility for providing aeronautical SAR services over certain portions of the surrounding oceans. This responsibility is administered by the National SAR Secretariat. Air-based SAR is executed by the DND/CF. Response to marine and ground SAR incidents occurring within national parks is also within the federal mandate. Note that response to all other ground SAR and inland waters marine SAR incidents falls within the mandate of provincial/territorial authorities.[21]

The federal government is responsible for the safety of Canadian aerospace.[22] It establishes the regulations for this through the Aeronautics Act. The Act applies equally to civil and military aviation. In response to the requirements of the Act, the DND/CF has established a formal air worthiness program which includes a technical air worthiness authority, an operational air worthiness authority and a flight safety program. Air worthiness requirements permeate all aspects of the DND/CF AEM function. It thus aligns with the federal responsibility of ensuring the safety of Canadian airspace.

2.2 Performance (Effectiveness)

The following section provides the findings with respect to how effectively the CF AEM program has achieved its intended results (outcomes) over the course of the evaluation period.

For this evaluation, only immediate and intermediate outcomes were assessed as the long-term outcomes are heavily influenced by numerous activities and factors external to this evaluation.

The immediate outcomes of the Aerospace Maintenance Program were determined to be as follows:

  • Immediate Outcome 1: High-Quality Maintenance
  • Immediate Outcome 2: Establishment of a Supply Chain
  • Immediate Outcome 3: Establishment of an effective Program Structure

2.2.1 Immediate Outcome 1: High-Quality Maintenance

Finding #4. ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

Finding #5. The RCAF and ADM(Mat) must consider the long-term effects on their capability to maintain fleets where blue suit maintainers are removed from the second-level maintenance functions. This may affect the development of comprehensive maintenance programs, dilute the challenge function when services are contracted out, and limit innovation.

Based on interviews and an examination of documentation, the evaluation team noted that the requirement to ensure that there is a professional and sustainable workforce is recognized by the CF and that oversight in this area is improving.23 Every maintenance procedure can be reduced to a number of tasks, each of which takes a certain amount of time to perform.24 The cumulative time it takes to carry out a maintenance procedure, if all parts and resources are available to complete it, depends not only on the number of tasks necessary to complete it but additionally on the experience of the technicians performing them.25

2.2.1.1 Experience of Technicians

Although it was noted that there is a strong commitment to providing quality technicians, and that their training and development programs are effective, staff at Cold Lake, Winnipeg, Trenton and Greenwood indicated that ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 26 There is a link between the experience of technicians and the amount of time on task they require to complete work.27 Generally, less-experienced technicians take longer to complete tasks than more experienced ones.28

The evaluation found that ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 29 ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 30 The RCAF is now devoting more attention to the production and training of its apprentice and journeymen technicians.31 Each fleet has a fleet employment and training plan to aid in the management and development of its technicians.32 Most SAMAs are well served where they appoint a wing employment and training officer to monitor the training, certification and status of its assigned personnel.33

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 34 ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ Although contractors (many former CF technicians) have in the past been hired to support older fleets, such personnel would not be capable for newer aircraft due to lack of experience on the model.

The CF employment policy whereby personnel are posted every few years is also a contributing factor to the issue of minimal experience and lack of experience on task.35 Having a stable workforce which has had the opportunity to work on the same fleet for an extended period of time allows for the creation of an intimate level of knowledge, understanding, and team building.36

This lack of experience, where many maintainers are not yet at the level where they can perform maintenance efficiently (what would normally take an experienced technician one hour can take an inexperienced one double or more time) or effectively (working on their own, or signing off on their work, or the work of others), makes preventative maintenance and corrective maintenance less effective than it could be.37

2.2.1.2 Loss of Expertise

Even given time, AVS/AVN technicians may not acquire the necessary experience if the DND/CF no longer participates in periodic inspections.38 In the past, the RCAF would employ its own personnel in the second-level maintenance.39 The experience gained in the second level allows the maintainer to have a greatly increased appreciation of the situation when faced with a problem at the first line, especially when troubleshooting a fault.40

With the onset of the OWSM and ISSCF contracting frameworks, the use of blue suits in 2nd and 3rd line maintenance has diminished or ceased altogether.41 In some fleets the blue suit maintainers now get little experience at the 2nd level.42 This may limit the RCAF and ADM(Mat)’s ability to be “smart customers” when working with the contractor.43 Blue suit technicians with sufficient experience on the aircraft are necessary to provide the advice required to aid in the development of effective maintenance programs.44 Additionally, without sufficient technical competence the ability to challenge the contractor regarding additional charges and verify work performed will diminish.45 Further, the lack of technical expertise hampers innovation.46 A thorough understanding of the parts and system of an aircraft are often necessary to both troubleshoot and identify potential non-standard repairs.47

2.2.2 Immediate Outcome 2: An effective Supply Chain

Finding #6. The management of spare parts has been an issue for many years and has been reported previously by CRS Audits,[48] the Auditor General,[49] and the Public Accounts Committee.[50] Concerns exist with parts availability, storage and disposal, and the effectiveness of the inventory control system. ADM(Mat) has acknowledged these issues and has prepared appropriate MAPs.

Finding #7. Several OEMs, particularly those that promote ISSCF-like contracts, limit access to IP rights. Given the relative newness of the CC177 and the CC130J contracts, it is too soon to determine whether lack of access to IP will cause delays. However, one of the CC130J subcontractors did indicate difficulties in obtaining IP data out of the OEM. In the case of the CH149, the failure to acquire IP has been costly both in terms of money and time.

Every part on an aircraft has a documented, predicted life, and overall the fleet has an estimated life expectancy (ELE).51 This ELE is determined by a variety of factors such as the number of flying hours, elapsed (calendar) time, thermal and/or pressure cycles, or fatigue limits.52 At some point in time, as its ELE is neared or surpassed, the part will be inspected, overhauled or replaced.53

The competent management of spare parts is therefore vital to all aspects of AEM. This includes a reasonably accurate estimation of usage rates, determining where parts will be held and the necessary stock levels at those locations, ensuring airworthiness documentation accompanies parts wherever they might be, and organizing the procurement, repair, overhaul, and disposal of parts.54

Metrics to determine how well parts are supplied are difficult to identify given that there is little similarity between the systems involved in their procurement, holding and delivery.55 The ADM(Mat) materiel management system does not appear to have timely metrics to make it a useful tool, nor does it easily allow for tracking of spare parts.56

The complete life cycle management of spare parts needs a thorough review to address issues of timeliness and obsolete stock.57 Those interviewed reported that the management of spare parts has been an issue for many years and has been reported as an issue more than once. In an effort to address life cycle management of spare parts in a systematic way, ADM(Mat) has an initiative called Distribution Resource Planning (DRP), the purpose of which is to provide visibility and performance measurements of stock.58

For example, planning tools for spare parts estimation varied greatly between the fleets.59 The Logistics Management Planning Tool used by the CF188 fleet was reported as useful but has not been adopted by other fleets.60

Another concern noted was an inadequate attention given to disposing of surplus, redundant and obsolete stocks. These clog up the supply chain and impede its effectiveness.61 Proper storage instructions need to be issued for high value items and WSMs should regularly ensure that their stocks are being adequately housed.62 Greater attention is required in the management of expensive items with limited shelf lives to ensure that only the minimum optimal number is held. There is a need to continually monitor the disposition of stocks they are assigned and ensure oversight of the overall stock of their fleet.

2.2.2.1 Intellectual Property

IP refers to creations of the mind where exclusive rights are recognized under law.63 Under IP law, owners are granted exclusive rights to tangible and intangible assets such as copyrights, trademarks, patents, industrial design rights and trade secrets. In the AEM context, the OEM's IP represents the knowledge gained from their research, development and manufacturing, and knowledge gained from their industrial designs.

Having the authority to use IP is crucial in supervising, monitoring, conducting or securing AEM services.64 An OEM may have several sources of information to which the DND/CF may need access: technical data (manufacturing knowledge and industrial design); background IP (research and development knowledge); and foreground IP (IP created under the contract, some of which may be distributed to Canadian industry to maintain critical defence industry capabilities).65

Further, due to airworthiness requirements AEM activities are highly prescriptive; each part used must be designed and produced to exacting standards and then certified as airworthy. Thus access to IP can have an enormous bearing on the efficiency and effectiveness for any contractor providing materiel or services to the DND/CF and for the DND/CF to carry out independent analysis and verification.

Concerns arise when IP rights are not obtained. When Canada purchased the CH149, the government declined the opportunity to acquire IP from the OEM at a cost of $8 million.66 The RCAF then fully contracted out the fleet maintenance to a third-party contractor. By not securing the IP rights, this contractor was found to be impeded in performing engineering services, and as such parts which may have been produced locally had to be purchased at greater expense exclusively through Augusta Westland.67 In addition, the RCAF had no means to conduct independent analysis and verification.68 Similarly, with another fleet, the evaluation found that recently one of the CC130J subcontractors indicated it had difficulties in obtaining IP data from the OEM.69

Recommendation

1. In order to maximize value and options for maintenance approaches, ADM(Mat) should review procurement practices with respect to obtaining sufficient technical data and IP rights for new platforms and ensuring that maintenance support contracts such as ISS are negotiated at the time of the capital procurement. The review should give consideration to non-disclosure clauses, foreground IP, or transfers of IP after a set period of time, in order to maximize options for maintenance approaches.

OPI: ADM(Mat)

2.2.3 Immediate Outcome 3: Effective Program Structure

To assess the effectiveness of the RCAF aerospace maintenance program structure, the evaluation assessed whether ADM(Mat) was using best practices found within the defence aerospace industry.70 These practices include the following:

  • outsourcing to contractors who specialize in fleet management;
  • incentivizing contractors to maximize state-of-the-art approaches; and
  • use of value engineering.

2.2.3.1 Outsourcing

For much of the RCAF fleet, parts management has been turned over to contractors in the form of either OWSM or ISSCF contracts. The evaluation found that for these types of arrangements the management of spare parts was generally effective; however, some of the results were mixed.

For instance, the CF188 AVS OWSM contractor, Harris Canada, demonstrated effective planning and allocation of resources through a tracking system which monitors maintenance activities, failure rates, and component utilization. This allows them to effectively track trends enabling an improved understanding of present and future spare requirements. The results have been positive with increased availability (reported 30 percent), and reduced turn-around time (TAT) of spares, compared to when it was directly managed by the CF. Overall improvement in visibility of usage trends has also contributed to setting minimum/maximum levels accurately.

Under the ISSCF contracts for the CC130J and the CC177, the contractor manages the parts inventory. For the CC130J the number of parts to be held by the contractor was negotiated within the contract.

For the CC177 Canada obtains parts under the Foreign Military Sales (FMS) from a common pool and pays a fixed yearly amount in order to have access to that pool. This arrangement appears to be working well.71

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 72 ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 73 ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 74

Bell Helicopter Textron demonstrated effective management of sparing requirements through effective parts predictability tools, and monitoring of sub-contractors and strategies to reduce robs. This has contributed to increased availability of parts and improved turn-around time on parts requests.75

2.2.3.2 Incentivizing Industry to Improve its Contribution

Finding #8. Incentivizing industry is largely dependent upon establishing sound performance measures. However, the evaluation found the usage to be limited within ADM(Mat) and varied significantly from contract team to contract team.

Finding #9. ADM(Mat) does not have a formal manual on performance metrics. Although ADM(Mat) has issued an Optimized Weapon System Support (OWSS)Contract Performance Management Framework and Guide (30 June 2010), this document postdates the two ISSCF contracts evaluated. Performance Measure (metric) (PfM) were constrained by the FMS structure for the CC177 but it is unclear why the Guide was not used for the CC130J.

Finding #10. Prior experience and lessons learned do not seem to be taken into consideration when developing PfMs for ISSCF contracts.

One of the key aspects to incentivizing industry to improve effectiveness is through the use of PBL.76 PBL represents a dramatic shift from previous contracting methodologies. Previously, fleet operators sought to procure each individual aircraft part, and service contracts minutely specified the desired product—a nut, a panel, or an engine. Under PBL the requirement is not purchasing a specific item but an item that can fill a specific function. Within Canada, many of the major contracts being let or managed by ADM(Mat) are predicated on PBL. For example, this concept underpins both OWSM and ISSCF.77

For a PBL contract to be managed effectively, performance metrics are required to accurately measure if the contractor is delivering the performance that has been purchased. The document and literature review noted that our key allies use performance metrics in air force contracting. The United States Air Force (USAF) has standardized metrics. One of the first USAF publications on this issue was their 1991 Metrics Handbook. The Air Force Logistic Management Agency published Maintenance Metrics U.S. Air Force, a maintenance-specific metrics handbook, in 2001, and a revised version was distributed in 2009. The Royal Australian Air Force has also devoted considerable resources to the development of performance-based contracting. Many of their metrics for air force contracting are contained in the Defence Materiel Organisation’s Performance Based Contracting Handbook: Guiding Principles and Performance Framework, Version 2.0, February 2007. The UK's Royal Air Force makes use of performance metrics in managing its fleets.

ADM(Mat) uses performance metrics in OWSM and ISSCF contracting. As shown in Table 3, the number of PfMs varies between contracts. A review of such contracts found that they are inconsistent when describing the review and management process to be used to assess whether performance metrics have been met. When present, this oversight assures value for money by confirming that penalties or awards are appropriately given.

Table 3. Performance Measures – RCAF Contracts. This table lists the contract type, the number of performance measures for that contract type, what type of committee structure manages the contract, and whether or not incentive or penalty measures exist in the contract.

Table Summary:

Five different types of aircraft are listed in the left-hand row. Select the aircraft type and read across the row to determine the contract type, the number of performance measures for that contract type, what type of committee structure manages the contract, and whether or not incentive or disincentive measures exist in the contract.

Air Frame

Contract Type

Number of Performance Measure Metrics

Committee

Incentive

Penalty

CF188

OWSM AVS

3

Yes

Yes

No

CC130H

OWSM PAV

14

Yes

Yes

Yes

CC130J

ISSCF

3

No

No

Yes

CC177

ISSCF

7

USAF-led

Yes

Yes

CH149

Outsourced

9

Yes

No

Yes

A review of the contracts for the CF188, CC130H and CH149 found that the role and function of the review process and the oversight committee were described clearly. For the CC130J, while a number of committees are described, the evaluation found that specific responsibility for calculating and approving the performance measure adjustments was less evident.78 For the CC177 only select PfMs out of the total for the overall Global Sustainment Partnership are applicable to Canadian usage.79 Canadian data is rolled up as part of the overall Partnership data to determine contractor incentives and disincentives. Several programs are based on weighted composite scores that are tabulated bi-annually. Weighting may be re-assigned to PfMs annually and be reflected in the annual operating plans.

An underlying concept of PBL contracting is to create incentives for suppliers to be more effective and efficient.80 Theoretically, negative incentives cause the contractor to ensure they meet the standards required and positive incentives encourage them to exceed those set standards. The evaluation team examined contract data and noted that not all contained both aspects of this process. First, in the case of the CC130J contract the incentives are purely negative. Second, certain PfMs will always be heavily weighted. For example, due to the SAR requirement for the CH149, “aircraft availability” as a PfM is set very high.81 This makes the ability of the contractor to achieve a “higher” standard exceedingly difficult rendering the incentives on this contract primarily negative in nature. It is doubtful that a solely negative structure incentivizes a contractor.

Interviews (n=14) were conducted with WSMs and contractors to determine the extent PfM are used within the current program for incentivizing contractors to apply more effective measures in maintenance. The interviews found that these measures are not used effectively as only a few (2 of 14) of the contract development staff had received training in PfMs. Some contractors believed that certain PfMs added overhead to their cost of business for little meaningful gain to the Crown.

When WSMs or their staffs were questioned (n=9) as to how they developed PfMs, most respondents indicated that they were self-taught. Most (7) noted that they made very little effort to consult with those developing PfMs for other fleets. For instance, the CH148 SAR fleet staff did not look at the CH149 fleet (an aircraft solely used in SAR) when developing the metrics for CH148s. While metrics for the CC130J were developed after consultation with those who had developed PfMs for the CC130H, the metrics are dissimilar. Metrics for ISSCF contracts, for the most part, seem to be developed in isolation. In several interviews (n=9, senior military officers and non-commissioned officers (NCOs)), when staffs were queried as to whether they knew of the existence of the American manual, few (two) indicated that they were aware of it. Some divergence in metrics may be attributed to the fact that they were negotiated with each individual contractor. Though most Performance Review Boards are chaired at the WSM level, it is unclear whether fleet-contractor performance is easily comparable across fleets.

2.2.3.3 Other Fleets

Performance metrics are being included in other RCAF contracts. They are contained in the CH146 OWSS contract where they closely match those recommended in the Guide. They are also in the CH148 and the CP140 contracts.

Recommendation

2. Performance metrics must be better understood, used more consistently, standardized where possible, and included in all major contracts. Staff must be formally trained in their use.

OPI: ADM(Mat)

2.2.3.4 Value Engineering

Value engineering can be described as an organized effort directed at analyzing the functions of systems, equipment, facilities, services, and supplies for the purposes of achieving the same standards of required performance, reliability, quality and safety but at a lower life-cycle cost.82 It typically increases some combination of performance, reliability, quality, safety, durability, effectiveness, or other desirable characteristic.83 Generally, savings from a Value Engineering Change Proposal (VECP) are split 50-50 between the contractor and the government.84

Value engineering is used extensively in federal government contracting in the United States.85 It is estimated that for the USAF alone more than a billion dollars in savings and cost avoidance are generated annually.86 It is used extensively in the United States Government including the military. The Office of the Under Secretary of Defense for Acquisition, Technology and Logistics has published the Contractor’s Guide to Value Engineering (May 2006) in order to make contractors aware of the procedures to be followed to participate in value engineering programs.

Within Canada, the concept of value engineering is not emphasized in GC contracting policies, documents or manuals.87 Within DND itself it is only briefly mentioned.88 Value engineering does appear, however, in larger ADM(Mat) contracts.89 The sections of the contracts describing the concept vary considerably in depth but all include a general definition, a cost-sharing formula and mechanism, and instruction on how a VECP proposal will be submitted and addressed.

Through an examination of contracting documents, the evaluation team determined that value engineering has seen limited use in the current major contracts of ADM(Mat):

  • CF188—two proposals were submitted and approved under the CF188 AVS OWSM. The first was expected to achieve an annual savings of $156,000 and the second $100,000.
  • CC130—there were submissions under the CC130H PAV OWSM but the contracting authority did not process them. Provisions for value engineering have not been included in the CC130J contract.
  • CC177—value engineering as a formal clause does not exist on the CC177 contract. Even so, the USAF has a contract which is expected to recoup over $12.9 billion in projected savings over its 10-year life,90 in which Canada will receive a share.
  • CH149—value engineering is included as cost of ownership incentives but these provisions have not been exercised.

Other fleet sub-contracts—value engineering appears in the T56 Engine Support Systems (2002), the Repair and Overhaul Twin Otter Engines (2005) and it is included in contracts for the CP140, CH146 and CH148.

2.2.4 Intermediate Outcomes

Intermediate outcomes assessed by the evaluation include the following:

  • sufficient number of mission-ready aircraft are available when needed;
  • aircraft are flown to their maximum lifespan; and
  • support and sustainment of a strategic industrial capacity.

2.2.4.1 Sufficient Numbers of Mission-Ready Aircraft are Available when Needed

Finding #11. Sufficient aircraft are being kept serviceable and mission-ready to allow the RCAF to accomplish its force generation and FE requirements.

Examination of ADM(Mat) records demonstrates that fleet YFR[91] requirements are being met[92] (see Figure 2). This was further reinforced by interviews (n=33) with the operational staff in 1 CAD, Winnipeg, Cold Lake (CF188), Trenton (CC130) and Greenwood (CP140 and CH149), who confirmed that sufficient aircraft are kept serviceable and mission ready to allow the RCAF to accomplish its projected force generation and FE requirements. The availability of aircraft is monitored and managed by operations staff at the squadron, wing and 1 CAD levels. Maintenance staff are responsive to forecasted demands and priorities.

Where necessary, such as the CF188 operations in Libya, additional resources are committed to maintenance programs to ensure aircraft are maintained to the required levels. Air worthiness is monitored and maintained through audits, inspections, certifications, and standardization. Services and materiel to support maintenance activities are made available.

Figure 2. Target YFR versus YFR Flown – FY 2007/08 to FY 2011/12. This figure shows the actual YFR and compares it to the target YFR for those aircraft maintained by DGAEPM. ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

Table 4. Target YFR vs YFR Flown.

Table Summary:

Rows list hours of target YFR and hours of YFR flown. Columns list year.

Target YFR is determined by the RCAF in consultation with 1 CAD and is the amount of hours the RCAF estimates it needs to fly to meet its mission. Target YFR is then used by DGAEPM to determine the amount of NP money required. If the NP allocation cannot fund the Target YFR, or if there is a reduced number of aircraft available, the WSM will identify a supportable YFR.

YFR Flown is influenced by a number of factors which have no linkage to the NP funds allocated, such as aircrew availability, number of trained technicians, and weather. Providing more NP funds to certain fleets may not have any impact (or increase) on the YFR Flown.

2.2.4.2 Aircraft are Flown to their Maximum Lifespan

Finding #12. The aerospace maintenance performed by the RCAF allows it to fly its aircraft to their maximum lifespan.

The aerospace maintenance program is successful in extending the life-expectancy of its aircraft fleets (see Table 5). The RCAF tends to keep its aircraft flying for long periods of time, in excess of most other air forces. For example, the average number of flying hours for the legacy Hercules fleet (E models) ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ [93] The USAF, United States Air National Guard and USAF Reserve units retire their Hercules aircraft at an average age of 19,800 flying hours acquired over a period of 28 years.[94]

Table 5. Number of Aircraft and Average Age in Service. This table shows the RCAF fleet managed by DGAEPM, the number of aircraft purchased and now remaining, the average age of the remaining aircraft, the date of entry into service, the total number of flying hours for the fleet, and the average yearly flying rate for the fleet.

Legend:

N/A – Not Applicable

Table Summary:

Five aircraft types are listed in the left-hand column. Select the aircraft type and read across the row to determine the number of aircraft purchased, the number now remaining, the average age of the remaining aircraft, the date of entry into service, the total number of flying hours for the fleet, and the average yearly flying rate for the fleet.

Aircraft Fleet

Number of Aircraft Bought

Number of Aircraft Remaining

Average Age of Current Fleet (Years)

Date of Entry in Service

Total Air Force Hours – Fleet

Average YFR

CF188 Hornet

138

77

27

1982-1988

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

CT114 Tutor

190

25

47

1964

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

CC130H Hercules

16

13

26

1974-1975,
1985-1986,
1991, 1997

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

CC146 Griffon

100

84

17

1994-1997

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

CC150 Polaris

5

5

19

1993-1994

▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓

CC115 Buffalo

15

6

45

1967

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

CC138
Twin Otter

9

4

42

1971

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

CC177 Globemaster

4

4

4

2007-2008

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

CP140 Aurora

18

17

31

1980-1981

▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓

CH124
Sea King

41

28

49

1963

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

CH149 Cormorant

15

14

10

2000-2003

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

In addition to the Hercules, the CF188 fleet was supposed to be retired in the early 2000s after a life of 25 years and now has an ELE of 2020. The CH124 Sea King and CT114 Tutor have received several extensions to their life expectancy.

2.2.4.3 Support and Sustainment of a Strategic Industrial Capability

Canada's aerospace, defence, space and security industries are major contributors to the Canadian economy.95 The aerospace industry alone is made up of more than 400 firms across the country and employs approximately 80,000 Canadians.96 Canada's defence and security industries employ more than 70,000 Canadians and generate $10 billion in annual revenues.97 Although Canada no longer manufactures military aircraft, the many hundreds of military aerospace maintenance contracts awarded in Canada have continued to support a viable strategic industrial capability for the maintenance of military aircraft.

Current practices may be impacting on this capacity. Data analysis shows that within OWSM-type contracts, the majority of suppliers of aerospace maintenance remained as Canadian contractors. However, the ISSCF contracts are with the OEM, which so far have been American companies (Lockheed Martin (CC130J), Boeing (CC177, CH147), and Sikorsky (CH148)). ISSCF-type contracts allow the OEM to decide who will be the delivery agent for the maintenance. And although the ISSCF contract for the CH149 was with a Canadian company, Industrial Marine Products of Halifax, the lack of IP meant that even though this company was prime, it did not have access to data from Augusta Westland (the OEM).

For the CC177, all second- and third-line maintenance is performed by Boeing at a facility in the United States. Canadian aircraft are not differentiated from the USAF fleet for purposes of maintenance. Only first-line maintenance is done by the RCAF in Canada. Similarly the CC130J is maintained through an ISSCF contract with Lockheed Martin. Lockheed Martin, in turn, has contracted with some Canadian firms, or Canadian-based operations of foreign firms. This includes Cascade Aerospace for third-level airframe maintenance, CAE for operational training, Industrial Marine Products for materiel/warehouse management, Rolls Royce for engines, and GE/Dowty for the propellers.

2.3 Performance (Efficiency and Economy)

To assess the efficiency of the AEM program the evaluation conducted a quantitative and qualitative assessment of the program outputs and outcomes from the following perspective:

  • Are material and services acquired in a manner that delivers best value for the program? (Output 1)
  • Does the supply chain function in an efficient manner? (Immediate outcome 1)
  • Is there an efficient use of maintenance personnel? (Output 2)
  • Does the management structure and governance drive best practices in efficiency? (Immediate Outcomes 2 and 3)

2.3.1 Efficiency of Acquisition of Material and Services (Output 1)

A data review of procurement activity was conducted to assess the efficiency of Output 1.

Efficient procurement activity should be designed to deliver best value for the end user. The procurement should deliver the good or service for the best possible price, taking into consideration all needs, including both quality and technical requirements. To obtain best value, leading procurement organizations attempt to strike a balance between the frequency and size of individual orders, and the number of potential competitors. The establishment of strategic partnerships with key suppliers is also a best practice, since it can improve the integrity of the supply chain and promote innovation.98

While the volume of aerospace goods and services purchased by PWGSC on behalf of the DND/CF varies from year to year, the vast majority is sole-sourced (purchased without competition). As shown in Figure 3, the dollar value of aerospace goods and services purchased annually varied from $250 million to $1.9 billion in any one year, with an average of 85 percent of those purchases being sole-sourced. This represents a very high percentage of sole-source contracting activity and reflects the nature of aerospace maintenance, which typically requires components from original equipment manufacturers, or that of a proprietary nature, which limits competition. Due to this reality, the value of strategic partnerships such as OSWM contracts is very important, and there is a need to ensure that these partnerships are put in place as much as possible.

Figure 3. Value of Aerospace Equipment Purchased by PWGSC. This figure shows the value of aerospace equipment and services, separated by competed and sole-sourced acquisitions, purchased by PWGSC on behalf of DND/CF for FYs 2007/08 to 2011/12. The data is shown in Table 6.

Table 6. Dollars by Fiscal Year—Competed or Sole-sourced. This table lists the amounts in millions of dollars spent by PWGSC on aerospace equipment by fiscal year for competed and sole-sourced acquisitions.

Table Summary:

Rows show whether the contract was competed or sole-sourced. Columns list the amount of dollars spent by PWGSC on aerospace equipment by fiscal year.

FY 2007/08
($ millions)

FY 2008/09
($ millions)

FY 2009/10
($ millions)

FY 2010/11
($ millions)

FY 2011/12
($ millions)

Competed

$74.3

$71.5

$175.1

$107.7

$45.4

Sole-Sourced

$914.3

$596.4

$1,698.2

$246.2

$198.2

Through a review of existing contracting documents,99 the evaluation team found that within the DND AEM program, there have been significant efforts taken to maximize order size and build strategic partnerships. This is demonstrated largely through the competition of OSWM contracts. By bundling many smaller orders, establishing a long-term arrangement with a supplier, but still having a competition for that contract, OSWM should represent best value. At present, these contracts account for approximately 30 percent of aerospace maintenance service contracting.

2.3.1.1 ISSCF

Human resources are a factor in the support of fleets. In the context of reduction in the number of civilians, in the number of contractors providing professional services, and a cap on military personnel, supporting existing fleets and introducing new ones is a growing challenge. The ISSCF concept shifts the management of a larger scope of the activities to the contractor which in turn reduces the number of “blue suiters” and civilians necessary to perform functions that were previously residing within the WSMs or at wings. This evaluation did not explore the impact of human resources on ISSCF, and ADM(Mat) may wish to examine this impact further.

Due to the lack of long-term experience with ISSCF, it was unclear at the time of this evaluation whether the ISSCF-type contract vehicle provides value for money. It is difficult to determine what alternative costs would be, particularly over the long term. The risk is that as the contracts progress the ability of ADM(Mat) to challenge any extra charges, or even find comparative pricing may diminish as expertise and competition is lost. Other countries have similar issues—in fact the Australians have reverted from their ISSCF-like contract for the CC130J to an OWSM-like one.100

The risk is in the lack of ability for ADM(Mat) to “challenge” links with the difficulties the evaluation team found within the RCAF regarding the ability to provide the complete costs to perform AEM. Though wing-level business plans contain exacting data on minor expenditures, the RCAF could not provide detailed costs of performing AEM, and defaulted to providing data from the Cost Factors Manual. While a framework of some nature must have underpinned the letting of the current ISSCF contracts, detailed frameworks for all fleets would give ADM(Mat) better visibility into its overall costs and permit better long-term planning and business case analysis.

The evaluation found the following issues with respect to ISSCF contracts:

  • In the ISSCF contract examined101 the contractor was the OEM, and although the contractor had extensive and in-depth experience as a manufacturer, it had limited experience as a support provider. This inexperience manifested itself to the CF in the form of the OEM being a contractually rigid ISS provider. Part of the issue is that the OEM uses a matrix support concept versus a dedicated integrated product team. Consequently, any question raised by the CF to the contractor could be looked at by many of the contractor’s staff, who generally are not co-located, and who have other manufacturing responsibilities. The contractor is not a service industry, and is not organized to respond in a manner indicative of a service industry.
  • Canada did not incorporate non-inherent repairs into the support contract.102 Non-inherent repairs, such as damage resulting from bird strikes and gravel damage to the belly of the aircraft, are not uncommon given Canada’s environment and unpaved landing strips, but the ISSCF contract did not allow for these issues and the rigid interpretation of liability by the contractor has caused delays.
  • Although the ISSCF contract103 mentions additional work requests (AWR), the process time for the contractor to respond to an AWR and provide a cost to Canada is in terms of months, not days. This may ground an aircraft for extended periods.
  • There is no visibility into the contract costing.104 Because the contractor is not located in Canada, the DND/CF has no access to the contractor’s financial systems. This is a cause for concern as there is no way for the DND/CF to verify the amounts claimed on invoices, nor to examine their supporting documents.

In correspondence with ADM(Mat) staff105 they indicated that the “ISSCF concept shifts the management of a larger scope of the activities to the contractor which in turn reduces the number of ‘blue suiters’ and civil servants necessary to perform functions that were previously residing within the Weapon System Manager organization or at wings.” The evaluation team agrees with this point; however, the fundamental raison d’être for the creation of an ISSCF-type contracting vehicle was because the DND/CF had money and not enough people after the Force Reduction Program of the 1990s. What the evaluation questioned is whether ADM(Mat) or the RCAF retained enough people and expertise to manage the new way of doing business. The evidence of contract management expertise within the WSM organizations suggests that it could do better.

Recommendation

3. A review of the ISSCF contract mechanisms should be conducted to ensure the ability to manage costs and performance are maximized over the long term. The review should give consideration towards splitting the ISSCF into smaller contracts (i.e., for airframe, avionics, and engine systems), rights to conduct cost audits, and consistent approaches towards value engineering, incentives, penalties, etc. This is closely linked with Recommendation 1.

OPI: ADM(Mat)

2.3.2 Efficiency of the Supply Chain

Finding #13. Robbing increases the burden on maintenance organizations. The data on rob rates by fleet was readily available in Performa.[106]

In examining the efficiency of the supply chain, the evaluation noted several issues. These include:

High-Priority Requests

Supply centres generally should order replenishment stock when items approach their minimum stock-on-hand levels. Due to the quantity of materiel ordered from the depots by all environments, prioritization of demands is important. There are four priorities assigned by the user, the highest of which is the high-priority request (HPR). HPRs should be picked up and shipped as soon as practicable with targeted delivery normally estimated within seven days or less (see Table 7).

Table 7. High-Priority Request Performance Data (2011). Based on a sample of three fleets, the HPR target delivery date of seven days or less was not met in 58.3 percent of cases.

Legend:

N/A – Not Applicable

Table Summary:

For each aircraft type listed in the left-hand column, read across the row to determine the measurement criteria per quarter.

Fleet

Measured

January-March

April-June

July-
September

October-December

CF188

Average wait time (days) for HPRs

▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓

Average number of HPRs

▓▓▓▓▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓

CP140

Average wait time (days) for HPRs

▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓

Average number of HPRs

▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓

CC130H

Average wait time (days) for HPRs

▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓

Average number of HPRs

▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓

When parts are not available at the base or from the warehouse in a timely manner, the practice of “robbing” occurs, which impacts the efficiency of the maintenance program. Robbing is the controlled removal, with intent to replace, of a serviceable part from one aircraft (typically already out of service) to use on another aircraft. Robbing allows operational requirements to be met when spares are not available within permissible time constraints.

There are occasions where a maintenance crew will rob a part as a matter of convenience when the part is available in supply (i.e., technicians (on night shift) may rob a part from an aircraft undergoing periodic inspection rather than recalling the duty supply technician; technicians may rob parts even though the part is at the depot in order to get the aircraft back up in the air sooner). While robs are convenient, they are sometimes overused.

HPRs give a much better indication of the status of supply deficiencies. Aircraft engineering officers (AEO) and their LCMM normally review HPRs to determine the issue and implement a long-term solution. Their effectiveness in resolving the issues can be limited by the lack of action from field units, which have a role to play in establishing min/max levels. Many parts and consumables are locally procured (i.e., through a local procurement officer) and the field unit is required to set the minimum/maximum inventory levels. For items centrally managed, if a part is continuously robbed (i.e., not enough repairable spares), it is the field unit’s responsibility to review the minimum (level at which the part is re-ordered) and maximum (quantity to be stored at that location) levels at that unit and recommend a change to the AEO. In the event that parts are unavailable due to a short supply chain (i.e., not enough parts in the pool), then the LCMM can rectify by procuring new items or work with the repair facility to reduce turn-around time.

When the evaluation team questioned the RCAF and commercial air operators over the practice of robbing we were told that it is a necessary process in aircraft maintenance because aircraft must be available to meet the mission (or scheduled passenger departures), and the necessary parts are not always available in a timely manner. Due to the high cost of spares, robbing is sometimes the only way to maintain aircraft availability; however, it is a highly inefficient practice.

Robbing, at a minimum, costs twice as much in labour as the straight installation of an available part from the supply system—the part must be removed from the robbed aircraft, installed on the receiving aircraft, and once a replacement part arrives it must be installed on the robbed aircraft. There is a risk of parts breakage or damage to the aircraft when removing the part. Additionally, due to airworthiness requirements all these procedures must be recorded.

Robbing imposes a burden on maintenance organizations in that the extra labour expended on robs could be assigned elsewhere. Given the manpower demands with the RCAF for new fleets, this matter needs attention. ADM(Mat) and the RCAF need to ensure that rob rates do not become excessive due to inventory control factors within their institutional control. The evaluation found that although the RCAF and ADM(Mat) review rob rates on an on-going basis,107 neither the RCAF nor ADM(Mat) have been effective in reducing rob rates or improving spares management.108

An effective means to reduce robs was shown through collaborating with SAMEOs. The SAMEO developed a “top 25” list of robbed items and the contractor then shifted planning and allocated resources accordingly to meet the needs of the squadron. Interviewees (n=5, NCOs and contractors) reported that collaboration was inconsistent with some SAMEOs collaborating with contractors to address sparing needs while others continued to work in silos. Partnership between individual WSMs and the contractor must be constantly monitored to ensure that the provision of parts is not impeded.

Disposal

The evaluation also examined performance of 25 CFSD’s disposal process. To maximize storage space requires effective monitoring of stock, accompanied by a fluid process for disposal when parts are no longer required. The process for disposal of obsolete/non-repairable items requires the fleet supply managers to issue the request to conduct the disposal process. Since fleet supply managers do not issue instructions to dispose of obsolete equipment on a timely basis,109 obsolete equipment sits in storage for extended periods. Delays were reported as commonly lasting up to years.110 For example, the extent of dormant equipment remaining in holdings was highlighted by certain parts used during the Vietnam War being disposed of only recently.111

2.3.3 Efficient Use of Maintenance Personnel (Output 2)

Human resource data for military and civilian personnel from the wings/bases and National Defence Headquarters was gathered to determine the efficiency of personnel conducting maintenance. Due to changing operational requirements (flying hours and type of flight), needs (aircraft type, model and age), funding, and priorities, it is difficult to make comparisons for specific fleets on a year-over-year basis, or to compare them to similar fleets in other air forces. However, for evaluation purposes, data for two of the largest fleets (CF188 and the CC130) was examined in detail as case studies.

The most data was available for the 26 aircraft in the legacy CC130 fleet (13 CC130E and 13 CC130H (see Figure 4).112 An annual average of 657 military and civilian personnel worked on this aircraft from FY 2006/07 to FY 2009/10. This represents an average of 21.6 maintenance personnel per aircraft (older CC130). Beginning in 2010, the CC130 fleet changed in number as the new CC130J arrived and the 13 CC130E models were retired and disposed of. The 17 new CC130Js assumed the role of tactical airlift, and the remaining 13 CC130Hs were transferred to an SAR role, for a total fleet size of 30 aircraft. In this configuration, the annual average number of military and civilian personnel involved in the maintenance of the CC130 is 455 maintenance personnel, or an average of 15.3 per aircraft, which is a reduction of over 30 percent. This reduction could be attributed to several factors: the outsourcing of maintenance to the contractor under the ISSCF; the reduced maintenance demands of the newer aircraft; and/or reduced YFR.

Figure 4. CC130 Personnel to Aircraft Ratio. This figure shows the number of CF and civilian personnel supporting the maintenance function for the CC-130 Hercules, by FYs 2006/07 to 2011/12. The figure shows the number of personnel, the number of aircraft, and the ratio of personnel to aircraft. The data is shown in Table 8.

Table 8. CC130 Personnel to Aircraft Ratio. This table shows the number of personnel, the number of aircraft and the ratio of personnel to aircraft per fiscal year, ranging from FY 2006/07 to 2011/12.

Table Summary:

Rows list number of personnel, number of aircraft, and personnel per aircraft. Columns list year.

FY 2006/07

FY 2007/08

FY 2008/09

FY 2009/10

FY 2010/11

FY 2011/12

Number of Personnel

717

567

631

712

496

414

Total Number of CC130

26

26

26

26

30

30

Ratio of Personnel/Aircraft

27.6

21.8

24.3

27.4

16.5

13.8

2.3.4 Program Management and Governance

To evaluate the efficiency of the approach to maintenance taken by ADM(Mat), a qualitative assessment of the use of best practices in efficiency by the aerospace industry as a whole was reviewed.

Efficient air-maintenance practices113 were seen to include the use of the following:

  • modern centralized governance
  • joint user groups
  • efficiency performance measures
  • other practices to maximize efficiency

2.3.4.1 Modern Centralized Governance Activities

Modern aerospace maintenance is focussed on centralized centres of expertise which provide coordinated planning, control, material management, and maintenance functions for the all fleets across the organization, as opposed to decentralised, independent functions occurring at the local level, typically where each aircraft is based.114

ADM(Mat) uses a centralized governance and management model for AEM. Given the defence tasks that have to be performed and the fleets and aircraft available, the 1 CAD’s operations staff determine how many flying hours are required to perform those tasks. These flying hours are rolled into YFR per fleet by fiscal year. Based on the YFR expected to be flown, WSMs estimate the cost to keep sufficient aircraft mission capable. Each WSM has a system, some more sophisticated than others, to predict their requirements.

Each fleet is examined to ensure that an appropriate level of service is planned which aligns with operational and strategic priorities. Gaps in coverage are acknowledged and if possible addressed, and other resource utilization direction is given.

2.3.4.2 Joint User Groups

Each major airframe the RCAF operates has an international community of interest that focuses on how to maintain them. Commonly called user groups, they include working groups such as the Joint User Group for the CC130J, the Hercules International Conference for the CC130J and the CC130H and the Hornet User Group for the CF188. Through these working groups lessons, programs and data are exchanged.

For instance, for the CF188 the data from the fatigue/strain testing which the DND/CF conducted is often sought by members of the Hornet User Group.115

Participation in these user groups allows ADM(Mat) to be proactive in its maintenance planning by gaining access to specialized data (tests that individual countries have performed) plus obtaining trend data from across a wider representative sample.116 Participation in these groups thus contributes to potentially greater economies and efficiencies in the DND/CF programs.

The civil aviation community also has a long history of pooling assets.117 Pooling is also the current practice under the Global Sustainment Partnership for CC177 users. Canada contributes a share of funding to the program and receives access to pooled parts wherever its planes deploy in the world. This has proven good value for money for the DND/CF as it has access to a less costly and diverse inventory than if it had purchased a complete inventory for its sole use.

2.3.4.3 Performance Management

Finding #14. Performance management systems and performance measurement data are inconsistent across the fleets.

In a few instances, performance management as it is done by ADM(Mat) employs procedures and practices to ensure work is performed properly,118 and, in some cases, management information systems are used well.119 Support efforts and results for aerospace maintenance are monitored by ADM(Mat) to identify problems and initiate improvement actions.120 Audits and inspections are employed, in the form of Air Force 9000+ Quality Management Programme for aircraft engineering and maintenance, operation and technical airworthiness reviews, flight safety surveys, general safety surveys, and force protection reviews.121

However, while these performance management systems exist, the concern is that the level of detail is inconsistent across fleets, and that they are not used consistently as a management tool. The evaluation found that some WSMs (n=3) do not use data collected for them by the contractor or the RCAF maintenance staff in decision making. Although some WSMs (n=5) reported that they used performance data in their decision making, others (n=3) were not aware that performance data was available.

2.3.4.4 Best Practices to Improve Efficiency

All aircraft are subject to periodic inspections122 and more detailed TLIRs.123 Each periodic and TLIR will be unique based on the specific work requirements necessary. Planning for periodics includes determining when a specific aircraft should enter the work bay124 and comprehensive pre-planning to ensure all the necessary resources are on hand so that work flows smoothly.125 Given their complexity, periodic inspections, if not well managed, can consume enormous amounts of time. The operational availability of aircraft and the availability of technicians to perform other tasks are increased by reducing the amount of time aircraft spend in periodic inspections.126

The RCAF has initiated several programs to reduce periodic inspection downtime. One initiative is known as Operation (Op) Production, an Air Force-wide program.127 Trials under Op Production began in early 2008 and it was gradually implemented throughout 2009. Op Production provided tracking tools to maintenance officers so that they could better manage periodic inspections.

OP Production was suspended in February 2010.128 Up to that point, it had been implemented at 9 of 17 units that conducted periodic inspections and the statistics collected indicated positive performance results (see Table 9129).

Table 9. Time Spent in Periodics. The table shows the reduction in Turn-around Time (TAT) and increase in operational availability arising from the implementation of Op Production.

Table Summary:

For each fleet type listed in the left-hand column, read across the row to determine the reduction in time at task and increase in operational availability.

Fleet

Pre-Intervention TAT[130] (2007)

New TAT (30 June 2009)

Reduction of Days

Reduction in Scheduled Maintenance Downtime

Number of lines of Periodic Inspections

Increase in Operational Availability

CH146

▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

CH124

▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

CP140

▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

CC138

▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

Some commanding officers of maintenance organizations were striving to reduce periodic inspection times.131 At 4 Wing, Cold Lake, the Commanding Officer of 1 AMS was closely involved in the design of the layout of the periodics bays and mapping out best practices to reduce the overall time CF188s were inducted. Similar attention was afforded to CP140 periodics in 14 Wing Greenwood. There, the average time had been cut by ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 14 Wing has unique software that allows for the detailed tracking of all activities associated with their periodics. This gives the Commanding Officer of 14 AMS enormous visibility over the process.

Contractors are engaged to both assist in the conduct of periodics or to conduct them themselves. For instance, L-3 MAS provided technical advisors to the CF188 periodic lines in Cold Lake.132 Contractors conduct the periodics for the CC177,133 both variants of the Hercules (CC130J134 and CC130H135) and the CH149.136

Recommendation

4. Best practices noted at CFB Greenwood regarding periodic inspections should be examined for opportunities to apply to other fleets.

OPI: CAS

Annex A—Management Action Plan

Performance (Effectiveness)

The management of spare parts by the DND has been commented on previously by the Auditor General, the Public Accounts Committee, and CRS in its audits. ADM(Mat) acknowledges that issues exist with the management and disposal of spare parts and has prepared MAPs to address these issues.

CRS Recommendation

1. In order to maximize value and options for maintenance approaches, ADM(Mat) should review procurement practices with respect to obtaining sufficient technical data and IP rights for new platforms and ensuring that maintenance support contracts such as ISS are negotiated at the time of the capital procurement. The review should give consideration to non-disclosure clauses, foreground IP, or transfers of IP after a set period of time, in order to maximize options for maintenance approaches.

Management Action

GC policy (Treasury Board) states that the Contractor be the owner of any foreground IP created by the Contractor arising by virtue of a Crown Procurement Contract, subject to eight “exceptions” provided for in said policy. One exception allowed is for national security reasons. Under this exception, Defence Administrative Orders and Directives (DAOD) item 3022-1, Management of Procurement of ISS for CF Platforms, provide the following direction with respect to IP:

“DND employees and CF members involved in ISS contracting shall:

  • secure all rights, including access and IP rights to the technical data needed for the GC to perform its activities, including the ability to conduct independent analysis and verification;
  • ensure that all background IP and foreground IP, and technical data necessary to re-compete or repatriate the work is provided;
  • normally take ownership of the foreground IP generated from the CF platform acquisition contracts and ISSCs relating to critical defence industry capabilities;
  • obtain a license to the necessary rights to the background IP to enable the GC to exercise its rights to the foreground IP;
  • normally grant licences to Canadian industry to commercially exploit the IP owned by the Crown and controlled by the DND or the CF; and
  • obtain all necessary permissions from foreign governments (e.g., licences or approvals under the International Traffic in Arms Regulations of the United States) to use and release to third parties as authorized, the technical data and IP to which the GC has secured rights.”

Moreover, DAOD 3022-0, Procurement of In-Service Support for CF Platforms requires that the ISS concept be submitted for ADM(Mat) approval prior to any contractual commitments being made. This review and approval process ensures that individual projects comply with the above IP policy. The IP policy outlined above is consistent with this recommendation. However, this policy was only issued in August 2010—after the CH149 support contract and some of the first ISSCF contracts were established. Given the publication of the referenced DAODs and ADM(Mat)-led ISSCF contract reviews, the actions required to address the CRS recommendation have already been completed.

OPI: ADM(Mat)

Status: Complete

CRS Recommendation

2. Performance metrics must be better understood, used more consistently, standardized where possible, and included in all major contracts. Staff must be formally trained in their use.

Management Action

Director Materiel Policy and Procedures (DMPP) 5 will consult with key stakeholders (Equipment Program Managers, Director Materiel Systems Plans and Requirements, Director Supply Chain Operations, Director Materiel Group Human Resources) to establish a detailed work plan with identified milestones in order to outline the approach required to meet CRS’ recommendation for performance (effectiveness). This activity will allow the Materiel Group to leverage the work being conducted within the context of the Materiel Acquisition and Support Value Chain Architecture.

OPI: ADM(Mat)/DGMSSC/DMPP

Target Date: January 2014

Status: Under way

Performance (Efficiency and Economy)

CRS Recommendation

3. A review of the ISSCF contract mechanisms should be conducted to ensure the ability to manage costs and performance are maximized over the long term. The review should give consideration towards splitting the ISSCF into smaller contracts (i.e., for airframe, avionics, and engine systems), rights to conduct cost audits, and consistent approaches towards value engineering, incentives, penalties, etc. This is closely linked with Recommendation 1.

Management Action

ADM(Mat) will conduct a review of the ISSCF framework to optimize cost and performance over the full life-cycle. Consideration will be given to a logical breakdown of the requirement, rights to conduct cost audits, and consistent approaches towards value engineering, incentives and penalties.

OPI: ADM(Mat)/DGMSSC/DMPP

Target Date: February 2014

Status: Under way

CRS Recommendation

4. Best practices noted at CFB Greenwood regarding periodic inspections should be examined for opportunities to apply to other fleets.

Management Action

1 CAD A4 Maint has re-started Op Production, whose goal is to improve the execution of periodic inspections in the RCAF. Through the output of Op Production, training will be delivered, tools made available and best practices will be shared between organizations and fleets.

OPI: CAS

Target Date: March 2014

Status: Under way

Annex B—Logic Model

Figure 5. Logic Model. This is a description of the program logic showing inputs, activities and outputs, and how these link to immediate, intermediate and ultimate outcomes.

Text for figure 5:

This flowchart is organized into four streams based on four activities: obtain materiel in the form of spare parts; acquire and use labour (i.e., “blue suits”) to perform aircraft maintenance; engage contractors in the private sector to perform aircraft maintenance; and create and employ the laws, policies, literature, and doctrine appropriate to aircraft maintenance. All activities contribute to the ultimate outcome that the CFDS mission is met.

Starting Point A: Materiel

  • Input/Activities: Spare parts are obtained
  • Outputs: Parts are available at an appropriate cost where and when required
  • Outcomes: Using an appropriate industrial capacity, quality maintenance is performed which maximises aircraft longevity and provides safe aircraft in sufficient quantity to meet the CFDS mission

Starting Point B: Labour

  • Input/Activities: Appropriately trained RCAF members are available to do the work
  • Outputs: Preventive, repair and overhaul maintenance is done
  • Outcomes: Using an appropriate industrial capacity, quality maintenance is performed which maximises aircraft longevity and provides safe aircraft in sufficient quantity to meet the CFDS mission

Starting Point C: Contractors

  • Input/Activities: Appropriately trained contractors are available to do the work
  • Outputs: Preventive, repair and overhaul maintenance is done
  • Outcomes: Using an appropriate industrial capacity, quality maintenance is performed which maximises aircraft longevity and provides safe aircraft in sufficient quantity to meet the CFDS mission

Starting Point D: Laws, Policies, Literature and Doctrine

  • Input: Regulations and standard operating procedures are in place to guide aircraft maintenance
  • Outputs: an appropriate operating framework which emphasises safety and quality
  • Outcomes: Using an appropriate industrial capacity, quality maintenance is performed which maximises aircraft longevity and provides safe aircraft in sufficient quantity to meet the CFDS mission

Annex C—Evaluation Matrix

Table 10. Evaluation Matrix—Relevance. This table provides the indicators used to assess the evaluation issues/questions for determining the relevancy of AEPM.

Table Summary:

The evaluation issues/questions related to relevance are indicated in the left-hand column. For each evaluation issue/question, read across the row to determine what the indicators are.

Relevance
Evaluation Issues/Questions

Indicators

1.1 Does AEPM continue to address a demonstrable need best filled and managed by the RCAF?

1.1.1 Evidence that the AEM program continues to address a demonstrable need best filled and managed by the CF. Evidence furnished by document and literature review, interviews, RPP, DPR, CFDS

1.1.2 Evidence that the program responds to stakeholders needs. Evidence furnished by document and literature review, interviews, RPP, DPR, CFDS

1.2. Is AEPM aligned with federal roles and responsibilities?

1.2.1 Degree of alignment with federal government roles and responsibilities. Evidence furnished by document and literature review, interviews, RPP, DPR, CFDS

1.3 Is AEPM aligned with departmental roles and responsibilities?

1.3.1 Degree of alignment with DND/CF roles and responsibilities. Evidence furnished by document and literature review, interviews, RPP, DPR, CFDS

Table 11. Evaluation Matrix—Performance (Effectiveness) – Immediate Outcomes. This table provides the indicators used to assess the evaluation issues/questions for determining the effectiveness (immediate outcomes) of AEPM.

Table Summary:

The evaluation issues/questions related to effectiveness (immediate outcomes) are indicated in the left-hand column. For each evaluation issue/question, read across the row to determine what the indicators are.

Performance (Effectiveness) –
Immediate Outcomes
Evaluation Issues/Questions

Indicators

2.1 Is quality aircraft maintenance done?

2.1.1 Is there a professional and sustainable workforce? Evidence furnished by program data, document and literature review, interviews, operational data

2.1.2 Are technicians trained appropriately? Evidence furnished by program data, document and literature review, interviews, operational data

2.1.3 Are aircraft safe? Evidence furnished by program data, document and literature review, interviews, operational data

2.2. Is aircraft management effective?

2.2.1 Are services and materiel available to support maintenance activities? Evidence furnished by program data, document and literature review, interviews, operational data

2.2.2 Are there incentives for industry to improve its contribution? Evidence furnished by program data, document and literature review, interviews, operational data

2.2.3 Are appropriate Performance Measures in place? Evidence furnished by program data, document and literature review, interviews, operational data

Table 12. Evaluation Matrix—Performance (Effectiveness) – Intermediate Outcomes. This table provides the indicators used to assess the evaluation issues/questions for determining the effectiveness (intermediate outcomes) of AEPM.

Table Summary:

The evaluation issues/questions related to effectiveness (intermediate outcomes) are indicated in the left-hand column. For each evaluation issue/question, read across the row to determine what the indicators are.

Performance (Effectiveness) –
Intermediate Outcomes
Evaluation Issues/Questions

Indicators

3.1 Is YFR achieved?

3.1.1 YFR targets are met. Evidence furnished by program data, document and literature review, operational data

3.2 Are aircraft maintained to achieve maximum service life?

3.2.1 Degree of success in achieving life expectancy of aircraft. Evidence furnished by program data, document and literature review, interviews, operational data

3.3 Are strategic industrial capacities sustained?

3.3.1 Viability of Canadian aerospace industry. Evidence furnished by program data, document and literature review, interviews, operational data

Table 13. Evaluation Matrix—Performance (Effectiveness) – Ultimate Outcomes. This table provides the indicators used to assess the evaluation issues/questions for determining the effectiveness (ultimate outcomes) of AEPM.

Table Summary:

The evaluation issues/questions related to effectiveness (ultimate outcomes) are indicated in the left-hand column. For each evaluation issue/question, read across the row to determine what the indicators are.

Performance (Effectiveness) –Ultimate OutcomesEvaluation Issues/Questions

Indicators

4.1 Is the CFDS mission met?

4.1.1 Degree of success in achieving operational demands of the RCAF. Evidence furnished by program data, document and literature review, interviews, operational data.

Table 14. Evaluation Matrix—Performance (Effectiveness) – Unintended Outcomes. This table provides the indicators used to assess the evaluation issues/questions for determining the effectiveness (unintended outcomes) of AEPM.

Table Summary:

The evaluation issues/questions related to effectiveness (unintended outcomes) are indicated in the left-hand column. For each evaluation issue/question, read across the row to determine what the indicators are.

Performance (Effectiveness) –Unintended OutcomesEvaluation Issues/Questions

Indicators

5.1 Is there evidence of unintended outcomes?

5.1.1 Evidence that when unanticipated effects or events arise they are addressed in a timely fashion. Evidence furnished by program data, document and literature review, interviews, operational data.

Table 15. Evaluation Matrix - Performance (Efficiency and Economy). This table provides the indicators used to assess the evaluation issues/questions for determining the performance (efficiency and economy) of AEPM.

Table Summary:

The evaluation issues/questions related to performance (efficiency and economy) are indicated in the left-hand column. For each evaluation issue/question, read across the row to determine what the indicators are.

Performance (Efficiency and Economy)
Evaluation Issues/Questions

Indicators

6.1 Are materiel and services acquired in a manner that delivers best value for money?

6.1.1 Evidence of materiel acquisition. Evidence furnished by financial information, program data, document and literature review, interviews, operational data

6.1.2 Evidence of services acquisition. Evidence furnished by financial information, program data, document and literature review, interviews, operational data

6.2 Does the supply chain function in an effective manner?

6.2.1 Evidence of supply chain framework and processes. Evidence furnished by program data, document and literature review, interviews, operational data

6.2.2 Evidence that data is collected and used effectively in decision making. Evidence furnished by program data, document and literature review, interviews, operational data

6.3 Is there an efficient use of maintenance personnel?

6.3.1 Evidence of personnel usage. Evidence furnished by program data, document and literature review, interviews, operational data

6.4 Does the management structure and governance drive best practices in efficiency?

6.4.1 Evidence of the management structure. Evidence furnished by program data, document and literature review, interviews, operational data

6.4.2 Evidence of the governance structure. Evidence furnished by program data, document and literature review, interviews, operational data

Annex D—Performance Measures – Five Fleets

Table 16. Performance Measures – Usage. This table shows the number of performance measurement metrics for the five aircraft types evaluated; at what date the performance measurement became in use; and whether the metric is an “award” for performance, is measured against a “standard”, or is used as a “correction” to existing methods.

Legend:

N/A – Not Applicable

Table Summary:

For each aircraft type indicated in the left-hand column, read across the row to determine the performance measures in terms of usage.

Air Frame

Performance Measurement Metrics

Effective Date

Award

Standard

Correction

CF188

3

Now

N/A

N/A

N/A

CC130H

14

Now

Year 6

N/A

Year 2

CC130J

3

1 April 2013

Not yet in use

CC177

7

Now

Not yet in use

CH149

9

Now

Currently not active

Table 17. Performance Measures – Review and Weight. This table lists the performance measurement metrics for five aircraft types.

Table Summary:

For each aircraft type indicated in the left-hand column, read across the row to determine the performance measures in terms of review and weight.

Air Frame

Committee

Performance Measurement Metric

Description

Weight

CF188

Contract Performance Review Board

1

Window Availability

50%

2

Window Response Time

30%

3

AVS Parts Quality

20%

CC130H

Not specified

1

Customer Satisfaction Qualitative

10 metrics selected annually out of the 14

2

Report Delivery Deviation

3

Cost Control

4

Project Management Maturity

5

Request for Assistance Support

Qualitative

6

Publication Quality

7

PAV Maintenance Deviation

8

PAV TAT Deviation

9

PAV Quality

10

PAV Parts Availability

11

PAV Parts Reliability Improvement

12

PAV Parts Failure Index

13

Consumable Parts Cost Index

14

Information Management/ Information Technology System Integrity

CC130J

1

TAT non-fully mission capable rate– Contractor Attributable

Not given a weight

2

TAT Mean Time Between Mission Aborts

3

TAT Scheduled Availability – Maintenance Training Devices

CC177

1

Fleet Performance Aircraft Availability

Part of GISP process

2

Mission Capable Awaiting Parts

3

Aircraft Depot Maintenance Scheduling Effectiveness

4

Serviceable Propulsion System/War Readiness Engine

5

Time Compliance Technical Order Timeliness

6

Cost Initiatives (Award Fee Only)

7

Significant Events

CH149

Senior Management Board

1

Aircraft Mission Reliability (SAR)

Not in use

2

Aircraft Mission Reliability - Scheduled and Operational Training Unit

3

Supplementary Work Request (SWR) Index

4

Repairable Parts Availability

5

Aircraft Availability (Scheduled)

6

Aircraft Availability Operational Training Unit

7

SAR Standby Aircraft Availability

8

SWR TAT

9

Cost Index

Table 18. Performance Measures – Award Calculations. This table shows the performance measurement calculation used to determine if an award in excess of the contract amount will be given for five aircraft types. Three aircraft use a Composite Performance Score. One aircraft uses three performance indicators, and one aircraft is part of a foreign fleet.

Table Summary:

For each aircraft type indicated in the left-hand column, read across the row to determine the performance measures in terms of review and weight.

Air Frame

Term

Method

Calculations

CF188

Contract Performance Adjustments

Composite Performance Score

Performance measurement calculation used to determine if an award in excess of the contract amount will be given.

CC130H

Award for Excellence

Composite Performance Score

Performance measurement calculation used to determine if an award in excess of the contract amount will be given.

CC130J

Performance Measure Adjustments

Individual

Performance measurement calculation used to determine if an award in excess of the contract amount will be given.
Performance measurement calculation used to determine if an award in excess of the contract amount will be given.
Performance measurement calculation used to determine if an award in excess of the contract amount will be given.

CC177

Incentive/ Disincentive Fee

Roll-up

USAF-led, Canadian data rolled in

CH149

Performance Pay Adjustment

Composite Performance Score

Performance measurement calculation used to determine if an award in excess of the contract amount will be given.

___________________________________________________________________________________________________________________________

Footnote 1 For purposes of program spending evaluation, this evaluation covers $1.75 billion due to $300 million being covered by the evaluations of Real Property, Training and Force Generation.

Footnote 2 DPR, 2010-11, Part III, Estimates.

Footnote 3 Description of Lines. Refers to where, organizationally, an activity will occur as part of the overall approved maintenance program for an aircraft weapon system. It is most often related to the frequency with which an event occurs and the proximity of the required recovery activity to the flying operation.

Footnote 4 Description of Levels. There are three “levels” of maintenance to support the DND/CF Airworthiness Program requirement for a maintenance organization to have a defined and authorized depth of maintenance.

Footnote 5 A government-mandated reduction in personnel and funding which ran from the early 1990s to 1996.

Footnote 6 Although the value of aerospace maintenance is approximately $2.05 billion annually, for purposes of program spending coverage for evaluation, this evaluation covers approximately $1.75 billion annually. Other monies are evaluated elsewhere and this number of $1.75 billion was arrived at by deducting from the $2.05 billion $140 million for real property; $11 million for training; and $155 million for force generation.

Footnote 7 DND/CF has 5,845 military members and civilians employed in aerospace maintenance. The Canadian private sector has an estimated 2,000 ex-military members and civilians supporting aerospace maintenance at DND/CF.

Footnote 8 Excluding Support to Deployed Operations Account. Typically the Aerospace Maintenance Committee (AMC) and the NPOC focus on the incoming year with the out years being considered as “good estimates.” Expenditures are refined as a fiscal year is approached. The FY 2011/12 NPOC cycle was the first time that NPOC looked at multi-year allocations, and the process was immature and not fully understood. For example, in the FY 2011/12 allocation letter, DGAEPM was originally funded at 80 percent of demand. It was agreed that 80 percent was not sufficient and additional funding was provided during the fiscal year. The subsequent cycle (FY 2012/13 allocation letter) NPOC decided to fund DGAEPM to approximately 89 percent of demand. However, demand was reduced as the fiscal year approached because some new capabilities (i.e., Maritime Helicopter Project) were delayed. The combination of both a lower actual demand with an increase in demand funded was reflected in the FY 2012/13 allocation letter. (Source: email, DGAEPM Comptroller, 1 November 2012.)

Footnote 9 AEPM Level 2 Business Plan, FY 2011/12.

Footnote 10 Director Military Strategy. 4,751 AEM positions on strength, 4,974 planned. Summer 2011.

Footnote 11 “Force generation” encompasses those activities and processes related to assembling, equipping, training, certifying and generally preparing military field and garrison forces, and activities required to maintain military forces in a defined state of readiness for FE. This does not include the activities and processes related to recruiting, basic military qualification training, or initial occupational training.

Footnote 12 Although CRS is responsible for the audit function within DND, “audit” as it is defined here refers to the Aircraft Maintenance Standards Evaluation Team and Air Force 9000+ quality control reviews and examinations (audits) performed by the Aircraft Maintenance Evaluation 2 group within A4 Maint. These audits ensure that RCAF aerospace maintenance units at the wings/bases maintain their airworthiness accreditation.

Footnote 13 Aeronautics Act, R.S.C., 1985, chapter A-2, section 4.2.

Footnote 14 DGAEPM, Target versus Actual YFR, 26 June 2012.

Footnote 15 Ibid.

Footnote 16 RPP, 2011-12, Part III, Estimates.

Footnote 17 DPR, 2010-11, Part III, Estimates.

Footnote 18 RPP, 2011-12, Part III, Estimates.

Footnote 19 Agreement Between the Government of the United States of America and the Government of Canada on the North American Aerospace Defense Command (NORAD), 28 April 2006.

Footnote 20 http://www.nss.gc.ca/site/reports/nsp/2006plan/programplan_e.asp, accessed 1 November 2012.

Footnote 21 Ibid.

Footnote 22 Aeronautics Act, R.S.C., 1985, chapter A-2, section 4.2.

Footnote 23 Interviews, n=33, senior military officers and civilian staff at National Defence Headquarters, wings and bases.

Footnote 24 Interviews, n=38, senior military officers and civilian staff at A4 Maint, wings, bases and National Defence Headquarters.

Footnote 25 Ibid.

Footnote 26 Interviews, n=6, senior military officers and non-commissioned officers NCO.

Footnote 27 Interviews, n=20, senior military officers and NCOs.

Footnote 28 Ibid.

Footnote 29 Interviews, n=20, senior military officers and NCOs.

Footnote 30 Interviews, n=7, senior military officers and NCOs.

Footnote 31 Interviews, n=8, senior military officers and NCOs.

Footnote 32 Ibid.

Footnote 33 Ibid.

Footnote 34 Ibid.

Footnote 35 Interviews, n=22, senior military officers and NCOs.

Footnote 36 Ibid.

Footnote 37 Interviews, n=9, senior military officers and NCOs.

Footnote 38 Interviews, n=12, senior military officers and NCOs.

Footnote 39 Ibid.

Footnote 40 Ibid.

Footnote 41 Interviews, n=15, senior military officers, SAMAs, SAMEOs and NCOs.

Footnote 42 Ibid.

Footnote 43 This lack of third-level experience may also manifest itself many years from now. In many instances the private sector uses former and retired RCAF maintainers to do the work. If in the future the RCAF technicians do not gain in-depth experience and understanding of the aircraft, then the private sector will no longer have a ready pool of talent which it may poach for its operations, and in turn will not be in a position to offer support to the RCAF.

Footnote 44 Interviews, n=15, senior military officers, SAMAs, SAMEOs and NCOs.

Footnote 45 Ibid.

Footnote 46 Ibid.

Footnote 47 Ibid.

Footnote 48 Audit of Inventory Management: Surpluses & Disposal, August 2009; Audit of the Halifax-Class Modernization/Frigate Equipment Life Extension (HCM/FELEX) Project, March 2011; Audit of CP140 Optimized Weapon System Support (OWSS) Avionics Contract, January 2013.

Footnote 49 2011 Fall Report of the Auditor General of Canada, Chapter 5; 2008 May Report of the Auditor General of Canada, Chapter 2.

Footnote 50 Proper name: House of Commons Standing Committee on Public Accounts. Inventory control and disposal of obsolete items are mentioned as issues requiring attention in the Management Letters of 2008-2009, 2009-2010, and 2010-2011.

Footnote 51 Interview, DGAEPM.

Footnote 52 Interview, Directorate of Technical Airworthiness and Engineering Support (DTAES).

Footnote 53 Ibid.

Footnote 54 Interviews (n=7), WSMs.

Footnote 55 Interviews, n=15, military officers, NCOs, and civilian staff.

Footnote 56 Interviews, n=8, life cycle materiel managers (LCMM) and civilian staff.

Footnote 57 Interviews, n=10, Senior military officers, NCOs and civilian staff.

Footnote 58 Source: ADM(Mat) 25 January 2013.

Footnote 59 Interviews with NCOs at Cold Lake (CF188), Trenton (CC130 H&J), and Greenwood (CP140 and CC130H). The DRP initiative may address this.

Footnote 60 Ibid.

Footnote 61 Interviews, n=5, senior military officers, NCOs and civilian staff.

Footnote 62 g., the CP140 nose cone stored in an outside environment and thus being rendered unusable due to water infiltration. Interviews, n=5, military officers, NCOs and civilian staff.

Footnote 63 Article 2, Convention Establishing the World Intellectual Property Organization, Stockholm, 14 July 1967.

Footnote 64 Review of Acquisition for the Secretary of State for Defence, An independent report by Bernard Gray, October 2007.

Footnote 65 Interviews with major aerospace suppliers.

Footnote 66 Interviews, Industrial Marine Products, Halifax, Nova Scotia and CH149 WSM.

Footnote 67 Ibid.

Footnote 68 Interview CH149 WSM.

Footnote 69 Interviews, CC130J Project Office.

Footnote 70 Interview Aveos Fleet Performance Inc.; Contracting Trends in Acquisition, Defense AR Journal, September 2007; Performance-Based Service Contracting, United States Government Accounting Office (USGAO), September 2002; Implementing Performance-Based Services Acquisition, RAND, 2002; ISSCF for Canadian Forces Platforms, DND, July 2009; Next Generation Performance-Based Support Contracts, Department of Defence, Australia, February 2010; Performance-Based Management, Canadian Air Force Journal, Winter, 2011; Positively Awesome OWSM, Frontline Magazine, May/June 2005; Review of Acquisition, Bernard Grey,October 2009; Improving Service Acquisitions, USGAO, March 2002; Use of R&M Measures, Royal Australian Air Force, March 2007.

Footnote 71 Interviews with DGAEPM and CC177 WSM, and annual reports to AMC Planning Cycle. The CC177 WSM reported that Canada has four of approximately 270 C17s in the world-wide fleet. About 250 of these aircraft are flown by the United States Air Force (USAF) and Canada is part of the USAF maintenance agreement with Boeing. Similar to the parts sharing pools used by the major commercial airlines (e.g., Star Alliance), Canada is a member of the C17 parts pool for which it pays an annual membership fee. The parts pool allows Canada access to C17 spare parts anywhere in the world. This allows the CC177 fleet to meet its YFR with a minimum of unavailability.

Footnote 72 Interviews, Industrial Marine Products and CH149 WSM.

Footnote 73 Interview, Industrial Marine Products.

Footnote 74 Interviews, Industrial Marine Products and CH149 WSM.

Footnote 75 Interviews, Bell Helicopter Textron and CH146 WSM.

Footnote 76 Performance-Based Management, USAF, undated; Performance Based Contracting Handbook, Australia, 2006; Performance-Based Management Guidelines, DND, January 1999; Performance-Based Management Master Guidance, USAF, November 2005; Performance Management at Defence, DND, undated; Weapons Systems Support and Beyond – Performance Based Logistics, Aviation Aftermarket Defense, spring 2007.

Footnote 77 Interviews, Chief of Staff ADM(Mat), DGAEPM.

Footnote 78 Interviews, CC130J Project Office and Contractor.

Footnote 79 Interview, CC177 WSM.

Footnote 80 Performance-Based Management, USAF, undated; Performance Based Contracting Handbook, Australia, 2006; Performance-Based Management Guidelines, DND, January 1999; Performance-Based Management Master Guidance, USAF, Nov 2005; Performance Management at Defence, DND, undated; Weapons Systems Support and Beyond – Performance Based Logistics, Aviation Aftermarket Defense, spring 2007.

Footnote 81 Interview, CH149 WSM.

Footnote 82 Value Engineering and Service Contracts, Mandelbaum et al, Institute for Defense Analysis, June 2009.

Footnote 83 Ibid.

Footnote 84 Ibid.

Footnote 85 Analytic Challenges in Measuring Performance, USGAO, May 1997; Contracting Trends in Acquisition, Defense AR Journal, September 2007; Defense Contingency Contracting Officer Representative Handbook, Department of Defence, 2010; Implementing Performance-Based Services Acquisition, RAND, 2002; Improving Service Acquisitions, USGAO, March 2002; Guidance Needed for Using Performance-Based Service Contracting, USGAO, September 2002.

Footnote 86 Value Engineering and Service Contracts, Mandelbaum et al, Institute for Defense Analysis, June 2009.

Footnote 87 search of the TBS website produced only three examples of “value engineering,” none of which referenced aerospace maintenance. The three were a Management Accountability Framework assessment of Canada Border Services fixed infrastructure, an analysis of VECPs during the life of a real property construction project, and a review of value engineering in standard fire alarm systems.

Footnote 88 A-LM-505-001/AG-001 Integrated Logistic Support.

Footnote 89 W8485-04QH10/001/SSC (CC130H), W8475-00HG60/001-CSH (CH149), W8475-0711A03/001/USW (CC177).

Footnote 90 $12.9 billion over a fleet of 270 aircraft. Canada has four aircraft.

Footnote 91 YFR has several definitions:

  • Target YFR is determined by the RCAF in consultation with 1 CAD and is the number of hours the RCAF needs to fly to meet its mission. Target YFR is then used by DGAEPM to determine the amount of NP money required. If the NP allocation cannot fund the target YFR, or if there is a reduced number of aircraft available, the WSM will identify a supportable YFR.
  • Supportable YFR is the amount of YFR which may be funded with the NP allocation, or which may be supported with the existing fleet. That number is then fed back to the RCAF and 1 CAD as the flying time available. If the RCAF and 1 CAD decide that the supportable YFR is not adequate to meet the mission, additional funding must be found to increase the amount of supportable YFR.
  • YFR Flown is influenced by a number of factors which have no linkage to the NP funds allocated, such as aircrew availability, number of trained technicians, and weather. Providing more NP funds to certain fleets may not have any impact (or increase) on the YFR flown.

Footnote 92 DGAEPM, Target vs Actual YFR, 26 June 2012.

Footnote 93 DGAEPM.

Footnote 94 http://www.globalsecurity.org/military/systems/aircraft/c-130-mods.htm.

Footnote 95 Strategic Aerospace and Defence Initiative, Industrial Technologies Office, Industry Canada.

Footnote 96 Aerospace Industries Association of Canada, April 2009.

Footnote 97 Canadian Association of Defence and Security Industries, May 2009.

Footnote 98 PWGSC Strategic Review 2010, Acquisition Branch.

Footnote 99 W8485-04QH10/001/SSC (OWSM CC130H); W8475-00HG60/001-CSH (ISSCF CH149); W8485-07AN05/001/BB (OWSM CF188).

Footnote 100 Interview Standard Aero and Cascade Aerospace; notes from CDA Australia; Report: Next Generation Performance-Based Support Contracts – Achieving the Outcomes that Defence Requires, Department of Defence, Australia, February 2010.

Footnote 101 W8475-00HG60/001-CSH (ISSCF CH149).

Footnote 102 ISSCF contract for CC130J.

Footnote 103 Ibid.

Footnote 104 Ibid.

Footnote 105 Email, 21 December 2012.

Footnote 106 A database management system developed by DGAEPM. It was not supported for a period of time and data was not collected, but has recently been re-started by A4 Maint.

Footnote 107 Interviews, n=7, WSMs, military officers, and NCOs.

Footnote 108 Interviews, n=4, senior military officers and civilian staff.

Footnote 109 Interviews, n=9, military officers, fleet supply managers, CFSD staff and civilians.

Footnote 110 Interviews, n=3, CFSD staff.

Footnote 111 Site visit to CFSD, Montreal.

Footnote 112 Sources: A4 Maint and DGAEPM.

Footnote 113 Aircraft Maintenance Performance: The Effects of the Functional Decentralization of On-Equipment Maintenance, USAF, March 2001; Guidelines for Performance Measurement, Department of Defence, June 1996; Integrated Performance Measurement, University of California, November 1999; Measuring and Managing Performance in the RAF, BWMC Ltd, September 2007; Performance-Based Management Guidance, USAF, November 2005; A Strategic Approach to Service Acquisitions, USGAO, March 2002; Weapons Systems Support and Beyond – Performance Based Logistics, Aviation Aftermarket Defense, Spring 2007.

Footnote 114 Interviews, AVEOS Fleet Performance Inc., CC177 WSM.

Footnote 115 Interview, CF188 WSM.

Footnote 116 Interviews, CC130 WSM, CC177 WSM and CF188 WSM.

Footnote 117 Interview, CEO Air Canada.

Footnote 118 Interviews, n=12, WSMs, NCOs and civilian staff.

Footnote 119 Ibid.

Footnote 120 Ibid.

Footnote 121 Ibid.

Footnote 122 A “periodic inspection” is a scheduled preventive maintenance inspection performed at a fixed interval specified in number of calendar days and/or number of flying hours.

Footnote 123 DTAES.

Footnote 124 Aircraft are “staggered” for their periodic inspection so that only a small percentage of the fleet is unavailable at any one time.

Footnote 125 Interview, A4 Maint.

Footnote 126 Ibid.

Footnote 127 Interviews, A4 Maint and DGAEPM.

Footnote 128 After February 2010, the project was put on hold due to procurement challenges. The program, which had been initially designed, developed, and implemented by a contractor, was to be assigned to the Aerospace and Telecommunications Engineering Support Squadron. Implementation was never completed, and no handover occurred. It was reported that the project is set to re-start in FY 2012/13.

Footnote 129 Source: A4 Maint.

Footnote 130 TAT.

Footnote 131 Interviews, (n=3), senior military officers.

Footnote 132 Interviews, (n=4), military officers and L3 MAS staff.

Footnote 133 Interview, CC177 WSM.

Footnote 134 DGMPD.

Footnote 135 CC130H WSM.

Footnote 136 CH149 WSM.

Page details

Date modified: