Evaluation of Air Force Training and Readiness Part 1 – Air Force Initial Occupational Training

November 2012

1258-188 (CRS)

Reviewed by CRS in accordance with the Access to Information Act (AIA). Information UNCLASSIFIED.

Acronyms and Abbreviations

Results in Brief

This report presents the results of a Chief Review Services (CRS) evaluation of the Air Force initial occupational training program. The evaluation was undertaken between March 2011 and February2012 to examine program relevance and performance for the period 2007 to 2011 in order to inform future management decisions related to the program.

The evaluation was undertaken on behalf of the Chief of the Defence Staff (CDS) and Deputy Minister (DM) of the Department of National Defence (DND). While the DM and CDS are the primary “clients” for the program, the key program delivery partner is Chief of the Air Force Staff (C Air Force). Other DND Level Ones (L1) (e.g., Chief Military Personnel (CMP)) and industry partners (e.g., companies involved with the delivery of Canadian Forces (CF) Pilot training) have a collateral interest.

Background

Air Force initial occupational training is individual training that follows recruit and basic military qualification training and is tailored for each of the 26 Royal Canadian Air Force (RCAF) officer or non-commissioned member (NCM) occupations. This training is intended to confirm selection for the occupation and develop the basic skills necessary for employment at the individual’s first unit and/or for more advanced training. Typically initial occupational training takes several months, with the duration determined by the complexity and range of skills and knowledge required by the occupation.

The Air Force initial occupational training program is expected to produce the required numbers of appropriately trained RCAF personnel. The required numbers are identified in the Strategic Intake Plan (SIP) which is developed annually and reviewed at the Annual Military Occupational Review (AMOR). The SIP identifies recruitment targets and training output targets to maintain trained effective strength (TES) at the preferred manning level (PML) established for each occupation in light of training and occupational attrition rates.

Annual direct program spending is approximately $350 million. Pilot training represents 89percent of total program cost.

Methodology

The evaluation used multiple lines of evidence to ensure the reliability of reported results. The evaluation incorporated both qualitative and quantitative methods, including the following:

• document review;
• key informant interviews; and
• program data analysis.

Please refer to Section 1.2 for a detailed description of the methodology used during the course of this evaluation.

Findings and Recommendations

Finding #1: Relevance – Continued Need for Air Force Initial Occupational Training

Because of attrition from the RCAF, the lack of a source of appropriately trained personnel in the private sector for RCAF occupations, and limitations on the use of civilian staff and contractors in operational roles, there is a continued need for Air Force initial occupational training.

Finding #2: Relevance – Alignment with Government Priorities

The program is aligned with government priorities and DND strategic outcomes. The program is an essential element in ensuring RCAF personnel have the appropriate skills and knowledge and the RCAF is prepared for CF missions that support government priorities for security.

Finding #3: Relevance – Alignment with Federal Roles and Responsibilities

The program is aligned with federal roles and responsibilities. Where cost-effective, the program uses external training providers and other DND training programs to ensure there is no overlap or duplication with other programs or initiatives.

Finding #4: Performance (Effectiveness) – Sufficient Numbers of Trained Personnel

The program has produced the required numbers of trained personnel for 18 of the 26 RCAF occupations. Eight occupations remain “critical” in 2012 (i.e., TES is less than 90percent of PML) but in most cases this is due to recruiting issues and/or higher-than-expected attrition rather than a failure on the part of the program to meet output targets. The program has demonstrated flexibility and successfully adopted short-term solutions in order to conduct training surges when required for critical occupations. The notable exception is Pilots where shortfalls due to bottlenecks in the program are a long-standing problem without an apparent solution due to limitations in the long-term Pilot training contracts.

Recommendation

1. If changes to the Contracted Flying Training and Support (CFTS)/NATO Flying Training in Canada (NFTC) training syllabi have not improved Pilot production output by December 2013, fully investigate all options to achieve desired throughput, ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

OPI: C Air Force

OCI: VCDS

Finding #5: Performance (Effectiveness) – Quality of Trained Personnel

Although some occupations require further on-the-job training and experience at operational units to be fully qualified, the program has produced personnel with the appropriate preparatory skills and knowledge to absorb more advanced training or for initial employment at their first field unit.

Recommendations

2. Develop doctrine and update the Construction Engineer (Const Engr) officer Job-Based Occupational Specification (JBOS) to ensure the relevance and effectiveness of training.

OPI: C Air Force

3. Integrate Qualification Standards (QS) for RCAF Const Engr and Army Engineer (Engr) occupations to improve the efficiency of officer training at the Canadian Forces School of Military Engineering (CFSME).

OPI: C Air Force

Finding #6: Performance (Efficiency and Economy) – Governance Structures and Performance Measurement Frameworks

The 2009 creation of 2 Cdn Air Div provided a dedicated focal point for RCAF doctrine, training and education, and improved the governance of the initial occupational training program. Direction and guidance on initial occupational training is clear and consistent and has been effectively communicated. An effective performance measurement system is in place and provides an adequate and realistic assessment of progress toward expected outcomes.

Finding #7: Performance (Efficiency and Economy) – Appropriate, Efficient and Economical Program Delivery

The RCAF has demonstrated a commitment to increasing the efficiency and economy of initial occupational training, and CRS saw numerous examples of successful initiatives. Reducing failure rates in some occupations through more effective selection systems, including improved testing and elimination of the "first past the post” selection method, would improve the efficiency and economy of the program. An overarching strategy and doctrine to guide adoption of new technologies and training methods across the RCAF training community would ensure integration and coherence and improve the efficiency of these investments.

Recommendations

4. Introduce cognitive ability tests that include sensitive measures of spatial ability and other requisite competencies that would screen out in advance candidates who lack suitability for Aerospace Control Officer (AEC) training and employment.

OPI: C Air Force

5. Fully investigate the impact on AEC failure rates of the distance learning training methodology. If the lack of two-way communication during training is determined to be a key contributor to the high failure rate, repatriate the training to the Canadian Forces School of Aerospace Control Operations (CFSACO).

OPI: C Air Force

6. Develop an overarching strategy and doctrine to guide adoption of new technologies and training methods across the RCAF training community and ensure integration and coherence.

OPI: C Air Force

7. Procure AEC Instrument Flight Rules (IFR) and weapons training equipment that more accurately reflect equipment in operational use.

OPI: C Air Force

Finding #8: Performance (Efficiency and Economy) – Opportunities for Improvement

A number of opportunities were identified to improve efficiency and economy, including balancing recruiting targets and initial occupational training capacity to reduce the number of Personnel Awaiting Training (PAT); reviewing options for a more coherent aircrew selection system; assessing issues related to allophone training; reviewing options for resolving Hawk fatigue problems; ensuring sufficient Instructor Pilots (IP) are available to the flying schools; and moving CFSACO from NAV CANADA Training Institute facilities in Cornwall, Ontario, to Canadian Forces Base (CFB) Borden.

Recommendations

8. Balance the SIP recruiting targets and initial occupational training capacity to reduce the number of PATs as well as the time they wait to commence training.

OPI: C Air Force

9. Review options for a more coherent aircrew selection system.

OPI: C Air Force

10. Determine the extent to which issues related to allophone training and integration into the workforce exist and are understood in order to facilitate the development of appropriate strategies.

OPI: C Air Force

11. Review options for resolving Hawk fatigue problems to ensure sufficient aircraft continue to be available for the NFTC program.

OPI: C Air ForceOCI: VCDS

12. Ensure that the required IP military manning is identified and delivered through a sustainable career management process. Develop strategic-level guidance on the desired training methodologies (military personnel versus contracted personnel). Guidance must be provided at the strategic level to provide 2 Canadian Forces Flying Training School (CFFTS), 3 CFFTS and 419 Squadron with the required numbers of IPs at the right time to optimize their effectiveness.

OPI: C Air Force

13. Move CFSACO from NAV CANADA Training Institute facilities in Cornwall, Ontario into existing infrastructure at CFB Borden at the earliest practical opportunity.

OPI: C Air Force

14. Attribute all costs associated with RCAF Pilot training to “wings” standard, including the CFTS contract, to Program Alignment Architecture (PAA) sub-sub-activity 1.2.2.2 Initial Individual Occupation Training.

OPI: C Air Force

Finding #9: Performance (Efficiency and Economy) – Resource Allocation and Expenditure

Training resources are allocated and expended as intended. When the need for adjustments to allocated resource levels is identified, the review, approval and oversight processes in place ensure that resources are fully rationalized and utilized appropriately.

Note: Please refer to Annex A—Management Action Plan for the management response to the CRS recommendations.

1.0 Introduction

This report presents the results of an evaluation of initial occupational training for the 26 RCAF-managed occupations, a program that represents a total annual direct program spending of approximately $346.65 million. The evaluation was undertaken by CRS on behalf of the CDS and DM of DND, and examined program relevance and performance for the period 2007 to 2011 in order to inform future management decisions related to the program. The evaluation was conducted in accordance with the DND/CF Evaluation Work Plan for fiscal year (FY) 2010/11 and DND’s PAA.

While the DM and CDS are the primary “clients” for this program, the key program delivery partner is Comd RCAF. Other DND L1s (e.g., CMP) and industry partners (e.g., companies involved with the delivery of CF Pilot training) have a collateral interest.

This report represents Part 1 of a comprehensive two-part evaluation encompassing all aspects of Aerospace Training and Readiness. Part 1 examines RCAF initial occupational training while Part 2 will examine RCAF advanced training and readiness.

1.1 Profile of the Air Force Initial Occupational Training Program

1.1.1 Background

Air Force initial occupational training is individual training that follows recruit and basic military qualification training and is tailored to one of 26 RCAF officer or NCM occupations. This training is intended to confirm selection for the occupation and develop the basic skills necessary for employment at the individual’s first unit and/or for more advanced training. In essence, newly winged pilots, junior officers, and apprentice level technicians who complete initial occupational training have received sufficient training to be assigned to their first unit, but require follow-on training on specific aircraft, aerospace systems or construction engineering functions in order to be effectively employed.

Much of the more advanced training is provided at operational training units (OTU) and operational training squadrons (OTS) or back at the technical school that provided the initial training. Typically initial occupational training takes several months, with the duration determined by the complexity and range of skills and knowledge required by the occupation.

1.1.2 Objectives of the Program

The Air Force initial occupational training program is expected to produce the required numbers of appropriately trained RCAF personnel. The required numbers are identified in the SIP which is developed annually and reviewed at the AMOR meetings. The SIP identifies recruitment targets and training output targets to maintain TES at the PML established for each occupation in light of training failure rates and attrition rates.

The intended outcome of the program is RCAF personnel with appropriate skills and knowledge that contribute effectively to CF readiness and operations.

The activities, outputs and intended outcomes of the program are illustrated in the program logic model, shown in Annex B.

1.1.3 Delivery Approach

RCAF personnel receive their initial occupational training at training establishments across Canada. Most of these schools are managed by the RCAF, while a small number of occupations are trained at schools belonging to the Army or CMP in the interest of efficiency and economy, and occasionally through external providers (such as community colleges and private sector contractors) where there are clear efficiency benefits. Depending on the phase of training, instruction at RCAF training establishments is provided by either RCAF or civilian instructors. Flying training is delivered through two long-term contracts: NFTC, a $3.4-billion, 20-year contract with Bombardier Aerospace Military Aviation Training; and CFTS, a $1.77-billion, 22-year contract with a consortium headed by Kelowna Flightcraft Limited under the name Allied Wings.

Training is provided through a variety of means including the traditional classroom approach, but also increasingly through training methodologies that exploit technology, such as simulation and e-learning (including distance learning).

1.1.4 Program Spending

As shown in Table 1, annual direct program spending is approximately $346.65 million. Pilot training represents 89 percent of total program cost.

Table 1. Program Spending FY 2011/12 (millions). This table shows the total costs of initial occupational training for all Air Force occupations.

1.2 Methodology

The evaluation scope and methodology was set out in an evaluation work plan developed during the planning phase completed prior to the commencement of the evaluation. The evaluation planning phase was undertaken between March and May 2011. The evaluation work plan aligns with the federal government policy on evaluation.

1.2.1 Evaluation Objective and Scope

The objective of the evaluation was to examine the relevance and performance of the Air Force initial occupational training program for the period 2007 to 2011 in order to inform future management decisions related to the program. The evaluation examined training and related activities for all 26 Regular Force officer and NCM occupations for which C Air Force is the career field and occupation managing authority.⁵ For example, the evaluation examined initial pilot training as delivered under the CFTS and NFTC contracts, but not beyond, and technician training to a qualification level where personnel can be posted to their first employment on an air wing as an apprentice (e.g., Qualification Level 3 (QL3) for most NCM occupations).

Some of the RCAF NCM occupations are supervisory, such as Aircraft Maintenance Superintendent (AM Sup) and Construction Engineer Superintendent (CE Supt). Because these senior personnel are already technically trained and experienced in a “feeder” occupation, their initial occupational training was examined in lesser detail.

Military occupations that participate in or support air operations but are not managed by the RCAF (e.g., Traffic Technician, Logistics Officer) are outside the scope of this evaluation.

Advanced training specific to particular aircraft, aerospace systems, or engineering functions delivered at OTUs, OTSs and other CF training establishments, as well as collective training to meet readiness requirements, are also outside the scope of this report but will be covered in Part 2 of the evaluation.

1.2.2 Evaluation Issues and Questions

The evaluation of the Air Force initial occupational training program examined issues related to relevance and performance. See Annex C for the complete evaluation matrix as well as indicators and line of evidence for each evaluation question.

Relevance

Evaluation Questions

• Is there a continued need for the program?
• Is the program aligned with government priorities?
• Is the program aligned with DND strategic outcomes?
• Is the program aligned with federal roles and responsibilities?
• Is there overlap or duplication with other programs or initiatives?

Performance (Effectiveness)

Evaluation Questions

• Has the program produced the required number of trained personnel?
• Has the program produced appropriately trained and qualified personnel?

Performance (Efficiency and Economy)

Evaluation Questions

• Are there effective governance structures and performance measurement frameworks in place to manage delivery of the program?
• Are the most appropriate, efficient and economical means used to produce program outputs?
• Are there opportunities to improve efficiency and economy?
• Are resources allocated and expended as intended?

1.2.3 Data Collection Methods

The evaluation of the Air Force initial occupational training program included the use of multiple lines of evidence and complementary research methods as a means to help ensure the reliability of information and data collected. The following data collection methods were used to gather qualitative and quantitative information for the evaluation:

• document review;
• key informant interviews; and
• program data analysis.

1.2.3.1 Document Review

To obtain an informed appreciation of Air Force training and readiness, and in preparation for subsequent analysis, a comprehensive review of available literature, internal documentation, applicable policies, QS, Course Training Plans (CTP), Course Training Standards (CTS), and performance reports was undertaken. The review of background documents identified how Air Force initial occupational training has evolved to meet emerging requirements (e.g., new aircraft fleets and advances in technology and pedagogy).

Document review was a key methodology for assessing program relevance. Documents such as the Speech from the Throne, the Canada First Defence Strategy (CFDS) and DND Report on Plans and Priorities were reviewed to confirm alignment of the program with government priorities, departmental strategic outcomes and federal roles and responsibilities.

An extensive review of program documentation was conducted to assess program management practices, governance structures, performance measurement frameworks and training validation systems.

1.2.3.2 Key Informant Interviews

Key informant interviews served as an important source of information for assessing program performance. Over 50 interviews were conducted, including a number conducted as focus groups, with larger numbers of participants in attendance. These interviews and focus groups provided context to the document review and program data analysis as well as qualitative input on the evaluation questions.

In order to gain a full appreciation of the issues from the perspective of a diversity of stakeholders, a broad range of stakeholders was interviewed, including members of the RCAF senior leadership cadre, those directly involved in Air Force training management and delivery, and others, both military and civilian (including contracted support providers), who support Air Force training in various ways.

Interviews and discussions with the senior leadership cadre and training staff as well as contracted support providers were conducted during site visits. Focus group sessions were held with staff in some locations to gather confirmatory evidence to support the analysis of program data.

1.2.3.3 Program Data Analysis

Program data on student intake, CTSs, CTPs, pass/fail rates, PAT statistics, training delays, training output, and gaps between PML and TES were analyzed to assess program effectiveness. Training records, actual training costs, training standards and training validation information were collected during site visits to representative training establishments and HQ units responsible for the management of training.

1.2.4 Limitations

The evaluation methodology was designed to provide multiple lines of evidence in support of evaluation findings. The data and information were collected to respond to the evaluation questions and issues. As in all evaluations, there are limitations and considerations that should be noted.

The evaluation plan originally included a comparative analysis of training costs with allied nations. However, due to financial data limitations, it was not possible to calculate training costs per student for most occupations. Furthermore, as a result of differences in occupational structure and training between countries, direct, across-the-board comparisons to allied nations were not possible for most occupations.

Given the wide scope of RCAF training activities undertaken each year (i.e., over 1,200 individual training courses), all 26 MOSIDs were examined; however, much of the effort for this evaluation was concentrated on an analysis of those initial occupational training courses and training events that are the most costly and/or present the highest risk for follow-on advanced training and the development of RCAF capabilities to meet readiness requirements.

In the interests of offering a balanced view of initial pilot training, which is delivered via contracts with two prime contractors, CRS sought to interview CFTS Allied Wings contractor staff in Southport, Manitoba, and NFTC Bombardier Military Aircraft Training (BMAT) staff in Moose Jaw, Saskatchewan. While Allied Wings staff was made available to meet with CRS, BMAT staff in Moose Jaw was not authorized to meet with CRS. Consequently CRS data gathering with respect to NFTC training issues at the local level was limited to DND/CF and Public Works and Government Services Canada perspectives.

2.0 Evaluation Findings

2.1 Relevance

The following section examines the extent to which Air Force initial occupational training addresses a demonstrable need, is aligned with government priorities and DND strategic outcomes, and is aligned with federal roles and responsibilities.

2.1.1. Continued Need for Air Force Initial Occupational Training

The RCAF is responsible for all aircraft operations of the CF, enforcing the security of Canada's airspace and providing aircraft to support the missions of the Royal Canadian Navy and the Canadian Army. The RCAF is a partner with the United States (US) Air Force in protecting continental airspace under the North American Aerospace Defense Command (NORAD). The RCAF also provides all primary air resources to the National Search and Rescue Program.

In order to carry out these responsibilities, the RCAF requires manned and unmanned aerospace vehicles as well as pilots, other types of operational personnel and maintenance crews. To a certain extent, maintenance can be done by civilian maintenance staffs, and the RCAF was one of the first of the Canadian armed services to employ civilian maintenance staffs in significant numbers on their principal operational platforms through outsourcing and contracting for services in an attempt to reduce costs and numbers of uniformed personnel. Given legal and operational liabilities, some operational capabilities cannot be considered for contracting out.

The RCAF employs 26 Regular Force officer and NCM occupations. PMLs are set for each of these occupations in light of appropriate use of civilian staff and contractors. Attrition rates for these occupations average approximately 7.4 percent annually; consequently, there is a continuing need to replace military personnel who retire or leave the RCAF for other reasons.

RCAF occupations are highly specialized and in many cases there is no counterpart outside the CF. Therefore, there is no source of appropriately trained personnel and there is a continuing need for RCAF initial occupational training to ensure sufficient numbers of pilots, operational personnel and maintenance crews to fulfill RCAF mandates.

2.1.2 Alignment with Government Priorities

The CFDS, released by the Government of Canada in 2008, identifies six core missions that the CF must be prepared to conduct within Canada, in North America, and globally, at times simultaneously. Specifically, the CF must have the capacity to perform the following:

• conduct daily domestic and continental operations, including in the Arctic and through NORAD;
• support a major international event in Canada, such as the 2010 Olympics;
• respond to a major terrorist attack;
• support civilian authorities during a crisis in Canada such as a natural disaster;
• lead and/or conduct a major international operation for an extended period; and
• deploy forces in response to crises elsewhere in the world for shorter periods.

The CF role in supporting government priorities for security was re-iterated in the 2011 Speech from the Throne that stated that “The Canadian Armed Forces play a crucial role in defending our sovereignty and national security.” The Speech from the Throne also drew attention to the CF role in helping promote Canadian values and interests at home and abroad.

In order for the CF to be prepared to carry out the missions assigned to it and support government priorities for security, the RCAF must be ready to employ air power, as and when required. To do so requires that the RCAF have the right numbers of appropriately trained and qualified personnel. Meeting this goal starts with ensuring that RCAF personnel have the foundational skills and knowledge to undertake more advanced and specialized training.

The Air Force initial occupational training program is aligned with the Recruiting of Personnel and Initial Training Program Activity within DND’s PAA and contributes to the “Resources are acquired to meet Government Defence Expectations” strategic outcome.

The FY 2012/13 DND Report on Plans and Priorities identified four organizational priorities where efforts and resources will be directed to mitigate risks and address gaps in capability or capacity:

• ensuring sustainable operational excellence both at home and abroad;
• reconstituting and aligning the CF post-Afghanistan;
• strengthening the Defence Team; and
• maintaining Defence affordability.

The first two priorities involve focussing on fulfilling the responsibilities identified in the CFDS while the third priority includes “recruiting, developing and sustaining under-strength military and civilian occupations.” A number of RCAF occupations are under-strength and the initial occupational training program plays a key role in producing the required numbers of appropriately trained personnel.

2.1.3 Alignment with Federal Roles and Responsibilities

The 1867 Constitution Act describes how powers and authorities are to be divided between the federal government and provincial governments. The Act gives authority for defence and the military to the Crown. In Canada, the powers and authorities of the Crown have been delegated to the Governor General of Canada who acts on the advice of the cabinet. Subsequently, the National Defence Act established the CF and gave the Minister of National Defence authority to manage and direct all matters relating to national defence and the CF.

Because defence and the CF are a federal responsibility, ensuring RCAF personnel have the appropriate skills and knowledge to fulfill their mandate is also a federal responsibility. The CFDS defines the roles the CF and RCAF are expected to play and provides the basis for determining the skills and knowledge required for RCAF officers and NCMs.

While RCAF initial occupational training is a federal responsibility, DND does not necessarily have to deliver the training. Over the past ten years there have been efforts to determine the best mix of DND/CF owned and operated institutions. Where it is cost-effective, training is provided by external providers such as community colleges and the private sector under contract. In addition, training is co-ordinated internally with a small number of RCAF occupations trained at schools belonging to the Army or CMP in the interests of efficiency and economy.

2.2 Performance (Effectiveness)

The following section examines the effectiveness of the RCAF initial occupational training program by assessing whether it has achieved its intended results over the course of the evaluation period.

2.2.1 Expected Outputs – Quantity

The Air Force initial occupational training program is expected to produce the required numbers of appropriately trained RCAF personnel. The required numbers are identified in the SIP which is developed annually and reviewed at the AMOR meetings. The SIP identifies recruitment targets and training output targets to maintain TES at the PML established for each occupation in light of anticipated training failure rates and attrition rates. Figure 1 depicts personnel flow from recruitment to departure from the RCAF.

Figure 1. Personnel Flow. The careers of CF personnel flow from recruitment through corps/branch training, to employment and release from the CF.

The following indicators were used to assess the quantity of outputs:

• percent gap between TES and PML; and
• stakeholder satisfaction with the quantity of graduates.

2.2.1.1 Gap between TES and PML

Table 2 shows the health status of each of the 26 RCAF occupations. An occupation is considered “critical” (red) if the variance between the PML and the TES is 10 percent or greater.⁶ While the initial occupational training program is producing enough trained personnel to keep 18 occupations out of the critical category, eight of the 26 RCAF occupations were considered critical as of 15 February 2012.

Of the eight “critical” occupations, the six occupations of most concern and having the most impact on operational effectiveness were evaluated in depth to identify the reasons for the shortfall including: Pilot, Construction Engineer (Const Engr), Airborne Electronic Sensor Operator (AES Op), Aerospace Telecommunications and Information Systems Technician (ATIS Tech), Flight Engineer (Flt Engr), and Air Weapons Systems Technician (AWS Tech).

Table 2. RCAF Occupational Status (Effective 15 February 2012). The table indicates a snapshot in time of the TES/PML status of all RCAF-managed MOSIDs.

Pilot

Pilots, which are one of the largest RCAF occupations and arguably the most influential, had a shortfall of 253 as of 15 February 2012. The evaluation found that there has been a shortfall of between 200 and 250 for the past ten years and there are no indications this problem is being resolved. Bottlenecks in the RCAF contracted flying training program were found to be the key reason for the shortfall.

The CF has contracted out most of the Pilot training system in the form of two long-term contracts:

• CFTS, which includes a contractor-led Phase I Initial Flying Training/Pilot Screening program, plus a contractor-supported Phase III rotary wing and multi-engine training program at 3 CFFTS at Southport, Manitoba. The $1.77-billion, 22-year contract, which expires in 2027, is held by a consortium headed by Kelowna Flightcraft Limited under the name Allied Wings. The contract was amended in 2008 for a Phase II Grob to alleviate some of the training backlog being experienced in Phase IIA of the NFTC program.
• NFTC is a $3.4-billion, 20-year contract which expires in 2021, with BMAT. This contract includes Phase IIA, Basic Flying Training, originally for all CF pilots, Phase IIB for jet pilots, and Phase III, Advanced Jet Training at 2 CFFTS in Moose Jaw, Saskatchewan. The contract also includes Phase IV, Fighter Lead-In Training (FLIT) at Cold Lake, Alberta.

As depicted in Figure 2, RCAF pilot candidates are trained to “wings” standard in one of three streams; jet fighter, multi-engine and rotary-wing. The Undergraduate Pilot Training System (UPTS) begins with identification as a pilot candidate at a CF recruiting centre and testing at the Canadian Forces Aircrew Selection Centre (CFASC)at 8 Wing, Trenton, Ontario to confirm selection. Successful candidates proceed to Initial Flying Training/Pilot Screening (Phase I) at 3 CFFTS, followed by Basic Flying Training (Phase IIA) either through NFTC or through CFTS (Phase II Grob). Graduates then continue with either jet training (Phase IIB) through NFTC, or multi-engine or rotary wing pilot training through CFTS. Initial occupational training continues with Phase III on fighter jets, multi-engine planes or rotary wing equipment. “Wings” standard is obtained at the end of Phase III or equivalent. Fast jet students continue with Phase IV, Jet Transition Training and Phase IV, FLIT at Cold Lake, Alberta.

Figure 2. Pilot Training System. This flowchart illustrates the pilot training continuum from recruitment through to selection, primary and basic flying training, specific aircraft type training, and graduation from the CFTS/NFTC programs.

Bottlenecks in the Pilot training system are limiting the initial occupational training program output and increasing CF pilot training costs. Depending on the time of year, pilot candidates who have successfully passed the testing at CFASC can wait from three to nine months to attend Phase I. However, the principal bottleneck in the Pilot training system is Phase IIA – Basic Flying Training, conducted through NFTC. Wait times have been consistently more than 12 months for the last five years. Direct Entry Officer (DEO) student pilots awaiting training can cost up to $50,000 per year in salary. As a result of training delays between various phases, it can now take four years to graduate a pilot.

The Phase IIA bottleneck is due to limitations of the existing flying training contracts, most notably the NFTC contract. The NFTC contract stipulates that there are to be 131 Canadian student starts on Phase IIA each year and assumes that there will be 192 flying days available per year. However, this assumption has been proven to be incorrect and there is no flexibility in the NFTC contract to overcome flying days lost to weather or aircraft availability issues and allow graduation within specified course timings to meet output targets. From 2001 to 2010, Initial Flying Training output and student pilot starts have averaged less than 74 percent of the contracted goal.

Throughout the NFTC contract period, the bottlenecks have come under frequent scrutiny from all levels in the RCAF. Phase II Grob was initiated at CFTS for RCAF rotary wing and multi-engine student pilots in February 2010 in order to take some of the burden off this portion of NFTC training and reduce wait times.

Despite this additional capacity, backlogs persist and contract limitations preclude resolution of the problem. The RCAF contracted flying training program has never met its output targets for newly winged graduates. No amount of “tinkering around the edges” with the existing contracting regimes will allow the RCAF operational community to obtain the number of pilot graduates required to meet their longstanding operational pilot needs. Without re-opening the existing contracts, acquiring more aircraft and simulator assets, and providing additional CF pilot instructors, only minor progress towards producing intended program output quantities is possible. The RCAF is currently in the midst of making changes to flying training programs, including changes to the flying hours dedicated to each Phase to try to increase throughput.

In addition to bottlenecks in the RCAF contracted flying training program, the evaluation found pilot selection issues may also have contributed to the output shortfall. The CF has no age limit on pilot candidates and CMP/Canadian Forces Recruiting Group (CFRG) uses a “first past the post” selection method whereby the first applicants who meet the selection criteria are accepted for training. The alternative would be to assess all candidates who apply during a specified time period and select the best. The RCAF pilot training success rate from selection (after testing at CFASC) to “wings” graduation is approximately 59 percent compared to 85 percent for the Royal Air Force (RAF) which has an age limit of 23 for pilot candidates.⁷ Low success rates reduce initial occupational training program output and increase the cost of pilot training. Possible contributors to low success rates include the “first-past-the-post” selection method and the lack of an age limit. Promising steps have been taken recently to replace the “first past the post” selection method with one that selects the best overall candidates. This change has yet to be fully implemented by the CFRG.

At the heart of the CFASC Pilot testing capacity is the Canadian Automated Pilot Selection System (CAPSS), consisting of five flight simulators plus a Data Analysis Centre. CAPSS is now 15 years old, uses outdated and exceptionally limited central processors, and spare parts are becoming scarce. The simulator is also based on an analogue system which has proven challenging for some of the younger candidates who have no experience with this technology. The automated processing/scoring system has also been noted for generating some scoring anomalies, which has caused students to be retested to provide confirmation of scores. While CFASC has the capacity to take in 15 students in each of the 42 available weeks per year, 12 students per week is the preferred level to allow for simulator problems and to work with struggling students. Reduced testing capacity is not currently affecting initial occupational training program output because Pilot recruiting has been purposely slowed in order to catch up with PAT backlogs accumulated over the past decade.

Construction Engineer

Engineering occupations differ from the other RCAF-managed occupations in that training is provided at Army schools. Const Engrs are trained at CFSME. The evaluation found that the shortage of Const Engrs was primarily due to insufficient intake, not issues with the initial occupational training program. Steps are being taken to address the shortage and it is anticipated that the occupation will no longer be critical by March 2013.

The evaluation found that allophone officer trainees at CFSME, including Air Force Const Engr candidates, had a higher failure rate on their initial occupational training course than those whose mother tongue is either English or French. It is unclear to training staff at CFSME whether this is a language issue or a cultural issue, given that the weaknesses become most evident in leadership roles.

Airborne Electronic Sensor Operator

The CRS evaluation team reviewed training records and interviewed Comd 17 Wing, as well as 2 Cdn Air Div HQ senior staff to determine whether there were any AES Op training issues requiring attention. No major AES Op training issues were identified.

Aerospace Telecommunications and Information Systems Technician

Training for ATIS Techs is provided at the CFSCE, an Army school. C Air Force is responsible for funding additional courses that may be required to meet PML targets. C Air Force funded the hiring of three contracted instructors for a training surge commencing in November 2011 to address an influx of new recruits and reduce the number of ATIS Tech PATs. CFSCE had sufficient equipment and infrastructure to support the training surge and demonstrated the ability to ramp up ATIS Tech QL3 training, resulting in graduation of record numbers of students.

Flight Engineer

Flight Engineer (Flt Engr) candidates are already-serving RCAF members who are drawn from the AVN Tech or AVS Tech occupations or via component transfer from the Reserve Force. The occupation is in the midst of a transition that will see it eventually managed under the Air Maintenance umbrella. While the occupation’s status is currently “critical,” the RCAF is increasing training throughput and expects the occupation to regain health post-March 2013.

Air Weapons Systems Technician

The RCAF aircraft maintenance community (which includes AWS Techs) has completely rebuilt its apprentice and journeyman workforce in recent years (including re-introducing the AWS Tech occupation) and re-shaped its training. The recent Air Occupation Enhancement (AOE) initiative and Air Technician Transformation (ATT) has shortened training and certification times by making training scalable, sharable and deliverable anywhere with electronic delivery, modeling and simulation. With these two initiatives, air technician training times have been reduced from 42 months to 24 months. These improvements are expected to improve the health status of the occupation.

2.2.1.2 Stakeholder Satisfaction with Quantity of Outputs

In addition to assessing the gap between TES and PML, RCAF stakeholder satisfaction with the quantity of graduates was also assessed. Despite shortfalls in some occupations, which in many cases are a recruiting, and/or retention issue rather than an inability on the part of the training program to meet production targets, testimonials from senior RCAF leadership and other key informants indicate satisfaction with the quantity of graduates from the initial occupational training program with the exception of newly winged pilots. Where surge requirements arise as a result of increased recruitment to offset greater attrition, short-term solutions (such as scheduling additional course serials or employing contracted support to increase training capacity) adopted by training establishments provide flexibility and are utilized effectively.

Pilot shortfalls due to training program bottlenecks are a long-standing problem and the source of much dissatisfaction among RCAF stakeholders. The contracted pilot training programs (CFTS/NFTC programs) are only funded for approximately 105 newly winged graduates per year (following a funding increase and contract amendments in 2008) but have never met that target. Since 2008, RCAF senior leadership has consistently expressed dissatisfaction with both actual output and the production target and has called for an increase to 125 graduates per year.

2.2.2 Expected Outputs – Quality

The following indicators were used to assess the quality of outputs:

• up-to-date QSs;
• linkages between training, personnel strategy and career management;
• processes to capture lessons learned from operations and assess their implications for training;
• processes to obtain timely feedback from clients;
• processes to address training deficiencies and gaps noted by clients;
• time required to make changes to training content; and
• stakeholder satisfaction with the quality of graduates.

2.2.2.1 Up-to-date Qualification Standards

The Canadian Forces Individual Training and Education System (CFITES) is the management system used to control Individual Training and Education (IT&E). This framework is designed to optimize the quality and quantity of IT&E while minimizing the required resources. The CFITES model requires qualification standards to be reviewed on a five-year cycle which is a challenge for 2 Cdn Air Div HQ staff, given the available resources. Of the 716 Air Force QSs existing in October 2011,⁸ only 249 (35 percent) were both published and up to date (reviewed within the last five years). Comd 2 Cdn Air Div has made it a priority to address the gap.

The Const Engr officer JBOS has not been updated since the creation of the occupation in the mid-1990s, with the split of Air Force and Army Engrs, and doctrine is lacking. As a result, direction on the capabilities the Const Engr occupation should bring to the CF remains unclear, a factor that complicates the development of training content, as well as performance measurement.

Many capabilities are shared between RCAF Const Engr and Army Engr occupations and training has been integrated at CFSME in order to increase interoperability of RCAF officers who often find themselves in support of Army operations. However, the responsibilities for the QSs for these two occupations reside in two different organizations and CFSME will often get performance objectives (PO) for the same tasks that are sufficiently different to result in slightly different courseware and training requirements. Maintaining two or more sets of lesson plans and other material to teach essentially the same subject is resource intensive. For this reason CFSME has standardized the lessons and they are delivered to both Air Force and Army occupations. Training delivery would be simplified by ensuring alignment at the QS level, where applicable.

2.2.2.2 Linkages between Training, Personnel Strategy and Career Management

The AMOR is an annual military personnel management process to assess the health of CF occupations. The process involves occupational authorities, branch advisors, career managers, recruiting program managers, and training program managers. Issues such as trends in attrition rates, rank-to-rank ratios, SIP recruiting and training output targets, career paths and other issues affecting the health of the occupation are discussed to ensure appropriate linkages between training, personnel strategy and career management.

2.2.2.3 Processes for Obtaining Feedback and Making Improvements

Through interviews with RCAF senior leadership and training staff, the evaluation confirmed that processes are in place for capturing lessons learned from operations, obtaining feedback from clients and making changes to address deficiencies and gaps. The content and duration of initial occupational training courses is the subject of extensive ongoing review and discussion within the Air Force’s command and training hierarchy.

Training validation is the final phase of the CFITES Quality Control System, and is designed to verify that graduates are adequately prepared to meet specified departmental goals. Validation can also identify excessive or unnecessary training that impairs efficiency, and should be conducted on a regular basis to ensure that programs are kept up-to-date with changes in operational requirements and Departmental goals.

All Air Force initial occupational training courses conclude with a student evaluation of the course and the instructors, as well as an instructor’s review of the course and its content. Periodically, a member of the training unit’s staff will sit in on classes and report on course delivery and training standards. However, program staff pointed out that formal, systematic validation of all Air Force initial occupational training courses has not been undertaken due to a lack of resources. In partial compensation, other, less formal methods help validate training. For example, the leadership at training establishments confirmed that they and their staff receive beneficial feedback from force generators, field units and former students regarding course content and the level of preparedness of graduates for advanced training and employment. Regularly scheduled training conferences and meetings held throughout the year provide a mechanism for the exchange of validation information.

Interviewees for this evaluation, including senior leaders and force generators, as well as those involved with training management, delivery or support, were asked to provide an assessment of the appropriateness of the duration of initial occupational training. The overall assessment was that Air Force initial occupational training courses and training events have adequate time built in for trainees to be taught, to practice, and to be assessed against QSs, and that adjustments to content and duration are made on a timely basi where required to reflect the introduction of new fleets, equipment or technologies and lessons learned in the field.

2.2.2.4 Stakeholder Satisfaction with the Quality of Graduates

The initial occupational training program performance measurement framework includes qualitative indicators used to measure the satisfaction of students, instructional staff and “clients” with course content, relevance and training. These indicators reflect a high level of satisfaction with the program. In addition, through interviews with the senior RCAF operational leadership cadre, CRS confirmed that Air Force personnel who have successfully completed initial occupational training are typically seen to have the appropriate preparatory skills and knowledge to absorb more advanced training or for initial employment at their first field unit.

The one exception to this conclusion is the SAR Tech occupation. SAR Tech training is conducted at the CFSSAR at 19 Wing, Comox, BC. No CF aircraft have been made available for training due to resource constraints, and SAR Tech training is dependent on contractor support to provide the aircraft for fixed-wing para-rescue training. The contractor aircraft are considerably smaller than the CC-115 Buffalo and CC-130 Hercules that are used operationally by SAR squadrons. As a result SAR Tech trainees undergoing initial training do not gain experience working as part of a full SAR aircrew team and require additional training at the squadrons before operational employment.

2.3 Performance (Efficiency and Economy)

The following section provides the findings with respect to whether the RCAF initial occupational training program is demonstrating efficiency and economy.

2.3.1 Governance Structures and Performance Measurement Frameworks

The following indicators were used to assess governance structures and performance measurement frameworks:

• clear, consistent and accountable governance structures; and
• effective performance measurement frameworks.

2.3.1.1 Governance Structures

Governance “encompasses the policies and procedures used to direct an organization’s activities to provide reasonable assurance that objectives are met and that operations are carried out in an ethical and accountable manner.”⁹ Governance is exercised through a management framework, with clear accountabilities and clear and consistent policies, direction and guidance.

Prior to 2009, Comd 1 Cdn Air Div was responsible for individual training and collective training, both foundational and in support of readiness, as well as all aspects of force generation and force employment in an Air Force context. In 2009, Comd 2 Cdn Air Div was created and delegated oversight of Air Force IT&E, including initial occupational training and core Air Force developmental coursing and related support, in order to reduce the span of control of Comd 1 Cdn Air Div and focus efforts on RCAF doctrine, training, and education. Air Force stakeholders interviewed for the evaluation shared a consistent view that, despite the fact that 2 Cdn Air Div is still an embryonic organization that continues to evolve, training management has improved, and intended outcomes and training objectives are being achieved.

Through document review and interviews with key stakeholders, the evaluation considered the extent to which the direction provided by Comd 2 Cdn Air Div to subordinate training units, including Wings, has been clear, consistent, timely and communicated effectively. The evaluation found direct linkages between key Air Force directives and supporting documents, and the individual training being conducted. In addition, the business planning process has been used effectively to provide specific direction and guidance to unit commanders and to identify priorities.

2.3.1.2 Performance Measurement Frameworks

A review of the program’s performance measurement framework was undertaken by CRS to assess its effectiveness. The evaluation found that at the operational and tactical levels, performance measurement is recognized as a key activity to ensure that expected outcomes are achieved, and to support continuous improvement. CRS observed that measurable indicators have been developed to allow training program managers and administrators to assess performance against expected results and analyze trends. The quantitative indicators in place typically measure productivity against operational requirements for each occupation through indicators such as course graduation numbers, pass/fail rates and numbers of PATs. Qualitative indicators are also included in the performance measurement framework and are used to measure the satisfaction of students, instructional staff and “clients” with course content and training delivery and to help identify necessary improvements.

2.3.2 Appropriate, Efficient and Economical Program Delivery

The following indicators were used to assess appropriate, efficient and economical program delivery:

• pass/fail rates;
• unit costs;
• processes to identify areas requiring improvement, develop improvement plans and implement changes;
• processes to identify cost-effective alternatives;
• adoption of cost-effective alternatives; and
• use of up-to-date training delivery approaches, practices and technologies.

2.3.2.1 Pass/Fail Rates

High failure rates have direct consequences for the efficiency and economy of the initial occupational training program, including reduced output of graduates, extra instructional workload for training staff, and increased administrative burden for training organizations and HQ staff to manage the disposition of failed candidates. There is also a human cost associated with training failures. The unsuccessful individual has to contend with limited occupation reassignment options or potential release from the CF, after having dedicated a significant amount of time and effort training for a career in the RCAF.

As discussed in Section 2.2.1.1, the RCAF Pilot training success rate is significantly lower than the success rate in the RAF possibly due to the “first-past-the-post” selection method and the lack of an age limit for pilot candidates.

The AEC occupation also experiences high failure rates. AEC employment is a highly complex, cognitively loaded, skills-based occupation requiring particular competencies. It requires a high level of cognitive and spatial aptitude to be able to visualize an evolving aerospace “picture” in three dimensions, and failure rates have always been high compared to most other RCAF occupations, historically in the 35-60 percent range. While AEC candidates undergo selection testing to help identify those who may not succeed, selection criteria employed to date have not been sensitive enough to the type of spatial ability necessary and have proven to be limited in their ability to predict training success.

Until very recently, AEC has relied on the selection tests used to select CF general service officers and NCMs, tests which emphasize general cognitive ability and have limited ability to assess specific competencies such as spatial ability. More sensitive tests are used elsewhere, both within and outside Canada, for example, the Royal Air Force Aircrew Aptitude Test (RAFAAT), NAV CANADA’s web-based selection system and the web-based Air Traffic Control selection system used in Europe by the European Organization for the Safety of Air Navigation (Eurocontrol) since FY 2005/06. The RAFAAT battery of tests is used by ten other Air Forces around the world and, in November 2011, was introduced in the RCAF on a trial basis. However, a statistically significant assessment of the effectiveness of RAFAAT on AEC failure rates is still several years away. NAV CANADA selection management staff consider their system to be effective, efficient, secure and low-cost (approximately $300 per candidate). The NAV CANADA system and the Eurocontrol system appear to be worthy of consideration as a solution to this problem.

Other factors contributing to the high failure rates in the AEC occupation are the “first-past-the-post” selection method and a high “selection ratio” whereby there is only a small pool of applicants from which to draw. NAV CANADA processes 9,000 to 16,000 applicants a year for 120 ATC training slots at their regional training centres across Canada, a minuscule selection ratio that ensures those selected have an excellent chance of success in the profession. The failure rate of AEC candidates would be lower if the selection ratio could be reduced by increasing the number of AEC applicants recruited. Recruiting more AEC applicants remains an ongoing challenge.

The evaluation also found that the high failure rate in the visual flight rules (VFR) stream of the AEC occupation may be attributable in part to the distance learning methodology in use at the CFSACO for VFR stream core training (provided through a distance learning contract with a commercial supplier). The lack of interaction with classroom instructors does not promote two-way communication and is not conducive to the learning experience. The AEC occupation has identified this failure rate issue and is investigating whether VFR stream core training should be returned to CFSACO.

2.3.2.2 Unit Costs

It was not possible to conduct a comparative analysis of per student training costs for most RCAF occupations; however, a comparison was made between the costs of training a pilot to the end of Phase IV (i.e., beyond “wings” standard) through the CFTS/NFTC system and the costs for CF students taking the US Euro-NATO Joint Jet Pilot Training (ENJJPT) fast jet training stream, the program that NFTC was to mimic for the NATO partners.

It cost approximately $2.6 million to train a jet pilot in the CFTS/NFTC system in FY 2011/12 (expected to rise to $2.7 million in FY 2012/13) compared to $1 million per student for the ENJJPT program. At this point, the Phase IV ENJJPT program has slightly fewer flying hours than the NFTC equivalent but this gap will be closed if agreed changes to NFTC flying schedules are implemented in 2012. While these changes are expected to increase ENJJPT program costs, the CF will continue to pay significantly more per fast jet student pilot in order to keep initial occupational training and FLIT jet pilot training in Canada under the NFTC contract.

2.3.2.3 Processes to Identify Areas of Improvement and Adoption of Cost-effective Alternatives

The RCAF recognizes the need for continuous improvement as it pertains to training, and has committed to initiatives intended to increase both the performance and efficiency of training. The RCAF training vision involves the following:

• more performance-oriented training (focusing training on tasks/skills, rather than topics);
• reduced time-to-proficiency;
• improved training capacity (getting more people trained with fewer resources);
• blended learning approach (using the right mix of teaching methods and media);
• leveraging modern technological enablers (e.g., Virtual Task Training using electronic tools to learn, practice and refresh skills); and
• performance support, knowledge sharing and life-long learning.

Through documentation review and key stakeholder interviews, the evaluation confirmed that RCAF staff at all levels at training institutions and HQ are committed to seeking ways to streamline processes and organizations through continuous improvement, organizational restructuring and introduction of best practices to increase the efficiency and effectiveness of training. Given budgetary and workforce pressures, all are cognizant of the need to eliminate non-essential training and over-training, reduce duplication, and adopt new training methodologies that reduce costs and better meet the needs of the modern learner.

CRS noted that the need to be mindful of delivering value for money has been conveyed to all those involved with initial occupational training through the RCAF training chain of command, as clearly reflected in Comd 2 Cdn Air Div’s priorities for FY 2011/12¹⁰ and FY 2012/13,¹¹ which include the following:

• enhancement in selection and training;
• avoiding unnecessary repetition and over-training;
• sharing resources and lessons learned between training units;
• taking advantage of “white space” (course waiting period);
• re-energizing the validation phase; and
• technology, including developing modeling and synthetic governance, blended learning through AFIILE,¹² etc.

With the exception of Pilot training, where a number of complicating factors related to the training contracts limit the ability of the RCAF to take action, CRS saw numerous examples of successful efforts to make training more efficient and economical, including the introduction of modern training aids and simulators, distributed learning, and ongoing rationalization of course content.

2.3.2.4 Use of Up-to-date Training Delivery Approaches, Practices and Technologies

Skills are taught and knowledge is imparted in a number of ways, with the mix determined by the specific needs of the occupation. Training methods include traditional classroom training, hands-on practical training at schools or during on-the-job training, use of simulation, and increased use in recent years of e-learning¹³, either at training establishments or remotely through distance learning. Training strategies may also involve training provided by community colleges, industry partners or allied nations.

To varying degrees, schools conducting RCAF initial occupational training have introduced modern training methodologies that increasingly utilize technology and meet the training expectations (including the learning style and values) of Generation Y¹⁴ and Generation Z¹⁵ personnel. As this trend gains momentum, individual training increasingly reflects the benefits of information communication technologies that enable the development of shared skills and expertise via e-learning and virtual environments, including distributed learning.¹⁶

Training effectiveness is also improving through increased reliance on simulation where appropriate, and the term “synthetic environment” has become increasingly prominent in the military lexicon. The introduction of new aircraft fleets and modern systems has required that training methodologies evolve at a rapid pace and keeping up entails significant cost to acquire new equipment and systems as well as to develop the specialized skills required by instructional staff, software specialists and equipment maintenance staff at the training schools.

CRS observed that schools that conduct Air Force initial occupational training continue to review instructional methodologies and tools on an ongoing basis to ensure they are as up-to-date as affordability permits, while meeting the learning needs of the modern trainee. While this is occurring at individual training establishments, CRS did not observe significant integration of new technologies and learning methods or integrated training doctrine across the RCAF training community. In particular, there is no overarching strategy and doctrine to leverage current technology toward effective and efficient RCAF training solutions. Comd 2 Cdn Air Div has the lead role in introducing automation and simulation technologies and policy for the Air Force. As “champion” for automation and simulation, the Air Force Doctrine and Training Division of 2 Cdn Air Div should bring increased coherence to the many initiatives, directives and projects being undertaken by organizations throughout the RCAF at every level.

CRS noted a number of specific training technology issues:

• CAPSS is now 15 years old, uses outdated and exceptionally limited central processors, and spare parts are becoming scarce. The simulator is also based on an analogue system which has proven challenging for some of the younger candidates who have no experience with this technology. The automated processing/scoring system has also been noted for generating some scoring anomalies, which in some cases requires to be retested to provide confirmation of scores.
• There are continuing challenges at CFSCE where RCAF Communications and Electronics Engineering occupations are trained due to aging classroom equipment and the lack of simulation and distance learning in career courses.
• While VFR simulation equipment for AEC and AC Op training at CFSACO is considered state of the art, other stand-alone training equipment obtained incrementally through the Miscellaneous Requirements process is becoming increasingly difficult to maintain. It does not reflect current operational equipment and has no life cycle material manager to support it. Having the same equipment that the CF uses operationally would permit CFSACO to increase training efficiency (and relevance) and shorten training timelines.

2.3.3 Opportunities for Improvement

The evaluation noted a number of specific opportunities to improve efficiency and economy:

2.3.3.1 PATs

A number of RCAF occupations continue to be “critical” (i.e., more than 10 percent under-strength), a situation that typically leads to targeted recruiting as a first step in bridging the gap between TES and PML. As a result, some occupations have a significant number of PATs as the training system struggles to keep pace with the higher-than-normal influx of trainees. These individuals wait for initial occupational training for up to a year, and sometimes up to two years. This situation is de-motivating for the PATs, puts a significant extra management burden on training establishments and the various levels of HQ, and is costly in terms of pay, rations and quarters, and travel costs for PATs who are sent to wings and bases for incremental tasking and OJT.

A CRS analysis of PAT statistical trends over the past five years determined that, while positive progress to reduce PAT numbers and waiting times is being made, in April 2012 there were still over 550 personnel in RCAF-managed occupations awaiting initial occupational training, 119 of whom had been waiting over nine months.¹⁷ Air maintenance NCM occupations collectively account for over 300 PATs as of April 2012. Pilots are also one of the more problematic occupations, with 74 students awaiting Phase I and Phase II training as of April 2012.¹⁸

Given that PATs are untrained and thus largely unproductive in terms of their Air Force occupation, and that they must be managed while they await training, the costs and inefficiencies associated with PATs point to the need for remedial action. While attempts to use waiting time productively by taking other training and through incremental tasking at operational units is commendable, it does not solve the underlying cause, which is the imbalance between the intake plan and training schedules and capacities.

2.3.3.2 Aircrew Selection and Testing

Currently, only Pilots and AECs are sent to CFASC for testing. Pilot candidates complete AEC tests and thus can be offered AEC if unsuccessful as Pilots. Pilot applicants also receive a realistic job preview of the AEC occupation at CFASC. ACSOs are not sent to CFASC and are not tested for AEC. Furthermore, because ACSO testing is not conducted at CFASC, neither Pilot nor AEC candidates are tested for ACSO. Testing all three aircrew occupations at CFASC would allow unsuccessful candidates to be offered another occupation and provide candidates with a realistic preview of all three occupations with the potential to address the significant attrition attributable to applicants not understanding the occupation for which they are applying.

There are currently over a dozen organizations involved in aircrew selection with no single organization coordinating their efforts. While CFASC is well structured to deliver its current testing program, it is inadequately structured to advocate for an improved selection system. A more robust CFASC could play a key role in a more coherent aircrew selection system.

2.3.3.3 Allophones

Allophone Const Engr candidates have a higher failure rate in initial occupational training than those whose mother tongue is English or French. Training staff at CFSME believe that this issue may also affect other officer trainees at CFSME. In the course of the evaluation, CRS became aware of broader concerns related to the training of allophones, beyond officer candidates at CFSME. As Canadian demographics evolve and the CF seeks to be as representative of Canadian society as possible, the number of allophones recruited into the CF is expected to grow and it will be increasingly important to understand and resolve allophone training issues.

2.3.3.4 NFTC Aircraft and Equipment Issues

Equipment issues have affected the NFTC program and have had an impact on both student PAT backlogs and foreign country participation in the program. There have been a number of reported engine problems with the NFTC CT-155 Hawk since its introduction into flying training service in 2001 which have led to several forced landings and two separate aircraft crashes. Investigations by the CF technical authorities following the most recent Hawk crash also revealed several airworthiness problems. The Hawk training fleet has been grounded on several occasions following aircraft incidents which has led to training delays and lost course serials, and forced both the CF and NFTC partners to seek alternate training solutions to overcome training delays.

The higher fatigued Hawks at Cold Lake have not been rotated through the less stressful basic jet training in Moose Jaw, which has exacerbated an already difficult fatigue management problem. This approach runs counter to accepted fleet management practices, wherein use and fatigue life are normally balanced throughout the whole fleet. DND/CF does not own these aircraft and, under the NFTC contract, the contractor is expected to track wear, and report and manage the fatigue life of the contracted aircraft on a routine basis. Left unmanaged and with no mitigating action, current projections indicate that the Hawk fleet will not last until the end of the NFTC contract. If no resolution to the problem is forthcoming, the Hawks that have reached the end of their fatigue life will start being withdrawn from active training service as early as 2016, further reducing student pilot output.

2.3.3.5 Lack of Instructor Pilots

By contract, the CF has the obligation to provide qualified IPs for certain defined phases or functions, to both CFTS and NFTC. However, numbers of qualified IPs have been insufficient relative to numbers of students at each phase of training. Comd 15 Wing noted that the operational communities were not sending enough candidate instructors to any of the flying schools.

Interviews with key staff members at 2 and 3 CFFTS confirmed that potential CF instructor pilots posted to the flying schools are seldom sent in a timely manner that would allow Qualified Flying Instructor training (which can take anywhere from three to eight months) to be completed before outgoing IPs depart. IP numbers have also been affected by administrative policy issues, including a historical trend of instructors taking maternity and parental leave while at the flying schools and IPs with medical or administrative issues that limit their availability to perform their duties being sent to the schools. These factors result in a reduction of the numbers of qualified IPs available.

In addition, while no organization can demand that only the most highly skilled and qualified individuals are posted to their organization, pilots sent to CFTS and NFTC have had a failure rate approaching 10 percent¹⁹ on the instructor’s course. This would indicate that closer scrutiny of potential IP candidates is required.

2.3.3.6 Move CFSACO to CFB Borden

CFSACO trains AECs and AC Ops and is currently located at the NAV CANADA Training Institute (NCTI) in Cornwall, Ontario, where civilian air traffic controllers are trained. The CF ATC school was moved from CFB Borden to Cornwall in 1979 to exploit synergies. Sharing a training facility with the civilian counterpart organization proved cost-effective for the CF by eliminating the need to invest in simulator equipment. However, in recent years NAV CANADA has decentralized ATC training to its regional training centres across Canada, and CFSACO has acquired two simulators. As a result, CFSACO is no longer dependent on NAV CANADA and is self-sufficient in training AEC and AC Op students.

The current location of CFSACO leads to temporary duty costs and lost time as personnel travel to and from CFB Borden on business. In addition, CFSACO is in a civilian environment without a local support base resulting in additional administrative costs and loss of productivity when staff and students must travel to Ottawa for supply, medical and dental services, and to access a Military Family Resource Centre. Furthermore, the current location of CFSACO does not offer students a “military” environment to facilitate the development of military ethos and discipline which may affect the early socialization of new entries, especially NCMs.²⁰

As a result of the acknowledged inefficiencies associated with staying at NCTI, and coinciding with the end of the CFSACO lease, a business case²¹ was prepared in August 2011 to demonstrate that moving CFSACO to CFB Borden would save the Air Force approximately $17 million over 20 years. Payback of the initial capital cost of renovations at CFB Borden and associated one-time move costs through annual cost savings is expected by FY 2018/19. CRS reviewed the business case and supports the recommendation to move CFSACO into existing infrastructure at CFB Borden, ideally in FY 2014/15, following the planned move of the current tenant and set-up of the space to meet CFSACO needs.

2.3.3.7 PAA Anomaly

In the course of the evaluation, CRS noted that the CFTS contract ($1.77 billion over 22 years), a significant portion of Pilot training to “wings” standard, has been incorrectly attributed to sub-sub-activity 2.3.6.1 Aerospace Training rather than to 1.2.2.2 Initial Individual Occupation Training in the PAA. As a result, the cost of producing RCAF Pilots to “wings” standard is understated by over $80 million annually.

2.3.4 Resource Allocation and Expenditure

The following indicators were used to assess whether resources were allocated and expended as intended:

• approved business plans;
• tracking of resource expenditure; and
• planning and control framework for re-allocation of resources.

Based on a review of business plans and program financial data as well as interviews with key informants, the evaluation found that approved Business Plans are in place to govern the allocation and utilization of resources for RCAF initial occupational training. Expenditures are tracked on a continuous basis and, if re-allocation of resources is necessary to meet surge requirements or changing needs, an effective planning and control framework is in place.

Annex A—Management Action Plan

Relevance

Management Action

Given that aligning Pilot production with demand is critical to the development and maintenance of RCAF readiness and the successful conduct of CF operations, and recognizing that it remains a complex challenge that involves a number of L1 stakeholders, C Air Force will bring the issue to the attention of the Defence Capabilities Board before the end of FY 2012/13 in the interests of initiating the development of a corporate DND solution. (Note: This Management Action also addresses Recommendation 11 on the Hawk fatigue life issue).

OPI: C Air Force

OCI: VCDS

Target Date: 31 March 2013

Management Action

The Air Force has made significant progress on updating doctrine since the stand-up of the Canadian Forces Aerospace Warfare Centre, and with higher-level doctrine falling into place, and changes to the way the Air Force deploys and supports these deployments taking effect, Const Engr doctrine is being updated/revised as well. Regarding the Const Engr JBOS, significant work was completed in FY 2007/2008, but was never formally staffed for approval. This JBOS will be reviewed for relevancy and completeness and submitted for implementation.

OPI: C Air Force

Target Date: September 2013

Management Action

QSs should remain distinct as Const Eng and Army Engr are two distinct trades; however, a board will be convened to determine which tasks are common between the occupations and those tasks can then be grouped together as a common package in order to standardize similar training and realize efficiencies. It should be noted that since the 2007/08 Const Engr JBOS rewrite, and the follow-on Qualification Standard Writing Board, commonalities between these two trades were captured, which resulted in significant combined training opportunities that were leveraged by CFSME. Cooperation from both elements in the Engr Branch will continue to ensure that engineering tasks are assigned to the appropriate occupation and appropriately trained, so that RCAF Const Engrs and Army Engrs are both able to operate in joint environments, while still retaining core competencies required by their respective elements.

OPI: C Air Force

Target Date: September 2013

Management Action

The need to improve aircrew selection for Pilot, AEC, and ACSO was recognized by Director Air Personnel Strategy (D Air Pers Strat), Director Personnel Generation Requirements (DPGR), and Director General Military Personnel Research and Analysis (DGMPRA). A key enabler for this process was the recent establishment of a Personnel Selection (PSel) officer position at CFASC. The PSel officer will oversee Pilot, AEC, and ACSO selection and apply research issues in a timely and responsive manner.

The introduction of cognitive ability tests at CFASC in November 2011 to measure work rate, spatial abilities and other competencies for AEC selection are being monitored and assessed by D Air Pers Strat and DGMPRA with the aim to increase the number of tests once additional studies are finalized. The introduction of the RAFAAT battery at CFASC is expected to improve AEC selection. An upgraded version of the RAFAAT battery, which includes attention, situational awareness, working memory, and other measures, is being considered for purchase and inclusion in the AEC selection model. Additionally, DGMPRA is in the process of developing a general information processing measure that involves such domains of working memory and attention. Additional studies conducted at DGMPRA have demonstrated promising results with information processing and situation awareness measures.

OPI: C Air Force

Target Date: December 2012, and will continue to be monitored for success.

Management Action

Distance learning was deemed inadequate to meet the training requirements for ab-initio AEC officers. Starting September 2012, all training was repatriated to CFSACO where AEC officers will receive two weeks of intensive aviation common training prior to commencing their individual courses (IFR, VFR, or weapons).

OPI: C Air Force

Target Date: Completed September 2012

Management Action

The theme of developing an overarching strategy and doctrine for the adoption of new technologies and training methods is part of 2 Cdn Air Div processes. The development and introduction of systems like the Canadian Aerospace Synthetic Environment and AFIILE are just two simple examples of new strategies and technologies implemented by 2 Cdn Air Div over the past few years. Moreover, these themes figure predominantly in the current 2 Cdn Air Div’s Operational Plan for FY 2012/13 and will continue to do so for the foreseeable future. The Canadian Forces Aerospace Warfare Centre, through the Air Force Synthetic Environment Working Group has developed Terms of Reference for its role in this task and with the help of 2 Cdn Air Div HQ, will continue to develop a coherent and overarching governance on the matter.

OPI: C Air Force

Target Date: October 2013

Management Action

In March 2012 CFSACO received delivery of the PAR 2000 simulator which is identical to the next generation equipment recently installed at RCAF wings. There is, however, no visibility on a new IFR simulator. In September 2012, a Miscellaneous Requirement was approved which will provide CFSACO the same air defence simulator that is used at 22 Wing North Bay. To date, CFSACO has solidified the coordination with 1 Cdn Air Div A3 Aerospace Systems and RCAF Directorate of Air Programmes staff to ensure future operational system projects include training suites for the school. Wherever possible, the aim is to harmonize training and operational systems that will shorten training timelines and save on overall equipment sustainment costs.

OPI: C Air Force

Target Date: November 2014

Performance (Effectiveness)

Management Action

Formal integration at the strategic level continues to be refined and coordinated. D Air Pers Strat will improve the alignment of intake requirements with production.

OPI: C Air Force

Target Date: April 2013

Management Action

D Air Pers Strat and 2 Cdn Air Div have established a campaign plan to include Pilot, AEC, and ACSO selection at CFASC and it is progressing as planned. In addition, the creation of the PSel officer position at CFASC to oversee cognitive ability testing and assist with DGMPRA research into current and future selection measures is a step in the right direction.

OPI: C Air Force

Target Date: November 2014

Management Action

DGMPRA is currently engaged in data collection for a DPGR/Canadian Defence Academy co-sponsored project examining training failures of Allophones (those CF members whose first official language is not French or English) in order to provide DPGR with info that will be used to shape selection and training policy for Allophone applicants to the CF. Data from the RCAF training establishments has been included as part of this activity. An interim analysis report is expected by the end of FY 2012/13.

OPI: C Air Force

Target Date: March 2013

Management Action

Although the CF continues the dialogue and can influence on occasion the contractor’s actions, the management of aircraft fatigue is ultimately the responsibility of the contractor (BMAT). In that respect, the recommendation is outside the purview of the CF. BMAT acknowledges the current situation and has recently indicated that they should be able to present a proposal for the life extension of the aircraft to 2021.

In the interests of resolving the Hawk fatigue life issue, C Air Force will bring the matter to the attention of the Defence Capabilities Board in the context of a broader discussion of Pilot production (Note: Recommendation 1 Management Action refers).

OPI: C Air ForceOCI: VCDS

Target Date: 31 March 2013

Management Action

The requirement for IP at RCAF training units is the focus of both Air Divisions. It demands a careful and deliberate manning balance between operational readiness requirements and training needs based on expected throughput. The matter will be subject to a specific discussion led by the D Comd 1 Cdn Air Div at this year’s Combined Commanders Training Session to ensure appropriate apportionment of personnel resources is achieved. This matter has been a focus of attention for both Divisional Comds over the past several years.

OPI: C Air Force

Target Date: Completed and being monitored.

Management Action

In April 2012, the Minister of National Defence approved in principle the RCAF plan to relocate CFSACO from Cornwall to 16 Wing Borden. Planning has commenced with a tentative move date in FY 2016/17.

OPI: C Air Force

Target Date: November 2017

Management Action

D Air CBM will lead a review of all activities currently attributed to PAA 2.3.6.1 Aerospace Readiness, in order to identify those that should be re-attributed to PAA 1.2.2.2 Initial Individual Occupation Training. Once complete, the consolidated list will be forwarded to the Director Defence Force Planning (DDFP) staff for PAA re-profile action.

OPI: C Air Force

Target Date: December 2012

Annex B—Logic Model

Figure 3. Logic Model for Air Force Initial Occupational Training. This logic model describes the inputs and activities necessary to achieve the required number of appropriately trained personnel.

Annex C—Evaluation Matrix

Table 3. Evaluation Matrix—Relevance. Five evaluation issues/questions are used to determine whether the Air Force Training Program is relevant.

Table 4. Evaluation Matrix—Performance (Effectiveness). Two evaluation issues/questions are used to determine whether the Air Force Training Program is performing effectively.

Table 5. Evaluation Matrix—Performance (Efficiency and Economy). Four evaluation issues/questions are used to determine whether the Air Force Training Program is performing efficiently and economically.

___________________________________________________________________________________________________________________________

Footnote 1 The quoted number $87.85 million is the notional funding approved by the Program Management Board for the required growth to meet production targets of 105 newly winged graduates. When examining the cost of the CFTS contract the quoted cost is the notional cost (i.e., program cost) amortized over the life of the program. This does not account for infrastructure obtained or to be obtained which will be employed over the life of the program or unused monies returned to centre.

Footnote 2 Email 2 Cdn Air Div/CRS 15 December 2011. Figure includes all cost centres associated with 15 Wing HQ and subordinate organizations 2 CFFTS, 3 CFFTS and 419 Tactical Fighter Training Squadron.

Footnote 3 Email D Air CFG 4/CRS 5 January 2012 notes up to 19 CF ENJJPT (Phase III and Introduction to Fighter Fundamentals Phases only) pilot training students in 2011/12 at approximately $700,000 per student.

Footnote 4 Email D Air CFG/CRS 2 March 2012 indicates the FY 2011/12 costs. The appendices will provide both CFTS and NFTC figures for FY 2012/13. CFTS will increase to $92.5 million and NFTC will increase to $166.9 million. This will represent an approximate overall increase of $15 million (5 percent) to the overall pilot training costs.

Footnote 5 A-P9-000-001/PT-000, Canadian Forces Manual of Individual Training and Education, Volume 1, Canadian Forces Individual Training and Education System—Introduction/Description.

Footnote 6 An occupation is considered “red” (i.e., critical) if it has:

• a variance (between the forecast TES and the PML) that is 10 percent (or greater) below PML; and
• indications that Committed Production or Intake estimates will not be sufficient to recover to PML within two years.

An occupation is considered “amber” (i.e., caution) if it has:

• a variance (between the forecast TES and the PML) between 5 and 10 percent below PML;
• indications that Committed Production or Intake estimates will be sufficient to recover to PML within one to two years;
• a rapid change (year to year) from at or above PML to below PML, not caused by changes to the PML itself; or
• a variance greater than 10 percent above PML, with no changes to PML scheduled.

An occupation is considered “green” if it has:

• none of the above characteristics; or
• indications that, despite its percentage variation, it can recover to PML within a year.

Footnote 7 Factors other than age likely contribute to the higher pilot training success rate in the RAF including a larger pool of applicants from which to draw. Also, with minor exceptions, the CF demands that officers other than those commissioned from the ranks have a degree. Pilot candidates will often be more than 23 by the time they have finished their degrees or enrol as DEOs.

Footnote 8 "Enhance Agility,” LCol Harding, Combined Commanders Training Session 2011, 6 October 2011.

Footnote 9 Institute of Internal Auditors, The Role of Auditing in Public Sector Governance, 2006 – definition of “governance” as used in a 2010 Audit of the Treasury Board of Canada Secretariat Governance Framework, http://www.tbs-sct.gc.ca/report/orp/2010/agf-acg01-eng.asp.

Footnote 10 Comd 2 Cdn Air Div Planning Guidance/Business Plan FY 2011/12.

Footnote 11 2 Cdn Air Div Operational Plan FY 2012/13.

Footnote 12 The Air Force Integrated Information and Learning Environment (AFIILE) is being developed to deliver future Air Force-wide capabilities related to learning, including blended learning, distance learning and training coordination. The aim of AFIILE is to prevent the proliferation of training information management systems, to ensure standardized processes and increased re-use of learning assets, to increase training effectiveness, to reduce time to proficiency and to increase trainee throughput.

Footnote 13 “E-learningcomprises all forms of electronically supported learning and teaching. The information and communication systems, whether networked learning or not, serve as specific media to implement the learning process.” Tavangarian D., Leypold M., Nölting K., Röser M.,(2004). Is e-learning the Solution for Individual Learning? Journal of e-learning, 2004.

Footnote 14 “Generation Y is a label attributed to persons born during the 1980s and early 1990s. Because children born during this time period have had constant access to technology in their youth they have required employers to update their hiring strategy and training in order to incorporate updated forms of technology.” (BusinessDictionary.com).

Footnote 15 Generation Z includes those born since the early 1990s, also called “Millenials” or the “Internet Generation.”

Footnote 16 Distributed Learning—“The delivery of standardized training, education or professional development using multiple media and technologies when and where it is needed. It may involve learner-instructor interaction in both real time (synchronous) and non-real time (asynchronous). It may involve self-paced asynchronous learned instruction without benefit of access to an instructor. It does not necessarily involve a physical distance between the learner and instructor or need occur outside the confines of the resident training establishment or campus. The dispatch of instructors from a training establishment to a unit or another location to conduct training, or the hiring of qualified instructors from a training establishment to a unit or another location to conduct training, or the hiring of qualified instructors in other locations to conduct the training on behalf of a training establishment fall within the realm of Distributed Learning.” Defence Learning Network – Glossary of Terms, 27 June 2008.

Footnote 17 Air Force Personnel Awaiting Training/Disposition Report, April 2012.

Footnote 18 Ibid.

Footnote 19 4520-2 (WComd) 15 Wing Pilot Manning and Production – Update 2, 14 March 2011.

Footnote 20 Canadian Defence Academy report, Alternative Training Delivery and the Early Socialization of Non-commissioned Members, 2010.

Footnote 21 Defence Construction Canada, Business Case - Relocation of CFSACO to CFB Borden, August 2011.