Evaluation of the Canadian Defence Academy

January 2015

1258-209 (CRS)

Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

Acronyms and Abbreviations

Executive Summary

This report presents the findings and recommendations of the Evaluation of the Canadian Defence Academy (CDA). It examines the relevance and the performance of the CDA for fiscal years (FY) 2008/09 to 2013/14 inclusive.

The Evaluation was conducted by Chief Review Services (CRS) as a component of the Department of National Defence (DND) Five-Year Evaluation Plan (2013/14 to 2017/18). The Evaluation was conducted in accordance with the Treasury Board Secretariat Policy on Evaluation (2009).¹ An advisory committee was formed to support the evaluation and provide comments on the main orientation documents, under the direction of the evaluation team.

Program Description

The CDA was created in 2002 to act as the institutional champion of Canadian Armed Forces (CAF) professional development (PD). In this capacity, CDA has played a leadership role in the design, development and delivery of the individual training and education (IT&E) and the PD systems, to better prepare officers and non-commissioned members (NCM) for the tasks and requirements of CAF. For the purpose of this evaluation, the many programmes and IT&E support, management, and delivery activities within CDA are consider in their entirety as a program.

Through the operation of multiple schools and colleges, each year approximately 40,000 trainees are enrolled in 352 different training courses, representing 41 percent of the CAF. This includes common trades’ specialty training, language training to more than 4,000 CAF members, and university programs for 3,000 part-time and full-time students a year at the Royal Military College of Canada (RMCC) or the Royal Military College (RMC) Saint-Jean.

Relevance

The programs offered by the CDA continue to be aligned to the needs of the CAF. The CAF, unlike most public or private organizations and institutions, cannot recruit service members into non-entry level positions in the military structure. As a result, the CAF must train and manage the career of its members to generate and sustain the military establishment. Accordingly, the CDA is a critical component of the production system for common IT&E and PD at various stages of a member’s career.

Performance

The evaluation team has noted that the CDA program provides a curriculum that is aligned to the needs of the CAF, and courses are delivered in sufficient numbers to meet demand. There is a reasonable stakeholder satisfaction with the quality and delivery of the programs and a high level of student success is achieved. Ultimately, CAF members demonstrate sound competencies and have been recognized internationally as being extremely well trained and knowledgeable. This, to a large extent, can be attributed to the foundational role the CDA plays in supporting the IT&E and PD needs of the CAF.

CDA Headquarters (HQ) and the formation units have successfully designed and delivered developmental period 1–5 training and professional military education to CAF NCMs and the Officer Corps by providing training, education, and professional military education.

The program has also demonstrated reasonable economy. Excluding the military colleges, training programs represent only a small percentage (approximately 4.6 percent) of the overall cost of military personnel. The cost of operating the Royal Canadian Military College of Canada is also comparable both to Ontario universities and similar military academies in the United States and Australia.

These successes were achieved in a challenging environment. Since its inception in 2002, the CDA has faced numerous pressures due to the vast range of skills and knowledge required by military personnel, and the need to work with multiple stakeholders. Furthermore, there have been numerous reviews and initiatives undertaken that have kept the organization in a continuous state of planning and prevented the implementation of the transformation of the CDA business and its formation units. In spite of these challenges, production of trained and educated CAF members has continued throughout the evaluation period.

The CDA has also faced significant resource issues. Over the past five years, it has steadily had its funding reduced, while at the same time its scope and mandate have expanded. Furthermore, the CAF’s expansion during 2009 to 2011 significantly increased demands on CDA programs, as new recruits moved through many of the common training programs at the start of their careers. It is noteworthy that the cost per student in all programs steadily declined despite inflationary pressures. This was achieved through reductions in personnel and facility maintenance—to the extent that the program’s sustainability has, in some areas, come into question.

The evaluation noted some opportunities for improvement. As the CDA does not control the volume of courses it must deliver, in the face of declining funding it has simply reduced resources assigned to the management and support of training and education. This has impacted its ability to validate and evaluate its own effectiveness, and to implement a performance management framework that is supported by sound data-management tools. These resourcing issues have also impacted its ability to implement a large transformation initiative (CAF Campus) to modernize its system of delivery. As a result, there are concerns over the feasibility of this initiative in delivering an effective solution to the identified issues. However, recently, the CDA has implemented initiatives to overcome those issues, as outlined in the management action plan.

That being said, the evaluation found that during the past five years the CDA has developed several initiatives designed to provide efficiencies in delivering IT&E. These initiatives have much potential, and progress is being made to reduce costs while improving services. This also includes the evaluation of alternative training delivery options, as outlined in the IT&E Modernization Strategy (2011) and current CDA business plans.²

It is also recognized that there is a need for stronger departmental governance over the approach of delivering common training. Many of the challenges that the CDA faces are beyond its own scope of authority to resolve. This includes the determination of course demands versus their resourcing, and the improvement of course loading processes, infrastructure and IT systems.

Key Findings and Recommendations

Overall, this evaluation study made 27 findings and 8 recommendations for improvement. The following lists the most significant findings and all the recommendations.

Finding: The curriculum is reviewed thoroughly on an annual basis by key stakeholders, providing strong assurance that the needs of the Environments are met.

Finding: During the expansion of the CAF (2009–2012) the program faced numerous challenges with respect to the alignment of the timing of progressive training courses, resulting in many personnel waiting extended periods for training.

Finding: High success rates overall indicate the CDA’s success in transferring knowledge and skills to students.

Finding: Due to lack of personnel, the CDA has only validated 16 percent of its courses over the past 5 years. This limits CDA’s ability to ensure that course content continues to meet the needs of the Environments.

Finding: Recently, the CDA has undertaken many initiatives to improve CAF members’ satisfaction with Second Official Language Education and Training (SOLET).

Finding: Since 2010, many steps have been taken to address long-standing strategic gaps, but resourcing issues and the complexity of the program have resulted in a very long-term plan to deliver improvements.

Finding: The CAF Campus Operational Framework contains very little in the way of an actual implementation plan. Furthermore, funds or resources do not appear to be in place to deliver the four proposed projects.

Finding: The CDA has significantly reduced its overall expenditures over the evaluation period. This includes a ten percent reduction in personnel and a near-thirty percent reduction in infrastructure and capital expenditures.

Finding: Despite significant staff reductions, the under-utilization of RMC Saint-Jean is resulting in a high cost-to-student ratio.

Finding: Cost comparisons of Canada’s military colleges to other universities and nations’ military academies demonstrate that RMCC’s cost per student is reasonably comparable to those institutions.

Finding: Many initiatives to improve efficiency of the IT&E and PD systems have recently been developed, although they have not been utilized or implemented to their full potential.

Recommendation 1: The CDA should develop an implementation plan with set priorities and timelines to ensure the successful modernization of the PD system.

Recommendation 2: The CDA, as the authority for common training, should investigate means to coordinate all of the various factors at play (recruiting, demand, attendance) to ensure, as far as possible, that students can move between progressive courses with minimal delays.

Recommendation 3: The CDA must give priority to returning to a full-validation cycle of all its courses. The CDA should also ensure that the training establishments (TE) conduct course evaluations to examine the course materials and technology utilized in order to identify improvement opportunities.

Recommendation 4: Priority should be given to developing, in cooperation with Assistant Deputy Minister (Infrastructure and Environment) (ADM(IE)), Assistant Deputy Minister (Information Management) (ADM(IM)), and the Environments, a sound, scalable business case for the CAF Campus to confirm that the CAF Campus will provide the most appropriate solutions.

Recommendation 5: In an effort to ensure that the numerous key IT&E and PD documents are produced, reviewed, updated, or cancelled in a reasonable time frame, clear priorities and specific timelines should be established and monitored.

Recommendation 6: The CDA should expand the Resource Control Measures section of the Performance Measurement Framework (PMF)to include the cost of delivering the training, based upon a complete costing model. This model should be incorporated into the CAF Campus Business Model as a basis for developing further efficiencies and economies.

Recommendation 7: The recent reductions in CDA expenditures (personnel, capital, and infrastructure) must be examined to ensure the ongoing sustainability of training delivery and infrastructure.

Recommendation 8: The CDA should examine its infrastructure needs and portfolio in order to determine how much is actually required and what opportunities exist to increase occupancy and minimize costs. For example, RMC Saint-Jean could absorb some additional training programs/students, allowing other schools to be divested of the corresponding training programs/students.

1.0 Introduction

1.1 Context for the Evaluation

This report presents the findings and recommendations of the Evaluation of the CDA. It examines the program relevance and performance over FYs 2008/09 to 2012/13 inclusive.

The evaluation was conducted by CRS from September 2013 to August 2014, as a component of the DND Five-Year Evaluation Plan (2013/14 to 2017/18) in accordance with the Treasury Board Secretariat Policy on Evaluation (2009).³

1.2 Program Profile

1.2.1 Program Description

The CDA was created to establish a centralized organization to guide PD within the CAF. It seeks to champion lifelong learning and to promote the PD of members of the CAF.

The CDA plays a fundamental part within the force generation realm to ensure that the CAF remains relevant and ready for tomorrow’s challenges. Through the operation of multiple schools and colleges, each year approximately 40,000 trainees are enrolled in 352 different training courses, representing 41 percent of the CAF. This includes common trades specialty training, language training to more than 4,000 CAF members, and university programs for 3,000 part-time and full-time students a year at RMCC and RMC Saint-Jean. The CDA also delivers e-learning for 48,000 DND and CAF personnel through the DNDLearn system. In addition, the Canadian Forces College (CFC) runs seven advanced courses that annually graduate over 600 senior military and civilian students, including many from foreign militaries.

Since FY 2005/06, additional command responsibilities were added, including: the Canadian Forces Base (CFB) Borden/Canadian Forces Support Training Group (CFSTG), Canadian Forces Leadership and Recruit School (CFLRS), RMC Saint-Jean, Canadian Forces Language School, the Canadian Forces Military Law Centre, and the Conduct after Capture Centre.

Furthermore, since 2005 the CDA has taken responsibility for numerous other program areas, including: Basic Training List Management, Foreign Military, Out Service Training, Subsidized Education Management, Defence Learning Network, Out-of-Canada Advanced Training List, SOLET, Aboriginal Programs, Support Occupation Training Authority, and Education Reimbursement.

For the purpose of this evaluation, the many programs and IT&E support, management, and delivery activities within the CDA are considered in their entirety as a program.

1.2.2 Program Objectives

The CDA operates within the broader strategic context of the government’s direction provided in the Canada First Defence Strategy (CFDS), released in 2008, which set out increased force levels for the CAF (70,000 Regular Force and 30,000 Reserve Force) and a level of ambition across three primary roles: defend Canada, defend North America, and contribute to international peace and security. Additionally, government direction for departments to reduce expenditures through Strategic Review and the Deficit Reduction Action Plan are taken into consideration as key drivers and shapers of CDA efforts.

The objectives set out in the Charter are as follows:

1. Ensure a coherent and integrated Canadian Forces (CF) education and PD program;
2. Ensure academic rigour and accreditation for PD; and
3. Enable and encourage CF personnel to develop their intellectual potential.

As the delegated lead for all CAF-wide IT&E, the Commander CDA is responsible for promoting, designing, and delivering common IT&E to meet Military Employment Structure (MES) specifications⁴ that are the foundation of the Canadian Armed Forces Professional Development System (CAFPDS).

1.2.3 Stakeholders

For the purpose of this evaluation, the term “stakeholder” refers to individuals, groups, or organizations within CAF that may be accountable for the design and delivery of CDA activities. These specifically include the CDA, the formation units, the Professional Development Council, and the IT&E Committee.

CFSTG stakeholders include almost all members of the common support occupations⁵ for all DND branches, commands and environments. The Canadian Forces Language School (CFLS) stakeholders also include military personnel from other countries participating in the Military Training and Cooperation Program.

1.3 Evaluation Scope

1.3.1 Coverage and Responsibilities

This evaluation covers activities of the CDA and its formation units for FYs 2008/2009 to 2012/2013 inclusive. In particular, the focus will be on activities of planning and supervision of the IT&E and PD systems. In accordance with the 2013 Program Alignment Architecture, the CDA is responsible for sub-sub programs 4.1.5—Military Personnel Professional Development Training and 4.1.6—Military Personnel Occupation Training. These subprograms are parts of the overall Military Personnel and Organization Lifecycle subprogram and, by extension, components of the Defence Capability Element Production Program.

All training activities planned and offered by the CDA and its formation units are covered by this evaluation, with the exception of the Basic Military Qualification and the Basic Military Officer Qualification, offered at the CFLRS, which were the subject of a previous CRS evaluation in 2012.⁶

1.3.2 Resources

Over the five fiscal years covered by this evaluation, the expenditures of the CDA and its formation units were, on average, $431 million per year. These expenses include the salaries of military personnel assigned to the CDA and its formation units. The average number of CDA employees during the period covered by the evaluation totaled 3,841. This includes members of the regular forces, reserve forces, and civilian personnel. The details on expenditures and the number of employees are presented in section 2.5.1.

1.3.3 Issues

In accordance with the Treasury Board Secretariat Directive on the Evaluation Function (2009),⁷ this evaluation addresses the five core issues related to relevance and performance. An evaluation matrix listing each of the evaluation questions, with associated indicators and potential data sources, is provided at Annex D. The methodology used to gather evidence in support of the evaluation questions can be found at Annex B.

2.0 Findings and Recommendations

Evaluation findings and recommendations are outlined in sub-sections 2.1 through 2.5.

This section examines the extent to which the CDA continues to address a demonstrable need. To make this determination, one key indicator was assessed, as follows:

1. Extent to which the CDA is aligned with CAF needs.

The findings are based upon evidence from document reviews and key informant interviews.

2.1 Relevance—Continued Need

Key Finding 1: The CDA continues to address a demonstrable need.

The Government of Canada’s (GC) CFDS has committed the CAF to maintain the Regular Force at 70,000 personnel and the Reserve Force at 30,000 personnel. To maintain these levels and offset an annual attrition rate of 7.1 percent,⁸ the CAF continues to need to recruit and train new personnel for hundreds of CAF occupations and trades. Furthermore, the skills and knowledge of military personnel constantly evolve as they progress through their careers, and as the capabilities of the CAF change due to technological advances and changing policy and training demands.

As the delegated lead for all CAF-wide IT&E, the Commander CDA is responsible for promoting, designing, and delivering training to support the satisfaction of the MES specifications.⁹ The latter are the foundation of the CAFDS. They set the policy and standards and identify the functions, working conditions, and tasks for common and environmental military occupations and specialty capabilities of CAF members.¹⁰

The CDA is involved at every stage of a military member’s career. The management of entry-level courses requires the CDA to synchronize its planning and resourcing activities with the Strategic Intake Plan, which contributes to generating the training requirements.

While it is clear that the CAF has a need for a continued IT&E system, it is also recognized that the CDA could share the responsibility of the design and/or the delivery of parts of the training. The importance of evaluating alternative training delivery options is discussed in both the IT&E Modernization Strategy (2011) and CDA Business Plans.¹¹ A number of factors could influence the necessity of the CDA’s current role in IT&E course delivery. Depending on the course requirements and content, CDA involvement in IT&E can range from completely outsourcing the design and delivery to complete control over both functions.

2.2 Relevance—Alignment with Federal Roles and Responsibilities

Key Finding 2: The CDA IT&E and PD activities are aligned with the federal government’s roles and responsibilities.

According to the National Defence Act, the Defence Minister has the authority to establish training and educational institutions (section 47.1) and train (section 12.2) the CAF in a manner designed to ensure the defence of Canada, and that all provisions of the National Defence Act are met (sec. 48.2). It is the role and responsibility of the DND/CAF and to remain “continually prepared to deliver national defence and defence services in alignment with Canadian interests and values” and, when necessary, to conduct operations and deliver services to that end.¹² The CDA exists to support the preparedness of the CAF by contributing to the sustainment of the MES through the provision of relevant IT&E and PD activities in support of these outcomes.

2.3 Relevance—Alignment with Government Priorities

Key Finding 3: The CDA objectives are well aligned with federal government priorities and departmental strategic outcomes.

In the 2013 Speech from the Throne,¹³ the GC made it clear that the defence and security of Canada and Canadians is a priority. In support of this priority, the GC remains committed to ensuring that the DND and the CAF have the tools they need to prepare for future threats and challenges, which include proper training and education.

The CFDS recognizes the importance of investing in its people, DND/CAF’s most important resource.¹⁴ Dedicated personnel with the right expertise are essential to the operational effectiveness of the CAF. As stated in the CFDS, in order to compete with the private sector to recruit the “best and the brightest” from Canadian society, the GC recognizes the necessity of providing world-class “technical training and advanced education” and “continued development of a knowledge-based workforce.”¹⁵ The CFDS also identifies Readiness as one of the four pillars of the CAF and, accordingly, plans to improve and increase training for personnel. Readiness is dependent on the sustainment of MES that, as previously mentioned, further depends on the IT&E and PD systems.

The CDA’s activities are also viewed as an important contributor to the fulfilment of the Defence priority of “Strengthening the Defence Team.” Since FY 2010/11 and including the Defence Priorities document (2014–2018), strengthening continuous learning, PD, and leadership capacity—core activities in the CDA mandate and mission—have been Defence Priority Elements.¹⁶ By investing in the Personnel and Readiness pillars of the CFDS, the DND/CAF will be able to align the Defence Team to ensure the successful execution of this and other Defence priorities.

IT&E and PD systems contribute to obtaining for CAF the right knowledge and skills to support DND’s strategic outcomes and priorities. These outcomes are defined in the Program Alignment Architecture as (i) “operations and services to improve stability and security and promote Canadian interests and values”; and (ii) “continual readiness to support those operations.”¹⁷ Ultimately, the objective of the CDA is to support readiness and sustainment of the CAF, to promote stability and security, and to promote Canadian and the GC’s interests and values.

The work of the CDA also plays an important role in the mitigation of one of the key Corporate Risks at DND: Defence Team Capacity Issues.¹⁸ Defence Team Capacity is concerned primarily with the risk that the effectiveness of the Defence Program will be compromised by an inability to achieve the right balance and composition of the Defence Team in terms of size, mix of skills, and diversity of backgrounds. IT&E and PD are a key mitigating factor by addressing skills gaps through training. As the Report on Plans and Priorities for 2013-2014 states, DND continues to prioritize Defence Team Capacity by strengthening leadership capacity, succession planning, continuous learning, and PD.

2.4 Performance—Achievement of Expected Outcomes (Effectiveness)

The objective of the CDA is to contribute to the knowledge and skills of all CAF members so they can perform at the highest level over the course of their careers. The CDA must provide common IT&E in areas that members did not possess upon their entry to the CAF. To meet this objective, the CDA develops and delivers common IT&E and PD training.

To determine the overall effectiveness of the CDA program, the evaluation team assessed the program against its ability to meet the following key outcomes:

Immediate Outcomes

1. IT&E and PD systems reflect the current and ongoing needs of the CAF;
2. IT&E and PD are delivered in a manner that is best suited to ensure success;
3. The CDA, and CDA resources, are effectively governed, managed, and administered; and
4. Academic research and outreach activities are conducted in support of the CAF.

Intermediate Outcomes

1. The CAF requirements for common IT&E and PD are met.

Ultimate Outcomes

1. CDA contributes to excellence in the profession of arms.

The evaluation applied performance measures and key performance indicators against each outcome. Data for the performance measures was obtained from reports, documents, studies, and financial reports provided by the CDA. Documents were also provided by other stakeholders, and several interviews were undertaken with the CDA personnel and stakeholders. Based upon an analysis of all this information, the overall effectiveness of the CDA was determined.

2.4.1 Immediate Outcome 1: CDA IT&E and PD systems reflect the current and ongoing needs of the CAF

A key to ensuring that effective IT&E and PD programs are in place is to confirm that the curriculum meets the needs of the end users—in this case that the skills and knowledge taught by the CDA continues to be relevant for the needs of the CAF.

Performance Measure: Evidence that the CDA reviews and updates its curriculum and course content on an ongoing basis. The key indicators utilized included the following:

1. frequency and extent of curriculum review; and
2. alignment of the PD program to the needs of the CAF.

Indicator 1.1: Frequency and extent of curriculum review

Key Finding 4: The curriculum is reviewed thoroughly on an annual basis by key stakeholders, providing strong assurance that the needs of the Environments are met.

The CDA is supported in its responsibilities by the PD Council, established in 2010. The goal of this consultative body chaired by the Commandant CDA is to provide strategic guidance and oversight of the CAF Officer and NCM PD Framework, CAF IT&E, and CAF doctrine related to leadership, command, and the profession of arms. Three standing committees support the PD Council.

• The IT&E Committee, established to coordinate CAF-wide PD and IT&E, advises the Chief of Military Personnel (CMP) and other senior staff on CAF PD issues through the PD Council. It manages the implementation, coordination, and verification of the IT&E activities listed in Defence Administrative Order and Directive 5031-2 IT&E Strategic Framework.
• The CAF Leadership Advisory Committee advises the PD Council on the development and integration of doctrine, concepts, and research relevant to the PD of leadership, command, and the profession of arms in the CAF.
• The CFC Programs Curriculum Review Board is established to ensure that the specified CFC and NCM PD Centre programmes provide valid, consistent, and reliable training and education that is responsive and relevant to current officer and senior NCM PD needs.

Indicator 1.2: Alignment of the PD program to CAF needs

Key Finding 5: The PD system has only recently been examined. The preliminary results of the CAFPDS study have identified fundamental changes required for the PD system.

During the 1990s, the CAF created the Canadian Forces Officer Professional Development and Canadian Forces Non-Commissioned Member Professional Development systems. Over the years, changes in the operating environment have resulted in both the officer and NCM aspects of the PD system evolving. According to program staff, incremental deviations to the PD system have resulted in key policies and training documents that today no longer fully meet both current requirements nor align with current policy, regulations, or command authority.¹⁹

In 2014, CDA undertook the first review of the PD system as a whole. The CAFPDS study examined the system to ensure its relevance, effectiveness, and efficiency for the CAF’s future environment needs. The study used a three-tiered approach. Tier 1 examined the PD concepts and doctrine, while Tier 2 assessed the framework and structure of the Officer and NCM PD system. Finally, Tier 3, which is not yet complete, will review the existing common qualification courses for Officers and NCMs to verify that they are meeting the mandate, aim and audience.²⁰

The evaluation team reviewed the latest versions of the Tier 1 and Tier 2 reports. The reports identify multiple areas of the PD system that will require modification in order to meet the PD needs of the CAF for the future. The study proposed a greater emphasis on career-long self-development in various subject areas, such as communications, psychology, sociology, politics, history, and anthropology. The content of mandatory PD will need to be modularized further, and a system needs to be put in place to facilitate linkages between prior learning, PD requirements, and career management.²¹

In summary, the proposed changes to the PD system are substantial and will require careful business planning,²² appropriate funding, and continuous validation in a tightly constrained fiscal environment.

2.4.2 Immediate Outcome 2: IT&E and PD are delivered in a manner that is best suited to ensure success

In addition to providing the right curriculum, the IT&E and PD systems must successfully deliver the program. This includes taking into consideration the training demands and availability of forces members, providing quality instruction, and ensuring student success.

Performance Measure: Evidence that the IT&E and PD systems are meeting the demands of the CAF. Indicators included the following:

1. capacity to meet the training demand;
2. success rate of students;
3. stakeholder satisfaction with course delivery;
4. compliance with Official Language (OL) requirements; and
5. existence of an ongoing continuous improvement mechanism.

The findings are based on evidence from document reviews and key informant interview responses.

Indicator 2.1: Capacity to meet the training demand

Key Finding 6: During the expansion of the CAF (2009–2012) the program faced numerous challenges with respect to aligning the timing of progressive training courses, resulting in many personnel waiting extended periods for training.

A key component of the ability to manage the IT&E and PD program is to align resources with the demand for training.

Courses are offered frequently throughout the year in several sessions. Session numbers are determined each year through training demands submitted by the requirements advisors (RA) to the CDA Production Planning Staff through the Annual Production Planning Cycle. The number of training sessions required is calculated based on the total number of people who require the course and the minimum and maximum course load. These calculations then result in an annual course session offering.

The number of sessions during the past five years has remained fairly stable. As shown in Table 1, planned sessions have been consistently in the 3,200–3,300 range. The exception was in 2010/11 and 2011/12, when numbers peaked since the CAF had begun expanding in 2009. As the levels of personnel stabilized, course sessions dropped back to pre-expansion levels—meaning that there are now actually fewer sessions per personnel. Based upon stakeholder feedback, it appears that the CDA is offering sufficient courses to meet current demand.

Table 1 . CFSTG Training Matrix. This table provides the number of sessions CFSTG predicted, actually delivered, and the difference between them expressed in percentages, over FYs 2008/09 to 2012/13.

Course Loading

Another concern is that, regardless of the training demand numbers that are estimated, those numbers are not necessarily reflected in the actual course loading, which is reflected through the Military Individual Training and Education (MITE) registration system. While the CDA is the MITE business process owner, ADM(IM) controls the PeopleSoft program (MITE is centrally controlled by Director Human Resource Information Management). MITE appears not to meet the entire needs of the CAF quantity control system, and the CDA does not have the authority to direct changes or modifications to the system.

For example, there is no ability within MITE to restrict environments, career managers, or RA nominations to what they have identified as a need for training in the Production Planning Cycle. Furthermore, there is no accountability for the training demand requirements submitted, or consequences for over-booking or non-attendance. These limitations also lead to unfairness in allocation of training sessions, as MITE works on a first-come-first-served basis. Essentially, if an RA submits their requirements at the beginning of the production cycle, they are not guaranteed those seats or any seats, should another RA simply book all the seats available for a given course into MITE first. Additionally, one RA may get all the seats, while another may not get any, simply because of the way MITE functions. CDA staff have indicated that the process to make changes is slow and cumbersome.

Wait Times

Another indicator of how well the CDA is meeting CAF training demands is the timely delivery of training. This means ensuring that students move from training to occupations in a timely and seamless manner. Individuals who fall into an area between training and their occupation are identified as Personnel Awaiting Training (PAT). The evaluation sought data and reports on PAT from the CFSTG. CFSTG was unable to provide the PAT data for the years 2008–2013, as student history is deleted once a student reaches their Occupational Functional Point.²³

According to the CDA Team, the management of PAT is a challenge for the CAF overall, based on a number of factors that are often spread among various managing authorities, training authorities, TEs and administrative staffs (e.g., basic training list management). For example, on completion of basic training, individuals are posted to a TE based on their respective trades. Since each TE manages the movement and gainful employment of their PATs during their transitory periods, management and oversight of PATs across the CAF is an extremely challenging task. Not only are there challenges for each of the TEs (e.g., administrative oversight—moves, posting mechanisms, scheduling of courses) the current state of recruiting intake in some trades is having a bottleneck effect on moving PATs efficiently through the training system to an operationally functional point.

The CFSTG staff interviewed described the cost of keeping non-employable people waiting at CFB Borden beyond 90 days as substantial, although an exact cost was unavailable for the 2009–2013 period.²⁴ Even so, these costs are potentially significant. For example, it is estimated that 2,000 PATs held continuously at Borden in 2008 represents a cost of $1.7 million per week.

The CDA has noted²⁵ that the Basic Training List Management Working Group has identified a need to track and manage this data on a regular basis, and it is now a key performance measurement in the most recent draftPMF.²⁶

In summary, the lack of holistic data for the Initial Production Plan, and the lack of control for course loading and attendance, hinders the CDA’s ability to ensure that the right people and the right number of people are receiving the right training at the right time.

Indicator 2.2: Success rate of students

Key Finding 7: The high overall success rates indicate that the CDA is successful in transferring knowledge and skills to the students.

Key measures of the effectiveness of training are the success rate of students and the validation process, which indicate how well students have acquired the knowledge and skills taught.

Table 2 presents the success rates of the formation units and the CDA HQ Programs, summarizing the overall pass rates of the courses taught in each school.

Table 2. Training Success Rates, in Percentages. This table presents the training success rates by formation units and CDA HQ programs for FYs 2008/09 to 2012/13.

As the table shows, success rates tend to be very high for training in the CAF. Failure in training is taken very seriously because it can have an impact on CAF members’ careers and CAF operational missions. In some cases, a failure results in a delay in career progression, while in other cases, such as with the Conduct after Capture training, failure means the person cannot be deployed for a mission.

Overall, the high success rates indicate that the CDA is successful in transferring knowledge and skills to the students.

Indicator 2.3: Stakeholder satisfaction with course delivery

Key Finding 8: Due to the lack of personnel, the CDA has only validated 16 percent of its courses over the past five years. This limits the CDA’s ability to ensure that course content continues to meet the needs of the Environments.

To measure knowledge/training transfer (instruction), adequacy (completeness of content), and timeliness of training, CDA surveys both graduates and supervisors through the validation process.

According to the Canadian Forces Individual Training and Education System (CFITES) Manual on Validation, “all IT&E programs must be validated frequently enough and rapidly enough to detect and correct shortcomings in a timely manner.”²⁷ This validation is conducted to ensure that courses cover the necessary skills and knowledge to meet the immediate and ongoing needs of the student. In the event of a misalignment, the finding is directed towards the TE for action.

While a well-defined process is in place, resource issues appear to be creating significant challenges in the program’s ability to meet its validation needs. Positions dedicated to providing such assurance appear to have been lost during the various strategic review processes. Based upon the CDA data, between FY 2008/09 and 2012/13, the CDA validated only 16 percent of courses offered, or on average 8 out of the 237 courses offered each year (see Table 3). There are also concerns about how the completed validations were actually conducted. These validations were deemed to be “routine checks,” as opposed to more comprehensive reviews, as they did not review all tasks for each occupation. Nor did they review any potential tasks that may have been excluded from the instructional program as a result of decisions made by the Qualification Standards Board. According to the CDA Team, CDA is only able to conduct routine checks, as opposed to more comprehensive reviews, due to resourcing issues—which were exacerbated during the strategic review process. In the actual process, the CDA sends out a call letter each year asking key stakeholders to provide input for courses they feel require validation; once feedback is received, validations requirements are prioritized. When validations are conducted, each of the stakeholders is engaged. As an aside, there are other organizations that use CDA IT&E and may be consulted as well (Canadian Special Operations Forces Command, Chief of Defence Intelligence, and Canadian Forces Provost Marshal-Director General Health Services).

A great majority of the validations conducted also relied solely on survey responses from graduates and their supervisors, contrary to the process recommendation that multiple sources be considered. The validation surveys also suffered from low response rates. On average, over the five-year period covered by this evaluation, the response rate for the graduates and supervisors was approximately 45 percent.

In addition, in lieu of tracking validations against a rolling planning cycle, the CDA relies on stakeholder requests rather than proactively developing a database to trigger required validation courses. While this may respond to courses with significant concerns or needs, it is not a proactive management approach.

It must be noted, however, that the low validation rates of courses does not mean that courses are not amended to meet new/current needs. Lessons learned are brought in at various levels, and updated through specification management.

Table 3 . Validation Reports for FYs 2008/09 to 2012/13. This table shows the number of courses validated and the response rates in percentages for FYs 2008/09 to 2012/13.

Table 4 shows the results of validation surveys conducted between FYs 2008/09 and 2012/13. The reports examine the level of success of the courses within three main areas.

The first assessment area is the utility of what was taught. On average, 67 percent of students and 66 percent of supervisors who had completed the surveys stated that students had used the skills they acquired in their current occupation.

The second assessment area was the adequacy of the training. The reports found that 85 percent of the students had confidence in their ability to perform the skills acquired in the training regardless of whether or not they had used them. This was corroborated by 84 percent of the supervisors.

The third element is the timeliness of the training—meaning whether or not the training occurred at an appropriate point in a student’s career. This was measured as the percentage of students that utilized the skills acquired in the course within 18 months of its completion. Based on validation reports, 82 percent of students had done the job requiring this training within 18 months of course completion. This, too, was corroborated by 85 percent of the supervisors.

Table 4 . Results, in Percentages, of the Validation Process. This table summarizes the average training transfer, adequacy, and timeliness for validations published from FYs 2008/09 to 2012/13.

Key Finding 9: When validated, graduates and supervisors were satisfied with the quality of instruction and relevance of CDA training. However, there were concerns with how current the course content was and the lack of use of modern technology.

It must be noted, however, that the assertion in the finding is based upon the small number of validations that were conducted. To further support the findings, CRS also reviewed the Your Say Survey 2012²⁹ for IT&E to compare the answers provided by the CAF’s environments with CDA validations. The Your Say Survey is a survey of all CAF members on numerous issues, including training and PD. While the Your Say Survey 2012 includes all training conducted within the CAF, including all training that is environment-specific and not common to all environments, courses offered through the CDA make up a significant portion of all training.

These validation survey results were seen to be more favourable, though generally in-line with the Your Say Survey 2012. The survey revealed that 67 percent of all members agreed or strongly agreed that the CAF course prepared them to do their jobs. For those surveyed in 2012, 64 percent agreed or strongly agreed that the courses met occupational requirements. The 2012 Survey indicated that, for 52 percent of respondents, the course gave them the tools to keep learning after the courses are over.

Key Finding 10: Nearly 70 percent of students surveyed agreed or strongly agreed that the courses repeated skills or knowledge they already possessed.

Besides this 70 percent finding that course content often covers previous skills or knowledge, fully 42 percent agreed or strongly agreed that the content was up-to-date. Only 46 percent agreed or strongly agreed that courses placed sufficient emphasis on military culture.

According to the CDA, these issues associated with course content may be related to the perceived requirement (it may be the actual requirement in some TEs) that instructors must teach from the approved, previously-created lesson plan. The CDA program does allow instructors the flexibility to modify and update lesson plans, including the master lesson plan, should they feel the needs of the stakeholders are not being met.

Another area of concern is the use of modern systems of instruction. Only 45 percent of respondents agreed or strongly agreed that the courses made good use of modern technology.

As explained by the CDA team, it is believed that much of the content currently delivered in an online format was enacted by taking content intended for classroom delivery and placing it online without using the principles of design required for this delivery format. As well, the amount of time required to put content online often exceeds the benefit provided, in particular due to the sheer volume required for converting highly technical courses online. With regards to the technological innovations within the classroom, CDA staff raised several areas of concern, including procurement challenges and security/technology issues. Compared to modern learning institutions, the program is significantly behind the technological curve. One example cited was that students frequently show up in class with their own tablets, but instruction is still done with whiteboards, PowerPoint presentations and paper handouts, yet without technological linkages.

Indicator 2.4: Compliance with OL Requirements

This section covers two OL themes: course availability in both OLs and satisfaction with SOLET.

Key Finding 11: CDA HQ programs and formation units are complying with the Official Languages Act.

Key Finding 12: Although progress is being made, translation challenges for highly technical courses continue to pose difficulties.

Key Finding 13: CDA has recently undertaken many initiatives to improve CAF members satisfaction with SOLET.

Course Availability in Both OLs

All but one of CDA’s formation units provide their courses in both OLs, where applicable. The exception is CFSTG, which is the largest of CDA’s formation units and offers many technical courses. The main challenge is with respect to the availability of technical materials in both OLs, although a concerted effort has been made to address this and progress continues to be made. Table 5 shows the total number of courses, broken down by percent translated for the developmental period 1 (DP1) and non-DP1 training provided at CFB Borden and at CFSTG.

Table 5 . CFSTG Courses Translated. This table shows the percentage of CFSTG courses translated for DP1 and Non-DP1, showing the status in January 2014.

Quality of SOLET

The SOLET program is also seen to present concerns and challenges. This observation emanates from the Your Say Survey 2012,³⁰ which was completed by 1,446 Regular Forces members.

The survey showed that a great majority of members agreed or strongly agreed that there are issues with opportunities for OL training, fairness of the OL training selection, and opportunities to utilize their second OL at work after training. Only 23.4 percent of members agreed or strongly agreed that the methods for second OL training are appropriate, and a mere 21.8 percent felt that they were provided with appropriate opportunities to improve the second OL.

Regarding opportunities to improve members’ second OL, it is important to note that the vast majority of responsibility rests with the Environments, which determine who has access to SOLET through their career and succession planning. Furthermore, the majority of students (67 percent) are trained by the Environments via their own teachers or the National Individual Standing Offer.

However, the CDA has recently taken steps regarding its offerings for SOLET. In 2013, the CDA completed two major updates for both its English and French curricula, including more offerings for higher-level training. As for delivery modes, it has opened up the Tutorial à Distance program to the Environments and initiated two projects to examine the feasibility of virtual classrooms. CDA will be adding a survey mechanism to its website to gather user feedback.

The Director of Official Languages and the CDA have also collaborated to provide a recommendation for redistribution of SOLET-related duties. In February 2014, the CMP approved a document entitled Partial Transfer of SOLET-related Responsibilities from Director of Official Languages to the CDA. With the Director of Official Languages focusing on the second OL aspect, and the CDA focusing on education and training, SOLET’s management is now aligned with other IT&E training. ³¹ The CDA can now focus on providing guidance and direction to OL coordinators and on coordinating with the three Environments on delivery matters and providing consistent evaluation and validations of the SOLET program. Finally, the Director Professional Development has established two new committees (the CAF SOLET Committee and the Research and Development Committee) with regular meetings to improve the dialogue between the military schools, the Environments, and other stakeholders on questions of SOLET.

Indicator 2.5: Existence of an ongoing continuous improvement mechanism

Key Finding 14: Since 2010, many steps have been taken to address long-standing strategic gaps, but resourcing issues and the complexity of the program have resulted in a very long-term plan to deliver improvements.

Key Finding 15: The CAF Campus Operational Framework contains very little in the way of an actual implementation plan. Furthermore, there do not appear to be funds or resources in place to deliver the four proposed projects.

Effective organizations demonstrate continuous improvements. Over the past decade, various studies have been conducted on military training and education, and numerous recommendations for improvements have been made. Recommendations from these studies, however, were not implemented, as the CAF was undergoing the CF Transformation, and it wasdeemed necessary to delay consideration of the recommendations. As such, the program has been essentially operating in a steady state for over a decade, and has not addressed concerns about the approach and delivery of the program.

However, in 2010 the Armed Forces Council renewed the IT&E modernization mandate and, subsequently, a working group was created to identify strategic gaps in the IT&E system. That working group identified nine strategic gaps, as follows:

• inadequate exploitation of modern learning methodologies and technologies;
• inability of the current TE infrastructure to support the use of modern learning methodologies and technologies;
• inadequate resources to support the full spectrum of functions for effective IT&E, both in terms of quality control and quantity control;
• inefficient use of resources and a lack of synergy due to “stovepiping;”
• inadequate IT&E performance measurement;
• inadequate agility of the CAF as a learning organization;
• lack of instructor and IT&E manager development;
• lack of synchronization within the personnel generation system; and
• inadequate compliance with the Official Languages Act.

The IT&E strategic gaps identified have led to the development of the CAF Campus Operational Framework, which seeks to resolve the nine gaps by 2019. The following four CAF campus components have been pegged to move forward from now until 2019: the CAF Campus Enterprise Engine; the CAF Campus Performance Management Framework; the Learning Support Centres; and Common Capabilities.

This evaluation noted several concerns about the initiative. While the CAF Campus Operational Framework explains the intended results of the four proposed components,³² there is little in the way of actual implementation plans. Although the CDA operationalized the CAF Campus in September of 2013, and directed that work commence or continue on all the elements contained within the mentioned Framework, funding has yet to be committed to implement the project. All work completed thus far has been funded by CDA HQ with in-year funds.

It is also difficult to determine the feasibility of the proposed CAF Campus components, as a business case was not done prior to the development of the CAF Campus initiative. The CDA is now working on a business case, which should provide some insight on the economic feasibility of the CAF Campus implementation.

Aside from resource issues, there is concern with the envisioned scope and timing of the initiatives. Many appear to be underway concurrently, as opposed to involving a more logical roll-out. While priorities were originally established, the CAF Campus is a complex project that is intertwined with many other initiatives outside of the CDA, particularly those associated with Information Management/Information Technology improvements. For example, delayed implementation of the CAF Campus Enterprise Engine may result in a missed opportunity to leverage the Military Personnel Management Capability Transformation project that is currently underway. Coordination of the CAF Campus Enterprise Engine with the Military Personnel Management Capability Transformation is critical to its synchronization with the personnel generation system.

Based upon the resource and technological challenges facing the broad CAF Campus Project, it would appear that the most important task would be confirm what is achievable going forward, based upon resourcing and interdependencies, and the associated benefits to be had from each component. This should be addressed in the business case.

2.4.3 Immediate Outcome 3: The CDA and CDA resources are effectively governed, managed, and administered

This section examines the extent to which the CDA provides direction and guidance to the CAF and command and control (C2) to the formation units. To make this determination, three key indicators were assessed, as follows:

1. type and frequency of direction/C2 provided to formation units;
2. availability of clear, current policy and doctrine; and
3. oversight and performance measures of IT&E and PD systems.

The CDA plays two key roles in terms of the IT&E and PD systems: a strategic role that fosters excellence in the profession of arms through strategic policy and guidance³³; and a C2 role that provides direction and oversight for its formation units.

Indicator 3.1: Type and frequency of direction/C2 provided to formation units

Key Finding 16: The CDA is successful in providing guidance to the formation units on a regular basis.

The CDA is responsible for providing direction and C2 to its formation units. This is accomplished through the Command Council meetings and the CDA Campaign Plans.³⁴ The various committees that Commander CDA sits on also provide clarity and direction to the formation units on a regular basis. Individual direction is routinely provided from Commander CDA to subordinate commanders or between comptrollers through emails, video conferences and face-to-face visits.³⁵ Interviews with key informants³⁶ indicated that they felt that the CDA provided suitable direction and C2.

Recently, the CDA HQ developed a reporting system, including the Commander’s Update, the bi-weekly video teleconferences with the Level 3 (L3) commandants, and the bi-weekly reports. It has also created a Professional Development Board and a Human Resource Management Board and has begun reporting on eighteen key performance indicators.

Indicator 3.2: Availability of clear, current policy and doctrine

Key Finding 17: The program recently launched efforts to address long-standing issues with respect to its doctrine, replacing 13 outdated written manuals with three updated online documents (on policy, doctrine and tactics, techniques and procedures).

The CFITES,³⁷ which is explained in great detail in thirteen volumes, governs individual training and education for the whole CAF. The volumes are designed to optimize the quality and quantity of IT&E, while minimizing the resources dedicated to the IT&E programs. The CFITES use a six-phase quality control process: analysis, design, development, conduct, evaluation, and validation in order to maximize program effectiveness.

The evaluation noted that CFITES volumes have received very few updates and are not being reviewed on a regular or timely basis. An additional concern is that the volumes are published online but are only available on the Intranet since they do not conform to the GC web publishing guidelines. This is problematic for members who do not have access to the Intranet, and for contractors or organizations preparing bids, as they do not have access to these important, core IT&E documents.³⁸

In 2012, an effort began to modernize the CFITES manuals. Approval was given in spring 2014. The intent is to cease publication of the manuals and replace them with three levels of doctrine (policy, doctrine, tactics, techniques, and procedures).

The CDA has indicated that it will be taking an echelon approach to the policy review. A work plan with the policies to be addressed in the next 6–12 months has been presented to the Professional Development Management Board on June 10, 2014, and to the IT&E Committee on September 11, 2014. It will be presented to the Military Personnel Policy Committee on September 24, 2014.

Indicator 3.3: Oversight and performance measures on IT&E and PD

Key Finding 18: The CDA has recently begun working on improving its performance measurement through a PMF.³⁹

The 2005 CRS Evaluation of the Military Individual Training and Education⁴⁰ identified weaknesses in the performance measurement structures and approaches to IT&E as a major concern. Performance measures are essential in order to make informed decisions, assess effectiveness, and meet CAF strategic objectives.⁴¹ To date, this issue has not been resolved.

This evaluation found that the CDA has very few performance measures in place. The CDA receives various reports and data from the formation units, but these are not compiled and tracked in a consistent manner. The CDA does not have a central repository of data as the information is collected at the school level in various formats.⁴² The lack of performance measures is due in part to a lack of direction in data gathering.

The CDA is aware of the importance of meaningful performance measures, and has recently begun working on improving its performance measurement through a PMF.⁴³ In the short term, that PMF is meant to present an approach for systematically collecting, analyzing, utilizing and reporting on the performance of CDA programs and activities. In the long term, the PMF is envisioned to become the basis for the performance management of the CAF Campus.

This evaluation examined the latestPMF draft⁴⁴ and concluded that the key performance measurements identified are meaningful and will assist the CDA in optimizing its planning, management, and performance measurement. It will also provide senior management with timely information on the relevance, success, and cost-effectiveness of programs and activities. Additional benefits could be the development of best practices, lessons learned, and continuous improvements.⁴⁵ The Resource Control Measures section of the PMF could be enhanced to include the cost of delivering the programs and activities. A standardized costing model for training was recommended by the Auditor General in 1994-1995. It was again recommended in the 2005 CRS IT&E Evaluation.

The CDA should assess the most economical means of delivering on its programs and activities by utilizing a complete costing model that takes into account the following factors:

• student cost (student salary x course hours);
• instructor time (instructor salary x course/course preparation hours);
• course development cost;
• course materials cost;
• delivery (classroom or IT costs for distance learning); and
• travel and accommodation costs for students and instructors.

With the above costing approach, the CDA should be able to determine the most efficient means of delivering its courses (i.e., a comparison between classrooms, distance learning, blended, or alternate approaches (outsourcing)). Note that the most efficient method is not always the preferred approach, since the delivery mechanism must suit the needs of the skills being taught. The measure could, instead, be optimal delivery (i.e., a percentage of courses/programs that have been examined for an optimal delivery approach using a complete costing model).

2.4.4 Immediate Outcome 4: Academic research and outreach activities are conducted in support of the CAF

Performance Measure: Degree to which the academic research and outreach activities align with CAF priorities

Indicator 4.1: Research grants from DND/CAF compared to other sources

Key Finding 19: Academic research and outreach activities support DND/CAF priorities.

One component of the CDA’s mandate is to conduct various types of post-graduate level academic research through RMCC, RMC Saint-Jean, and CFC. While the total amount of funding available for research has declined since 2011, the percentage of research grants funded by DND and RMCC in direct support of CAF requirements has increased over the period covered by the Evaluation.

Since DND and RMCC fund most of the research done at RMCC, RMC SJ and CFC, the majority of research done is directly linked to the CAF mandate. This is one of the means by which DND/CAF can ensure that their research priorities are studied.

Indicator 4.2: Outreach activities are in line with DND/CAF Global Engagement Strategy

A review of the CDA formation units’ global engagement activities has demonstrated alignment with the DND/CAF priority objectives. An appropriate level of effort is exerted on the engagement activities, with emphasis on countries with vital interests to Canada’s foreign policy, defence, and security.⁴⁶

2.4.5 Intermediate/Ultimate Outcomes: The CAF requirements for common IT&E and PD are met and the CDA contributes to excellence in the profession of arms

Indicator 5.1: Perceived extent to which the graduates are well trained and ready to undertake their new responsibilities

The evaluation conducted key informant interviews to determine the perceived extent to which the CDA contributes to the excellence in the profession of arms and to CAF members’ ability to perform effectively at all levels.

All key informants stated that, overall, they were confident in the ability of their personnel to perform effectively in their respective roles. While that effectiveness combined training and knowledge that was obtained both from CDA training as well as from within specific training within each Environment, there was a sense of satisfaction (at the strategic level) with respect to the training provided by the CDA.

2.5. Performance—Demonstration of Efficiency and Economy

In order to analyze the economy and the efficiency of the program, the evaluation team examined the resources utilized to produce outputs, including the following:

• trends of expenditures;
• HQ and administrative costs;
• cost per student; and
• evidence of new innovations to drive efficiency.

The following data sources were used:

• DND financial systems;
• CDA financial reports;
• opinion of CDA program managers and stakeholders through interviews; and

• literature review and media scan.

2.5.1 Economy

Key Finding 20: The program is demonstrating reasonable economy in delivering its services.

The total cost of the CDA is a fraction (2.2 percent) of the overall expenditures of DND. Over FYs 2008/09 through 2012/13, the average annual cost for the CDA was $431 million, representing approximately $6,300 per regular force member (although the program also instructs some civilian and Reserve Force members). This comprises the cost of CDA personnel (both military⁴⁷ and civilian) and associated operating expenditures, including course materials and professional services. It also includes infrastructure costs for CFSTG/CFB Borden (not including lodger units) and RMC Saint-Jean. It does not include the infrastructure costs for other properties, such as RMCC or the CF Staff College. These values are part of the Kingston Base or ADM(IE) expenditures. The total does not include the amortization of the capital cost of the existing infrastructure (schools, campuses, etc.) or the salaries paid to students and Officer Cadets while on training.

Salaries and operations and maintenance (O&M) costs represent 85.8 percent of the $431 million CDA cost. The remainder is spent on infrastructure maintenance (10.7 percent) and on capital (3.5 percent). Table 6 presents the expenditures of the CDA by the formation units for the period of the evaluation. Over the past five years in general, the expenditures went down for CFSTG, CDA HQ, and CFLRS, while they remained relatively stable for the other formation units. Annex E provides the breakdown of the CDA HQ and formation units’ expenditures. Annex F lists the number of courses delivered by fiscal year.

Table 6. Expenditures, in Millions of Dollars, for FYs 2008/09 to 2012/13. This table shows the variations in expenditures of the CDA HQ and the formation units by FYs.

* Infrastructure costs at RMCC are excluded from this table.

Of the subordinate L3s/formation units, CFSTG-Borden accounts for 41 percent of the total CDA cost when the annual costs are averaged over the five-year evaluation period. The next largest are RMCC at 16.6 percent (which would be greater if infrastructure is factored in), CDA HQ at 13.7 percent, and CFLRS at 12.2 percent. The other L3s/formation units each represent less than 10 percent of total expenditures (RMC Saint-Jean 7.3 percent, CFC 5.7 percent, CFLS 2.7 percent, and RMCC Research 0.9 percent). Figure 1 illustrates the relative distribution of expenditures by school.

Figure 1. CDA Cost by L3. This figure shows the percentage of the average cost per formation unit over FYs 2008/09 to 2012/13. The data is summarized in Table 7.

Table 7. CDA Cost by L3. This table shows the percentage of the average cost per formation unit over FYs 2008/09 to 2012/13.

Indicator 2.5.1: Trends in expenditures

Key Finding 21: The CDA has significantly reduced its overall expenditures over the evaluation period. This includes a 10 percent reduction in personnel and an almost 30 percent reduction in infrastructure and capital expenditures.

As shown in Figure 2, on average over the five-year evaluation period, 61.7 percent of the CDA cost, was expended on personnel, 24.1 percent on O&M, 10.7 percent on infrastructure, and 3.5 percent on capital. During this period, CDA expenditures declined by 6.6 percent. This was primarily a result of decreased infrastructure and capital spending.

Figure 2. CDA Expenditures in Millions of Dollars. This figure illustrates the trends of CDA expenditures by categories over FYs 2008/09 to 2012/13. The data is summarized in Table 8.

Table 8. CDA Expenditures in Millions of Dollars. This table indicates the trends of CDA expenditures by categories over FYs 2008/09 to 2012/13.

While individual salaries have increased with inflation, since FY 2008/09, the total in personnel expenditures has remained flat due to reductions in staff numbers. The Strategic Review and Deficit Reduction Action Plan saw personnel numbers decrease by approximately 10 percent, from 4,039 in 2008/09 to 3,614 in 2012/13 (note that personnel numbers include full-time, part-time, and seasonal workers, and are not representative of full-time equivalents (FTE)). These decreases occurred primarily among the reserve force (50 percent reduction) and civilian personnel (10 percent reduction).

Costs for O&M also remained largely stable, but inflationary pressures have meant that this area has also experienced a reduction in the amount of services provided. Also due to funding constraints, the amount spent on capital and infrastructure expenses has been reduced by approximately $30 million annually. This represents a reduction in infrastructure and capital expenditures of almost 30 percent over a five-year period. Given that there has not been any divestiture of capital, there is concern over the impact of such reductions on the mid-to-long-term sustainability of the infrastructure.

In summary, the CDA operates today within a comparable budget allocation to what it had in FY 2005/06, but with additional units and programs. Over the last three years, the CDA has had to reduce $31 million of its baseline budget in response to Strategic Reviews and Deficit Reduction Action plan initiatives.

2.5.2 Efficiency

Efficiency was measured at the CDA HQ level and at the five TEs: CFSTG, RMCC, RMC Saint-Jean, CFC, and CFLS. CFLRS was excluded because the financial data covered mainly the Basic Military Officer Qualification and the Basic Military Qualification programs, which are excluded from the evaluation’s scope.

Overall, the program has undertaken significant efforts to drive efficiencies. While it has generally maintained its course-load, the number of training sessions offered, and responded to the needs of a larger CAF, it has done so while operating under a reduced budget. Furthermore, it has reduced its in-year funding pressure from $60 million in FY 2011/12, to $28.9 million in FY 2013/14, and to the current requirement of $18.15 million in FY 2015/16. This was done to generate a total savings of $73 million over three years.

Indicator 2.5.2: HQ and Administration Costs

Key Finding 22: CDA HQ expenditures have declined by 17 percent over the past five years and now represent only nine percent of overall CDA expenditures. Given the number of improvement initiatives underway, this may not be a cost-effective level of support.

When these costs are averaged over the five-year evaluation period, the costs for CDA HQ are approximately $59 million. The command function accounts for 21.2 percent of these total CDA HQ costs, and the remaining 78.8 percent is attributable to programs managed from HQ, including education reimbursement, SOLET, aboriginal programs, Canadian Forces Military Law Centre, and Conduct after Capture.

Despite the decreased budget, CDA HQ has been able to successfully reallocate funds in order to support the programs and activities it is responsible to administer. From FYs 2008/09 to 2012/13, overall spending at CDA HQ has decreased by 17.2 percent, although program spending increased by 8.4 percent. This was achieved through reductions in the Command and Capital expenditures (see Figure 3).

The Command Cell is one of the smallest sections of the CDA HQ. An internal reorganization in 2013 aimed to streamline a dysfunctional HQ (with two chiefs of staff divided into separate business lines) and align the activities within the unit to ensure that productivity was optimized. In addition, much of the CDA HQ Command budget was related to travel in support of the organization, and this has been greatly reduced.

The decrease in funding to the command function represents an improvement in the efficiency of HQ. However, given the serious challenges with respect to the effectiveness of the program’s delivery and management, there is concern that the reduction in funding may not be cost-effective. The reasons for such concern are the numerous program initiatives that are underway and the stated resourcing challenges, which are affecting their delivery. The program initiatives underway are: CAF Campus, PD alignment, doctrine renewal, SOLET renewal, and implementing the PMF and program validation and evaluation needs.

With respect to the capital budget within the HQ, it is important to note that this represents a change in budget management practices. In the past (FYs 2008/09 to 2010/11), capital was held locally and there was a command reserve. Now capital is given directly to the sub-organizations (L3s), and there is no command reserve. Furthermore, for FY 2011/12, ADM(IE) assumed more infrastructure responsibilities, thereby taking on the capital funding linked to that task.

However, regarding re-capitalization, the current rate is very low and may very well cost the CAF more in the long run—specifically, the current baseline capital rate of 2 percent ($4 million on $199 million) combined with an increasing reliance on in-year funding. This is a suboptimal business practice, and is likely unsustainable.

Figure 3. CDA HQ Expenditures in Millions of Dollars. This figure illustrates the CDA HQ command expenditures for FYs 2008/09 to 2012/13. The data is summarized in Table 9.

Table 9. CDA HQ Expenditures in Millions of Dollars. This table provides the CDA HQ command expenditures for FYs 2008/09 to 2012/13.

Indicator 2.5.3: Cost per student

Key Finding 23: In general, the cost per student decreased over the five-year evaluation period, as staff numbers were reduced while student numbers increased slightly.

In general, the CDA operated with fewer resources over the five-year evaluation period while providing the same amount of training. As mentioned, overall expenditures were reduced by 6.6 percent and staff levels decreased by 10.5 percent. Three of the four formation units for which data was available⁴⁸ saw increases in the number of students who attended. Subsequently, the cost per student decreased over the five-year evaluation period (see Table 10).

Table 10. Cost in Dollars per Student. This table allows comparing the costs per student by selected formation units from FY 2008/09 to FY 2012/13.

The CFSTG

CFSTG Borden was able to improve its efficiency of resources from FYs 2008/09 to 2012/13.

CFSTG accounts for the largest percentage of CDA spending, at an average of $177 million a year over the five-year evaluation period. The expenditures for CFSTG decreased by 7.2 percent over the five-year evaluation period, while staff levels also decreased by 15 percent over the five-year period—with regular force members down 6 percent, reserves down 51 percent, and civilians down 18 percent. The number of graduates, however, increased by 2.5 percent over the same timeframe (See Figure 3). This allowed the CFSTG to decrease its cost per graduate by 9.4 percent, from $7,409 in FY 2008/09 to $6,713 in FY 2012/13.

Note that the increase in cost per student in FY 2009/10 reflects the impact of the Vancouver Olympics. Fewer people were able to attend training during this time, resulting in higher costs per graduate as fixed program costs remained the same. There was a recruiting push shortly thereafter, resulting in a 60 percent increase in graduates the following year.

Infrastructure spending accounts for 20 percent of the total CFSTG budget. CFSTG is located at CFB Borden, which is the only base commanded by the CDA. Borden is also home to several lodger units, which is an added cost for the CDA. A lodger unit is an autonomous unit of one command that is normally lodged in a host unit of a different command. It is normally dependent on its host unit for most services, but is established to be independent in its primary role. Almost 70 percent of training at Borden is conducted by lodger schools.⁴⁹ It may warrant a closer look at whether this is an efficient use of the CDA’s resources, as they support activities that do not lead to the outputs of the CDA, namely graduates of CFSTG courses.

The CFLS

The CFLS also used resources with increasing efficiency over the four-year period FYs 2009/10 to 2012/13 (student data was unavailable for 2008/09). Costs remained flat, while student numbers⁵⁰ increased by 44 percent, causing the cost per student to decrease by 30 percent. The number of staff decreased by 14 percent, with decreases in regular force, reserve and civilian positions.

The CFC

The CFC also used resources with increasing efficiency over the five-year evaluation period. While costs increased by 6 percent, student numbers increased by 27 percent, leading to a decrease of 17 percent in the cost per student. In 2012/13, the cost per student was $37,631. The number of staff has decreased by 8 percent.

Military Colleges

Key Finding 24: RMCC Kingston and RMC Saint-Jean have maintained their level of expenditures despite inflationary pressures, through staff reductions.

Key Finding 25: Despite significant staff reductions, the under-utilization of RMC Saint-Jean is resulting in a high cost-per-student ratio.

Key Finding 26: Cost comparisons of Canada’s military colleges to other universities and other nations’ military academies demonstrate that the cost per student at RMCC is reasonably comparable to those institutions.

Based on data provided by the CDA, the average cost at RMCC has remained at approximately $72 million per year. This amount has remained fairly constant over the last five years.

In order to determine the efficiency of RMCC, a comparison was made with other Ontario universities. University costs vary based upon numerous factors, including the value of real estate, the type of programs offered (i.e., graduate and medical programs are more costly), salary costs (full-time professors versus part-time lecturers) and the amount of external funding received. Interestingly, larger universities, despite a greater student base to absorb fixed costs, are not necessarily more cost efficient than smaller schools.

As shown in Table 11, in 2012/13 RMCC had 1,897⁵¹ FTE students, and total costs of over $73 million, for a cost per student of $38,844. That ranks RMCC as one of the more expensive universities in Ontario based upon cost per student, and is 53 percent more than the average for Ontario universities. This cost per student would increase further if full infrastructure costs (likely 10 percent more) and the salaries of the officer cadets were included in RMCC’s costs.

In FY 2012/13, the cost for RMC Saint-Jean was over $31 million for 188 students attending the college, for a cost per student of $168,992 per year. This represents more than four times the cost per student at RMCC in Kingston per year.

The reason for this significant difference is the cost of infrastructure. Over 70 percent of the budget for RMC Saint-Jean is infrastructure and O&M, versus approximately 16 percent at RMCC. In part this is due to the low occupancy levels of the school, with only about 20 percent of its student capacity being filled. In an effort to control expenditures, the school has reduced civilian positions by over 33 percent over the past five years.

However, the evaluation recognizes the value added and the uniqueness of the RMCC. The education program is based on four pillars (education in sciences and humanities; bilingualism; fitness; and military ethos). It offers a unique and favourable environment (hence, role modelling) for the officer cadets, unlike civilian universities. The RMCC is also a research centre for DND/CAF. Besides academic and research values, the RMCC and the RMC Saint-Jean are both important recruiting tools, contributing to maintaining the force level as stipulated by the CFDS. Lastly, according to the literature review, RMCC graduates seem to serve longer than others in the CAF.

By way of comparison, the United States and Australia also utilize the military college system. In the United States, all three services have their own degree-granting academies—WestPoint, Annapolis, and the Air Force Academy. Costs are similar to those at RMCC. For example, the cost per student at United States Military Academy at WestPoint is approximately $53,500 per student per year.

The Australia Defence Force Academy has a 10-year contract with the University of New South Wales to provide accreditation of degrees (it is not a degree-granting institution and, therefore, must depend on articulation of agreements with civilian universities), teaching services,⁵² and determination of curriculum.⁵³ According to partial data collected in 2011/12, Australia spends $53.4 million per year for the academic training of 1,030 military personnel, which amounts to $51,844 per student.

Table 11. Cost per Student for Ontario Universities. This table shows a comparison of Ontario universities and RMCC in FY 2012/13 in terms of student numbers, global expenses, and the cost per student.

Indicator 4: Evidence of new innovations to drive efficiency

Key Finding 27: Many initiatives to improve efficiency of the IT&E and PD systems have recently been developed, although they have not been utilized or implemented to their full potential.

This section presents the main alternative measures established by the CDA during the evaluation period in order to obtain efficiency gains and improve the design and delivery of courses and training programs. The evaluation team consulted CDA documents and records and conducted interviews with key stakeholders in order to identify these main alternative measures (see Table 12).

Table 12. CDA Initiatives. This table summarizes the initiatives already implemented and being considered.

The evaluation found that, during the past five years, the CDA has developed several initiatives designed to provide efficiencies in the delivery of IT&E. These initiatives offer much potential. For various reasons, likely due to capacity issues within the CDA command function, many of these opportunities have not been leveraged to their full potential.

The CDA has in place tools and processes that create interaction and partnership with civilian post-secondary training and education institutions. These partnerships create opportunities for CAF to focus on operational training and education while using civilian institutions to deliver areas of common training that meet minimum military requirements, as defined by occupational authorities and training authorities. Rationalization of training delivery would occur throughout the CAF Campus, from approaches that leverage public and private IT&E opportunities to the ongoing optimization of the CAF Campus structure, organization, and operating concepts.⁵⁴

For example, a joint venture between the CAF and Algonquin College enables geomatic technicians to be trained at a civilian centre of excellence and earn a Red Sealcollege certificate. A similar venture is underway with a trial with cooks, to determine if parts of that trade’s courses can also be outsourced effectively. The potential exists to do the same for other occupations, but one must carefully analyze and develop a business case to ensure that we do not limit the institution and generate unexpected risks. The ability of the CDA HQ to deliver further similar initiatives is very limited on a yearly basis with the amount of staff available.

The CDA has reduced the number of training days required across its PD courses while maintaining the quality and effectiveness of related courses and still meeting CDA’s personnel generation mandate—which remains government-directed and non-discretionary. For FY 2013/14, the CDA Formation has seen a decrease of some 417,043 hours, or 69,507 training days. This has come in the areas of: the Radiation Safety course amalgamation; CBRN⁵⁵ course changes; and Prior Learning Assessment and Recognition. Other areas, such as blended learning methodologies, rationalized training delivery, and innovative technologies such as the Defence Learning Network, should also help to gain further efficiencies. However, the human capacity to lead and develop these changes is extremely limited. Therefore, without an investment in human capital or technologies, the pace of change will be marginal.

Annex A—Management Action Plan

CDA’S Overview of the CRS Evaluation Report

As mentioned in the Overall Assessment of the CRS Evaluation Report—and we fully concur with the statement—“CDA is a critical program for the CAF, delivering essential common IT&E and PD for members as well as providing a key functional authority role.” We believe, however, that the following summary statement does not necessarily identify the nature of these challenges: “Resourcing and structural issues have limited the ability of the CDA program to address several significant challenges.” For example, while we agree that there is a requirement for an additional strategic investment in military person years, we note that limiting the change to an internal re-allocation is not a sufficient option. Nevertheless, our commitment to monitor and appropriately adjust resource allocations remains in effect, as illustrated in FY 2013/14 when the Commander rebalanced internal military resources to reduce risks within the formation and invested 13 such military positions into the Conduct after Capture Centre.

The Overall Assessment ends with this statement: “Going forward the CDA program needs to reinforce timelines and plans to ensure that priorities are met and projects are implemented.” Our main point of effort is to ensure that such effectiveness, and significant strides, are being taken to improve our management reporting systems, such that the formation becomes better able to demonstrate improved effectiveness, efficiencies, and economies (“the 3 Es”). Initiatives include the development of a costing model for IT&E modernization, improved performance management (metrics and reporting), and a focus on updating and adhering to the validation cycle in CFITES.

As a Defence Renewal Team initiative, IT&E Modernization (Defence Renewal Team 5.1, which includes CAF Campus) will remain a primary focus of development within CDA. At the CAF Program Management Board in April 2015, for example, new resources that will be provided to enable Defence Renewal Team 5.1 were confirmed to be additional funding of $1.5 million for indeterminate public servant salaries and $18.5 million in O&M funding (over five years). Commensurate developmental activities have been enabled by a strategic investment of public servant resources from CMP to CDA, which will soon revitalize the Prior Learning Assessment and Recognition, Validation, and Performance Innovations sections, and curriculum development at the Chief Warrant Officer Osside Profession of Arms Institute.

A further expansion of CDA’s responsibilities, to include Recruiting Group and Personnel Generation Requirements, will result in CDA evolving into a new organization called Military Personnel Generation (MILPERSGEN) in June 2015. As CDA’s governance and communications strategies evolve to meet the new challenges associated with this broader mandate, our commitment to addressing the recommendations of the CRS evaluation report will not waver, and we will continue to pursue initiatives in terms of effectiveness, efficiencies, and economies to ensure that we optimize our activities to enable institutional and operational excellence. MILPERSGEN will continue to play a fundamental part within the force generation realm to ensure that the CAF remain relevant and ready for tomorrow’s challenges.

Management Action

The key driving force behind the changes to the PD system is the CAFPDS study’s recommendations that not only affect the PD system but CAF career planning and personnel appraisal systems as well. The intent is to implement these modifications by the end of 2017. CDA must manage three separate lines of operation (LOO) to enact these changes. First, many of the recommendations will require modifications to our Professional Military Education programs, which are our LOO 1, in areas such as content and delivery methodology. LOO 1 requires CDA to update the general specifications, quality standards, and training plans in succession so that the new and improved Professional Military Education programs can commence in 2017. Second, and simultaneously, CDA will develop Institutional Leader 2030, which is our LOO 2. It provides strategic guidance to all the CAF, explaining further complementary changes required to enhance the areas of PD, personnel appraisal, and career planning. This will require a joint coordinated effort from the various CAF stakeholders to analyze the requirements, develop actions to enact the direction, and, finally, carefully coordinate the implementation. Finally, LOO 3 will involve enacting the other recommendations of the CAFPDS study, such as policy changes or other directives that are not part of a specific Professional Military Education program, which is LOO 1, and not be covered by LOO 2. This, like LOO 2, will also require a joint coordinated effort from various CAF stakeholders to analyze, develop, and implement. LOO 1 implementation will be straightforward, since CDA is the training authority for common Professional Military Education, and future structural changes within CMP will reinforce this ability to synchronize further. However, LOO 2 and LOO 3 will be complicated as a consequence of the Functional Authority/Training Authority/Designated Training Authority governance structure, and will require Level 1 (L1) stakeholder engagement and support for all supporting plans within these two LOOs. CDA will need to carefully monitor implementation along LOOs 2 and 3 by all stakeholders. All LOOs will be key commandant’s priorities within the CDA Campaign Plan for the next two years, and they will require vigilant management by CDA to ensure that the key milestone of 2017 for implementation is realized. Follow-up will be vital to confirm that the changes are creating the intended output, and a CDA priority for 2018 and beyond.

OPI: CDA

Target Date: December 2017

Discussion

It is important to note that this is a complex process, and “one size does not fit all.” Accordingly, it is important to distinguish between the different types of IT&E that CDA is responsible for. For example, with respect to DP1 training: CDA trains folks for the Environmental Chief of Staff’s to employ at their operational units—so delays in graduation dates may unduly affect operations, as such, these decisions would need to be worked out through the affected Environmental Chief of Staff’s. Decisions to reduce the total number of serials offered by maximizing course loading may be more applicable to advanced IT&E. , the availability of students given operational requirements is a factor that has always been at play, and is the usual reason why courses cannot often be maximally loaded.

Management Action

When striving to optimize the CAF training system, its inherent complexity (including the associated non-discretionary demands) and the second- and third-order effects of change must be considered. Nevertheless, it is known that improved coordination between recruiting and initial trades training is an essential element of any plans to further improve PAT/Basic Training List Management. To that end, CDA has already introduced the Leader Development Model, through the L1 cadre to Armed Forces Council, to set the conditions for success by fostering an understanding of the need to integrate across all elements of the personnel generation function. This new perspective clearly highlights the systematic benefits associated with transferring Recruiting Group to CDA, which is now a key characteristic of the recently-approved CMP Reorganization Plan (effective January 2015, for implementation at Active Posting Season 2015).

In general, tactical decisions regarding the Course Production Planning process are supported by the pan-CAF Training Synchronization Working Group, at the CDA Formation level by the Production Resources Resolution Committee, and subsequently at CFSTG level by the Training Steering Committees. They consider the myriad of factors that are involved and make decisions to maximize “the 3 Es.”

CDA has also implemented a Rationalized Training Delivery Program, which is supported by the CAF College Opportunities Directory and which allows civilian colleges to match their programs against military training plans. This program has already shown success in a number of technical trades across the CAF. The cornerstone of the CAF campus Enterprise Engine is the Defence Learning Network, which has already enabled large resource savings with support for non-residential and blended learning. Furthermore, CDA is also attempting to implement Resource Control Measures with the Resource Management System trial in Borden. The intent is to use this System in support of a common user facility. If proven successful, the Resource Management System offers CAF-wide more efficient use of resources and infrastructure.

CDA remains committed to reinvigorating the Quality Control processes and policies to ensure that clear roles and responsibilities are established and understood; in particular, the correlation between the production of people through the IT&E system and its impact to business planning. The initial IT&E demand requirements sent to CDA from the Requirements Authorities a year in advance of the course dates, will be better “scrubbed” by staff in the future to allow for more realistic initial demand numbers (e.g., “same as last year” is not an acceptable return). Finally, nominations within MITE are based on current business processes, which should be updated to ensure that actual nominations better reflect the forecasted requirements that CDA has already been given for a particular fiscal year. This will help alleviate the course loading issues that have been identified.

OPI: CDA

Target Date: January 2016

Management Action

The action plan to remedy the deficiency of validation within MILPERSGEN will conclude during 2016. The plan includes: delegation, increased staffing, automation, improved doctrine and the development of validation policies. MILPERSGEN will delegate validation activities to MILPERSGEN Training Group. There will be an increased staffing of two FTEs (one for MILPERSGEN Training Group (EDS⁵⁶ 02) and one for MILPERSGEN HQ (EDS 03). The current Integrated Systems Approach to Training capability, which is growing within the CAF, will have a proposed automated validation tool along with other helpful survey tools. To bridge the capability gap of current Integrated Systems Approach to Training resources and its future validation and survey capabilities, MILPERSGEN will look at either an alternative to current tools (eListen) or, in conjunction with the Royal Canadian Navy, use the SNAP survey software. Furthermore, improved doctrine, reflecting clearer frequencies for validation, and a method for prioritizing qualifications and streamlined methodologies, will allow for validation resources to be focussed more efficiently. Lastly MILPERSGEN will develop policies and mechanisms for tracking and reporting in order to allow for validation to be integrated into performance measurement. It should be noted that the number of qualifications requiring a validation has been reduced by approximately one-half due to an active investigation scrutinizing and prioritising qualifications assigned to MILPERSGEN as a training authority.

OPI: CDA

Target Date: October 2016

Management Action

IT&E Modernization (CAF Campus) is a critical part of the CDA Campaign Plan and currently funded for FY 2014/15 using the Commander’s over-planning. CAF Campus is a recognized Defence Renewal Team initiative that is scheduled to be presented to the Defence Capabilities Board this spring/summer. The intent is for the CAF Campus to be a Defence Renewal Team-funded project by 2016.

OPI: CDA

Target Date: March 2016

Management Action

MILPERSGEN has made and will continue to make a concerted effort to ensure that the numerous key IT&E and PD documents are produced, reviewed, and updated in a reasonable timeframe. As indicated in the CRS report, the CDA has developed a four-echelon policy Campaign Plan to ensure that key policy documents are updated and aligned with modernization initiatives; echelon one is currently well underway. In order to ensure that the other policy initiatives are addressed in a timely manner, the current lack of resources is being addressed through the staffing of three policy personnel, as well as the interim hiring of a contractor. This infusion of additional resources will permit MILPERSGEN to advance policy work on echelons two and three simultaneously, alongside echelon one priorities. The policy team has commenced, in December 2014, briefing Commander CDA on the policy progress, including about timelines and priorities, on a quarterly basis. This interaction permits complete oversight of policy progress and ensures that, when necessary, policy progress can be reorganized due to emerging priorities.

The progress of the CFITES manuals is also underway. As these documents will be rewritten into a joint publication, MILPERSGEN will archive the current manuals, which will enable them to be available on both the internet as well as the intranet; the original aim was to have this completed by the end of June 2015. The re-write of the manual is a more complicated process requiring input from across the CAF. Although progress is being made in this area, it is estimated that a draft will only be available by November 2015. Once the draft is complete, MILPERSGEN will engage key stakeholders to review the document, with the aim of having it completed by June 2016. That being said, concurrent work on the tactics, techniques, and procedures, as well as on standard operating procedures, which are key to supporting the new joint publication, is progressing with the express intent of publishing them at the same time as the new joint publication. Similar to the regularly scheduled updates for policy progress, the doctrine team will be providing updates to their timeline and progress-to-date as of January 2015.

Finally, updates to the PD publications are progressing as per the established timelines that were laid out in the current CDA Campaign Plan. Three of the five key PD documents—Leadership in the Canadian Forces: Leading the Institution, Duty with Honour, and Institutional Leadership 2030—are all on schedule to be completed by June 2015. Updates regarding progress of these documents are provided monthly in the Commander’s Update Brief.

OPI: CDA

Target Date: Access to CFITES archive: June 2015; CFITES update drafted: November 2015; and CFITES update completed: June 2016.

Management Action

Performance measurement: the Performance Management Framework, as well as the latest version of the PMF, were provided to the CRS team. What they have not received is the proposed Quarterly IT&E Performance Measurement scorecard that was developed by the PMF Working Group and by the L1 stakeholders. The design/development/implementation of the dashboard (scorecard) data aggregation is underway—although we note that there are many more data streams than previously considered. The scorecard has been used within CDA IT&E to report to the Commander on the working group’s eighteen proposed measurements. Concurrently, we are refining and evolving the measurements to link them with a Kaplan and Norton-type strategy map to ensure that we have the right measurements for the questions being asked. The Working Group refined the scorecard on November 28, 2014, to provide more definition and clarity to the proposed measurements, which were redefined, reframed or deleted. The KPMG value mapping of the IT&E system under the auspices of DRT 5.1 identified key metrics of strategic relevance. These metrics will be the baseline for development of a list of strategic metrics. A “straw man” of operational metrics will be developed through staff analysis and proposed to the PMF working group to support development of appropriate algorithms to generate Training Authority/Delegated Training Authority decision-makers. Contractor support will be applied to review and evaluate existing metrics and to develop recommendations for improved metrics.

Costing model: as regards the IT&E Costing Model, the first working group was held with L1 stakeholders to discuss past attempts to create a model. Current resource constraints have stymied further progress. A Costing Model Working Group on the Learning Portal invites an exchange of ideas and collaboration with stakeholders. Director Costing Services has agreed to provide supporting validation to the model when it has been created. The costing tools must eventually allocate not only direct costs associated with a program, but also a proportion of indirect costs and be able to roll up from the tactical to strategic levels. The degree to which indirect costs will be attributed to IT&E costs must be considered. The model will describe detailed costing methodologies to be adopted at all levels. Existing CAF models and commercial off-the-shelf alternatives will be evaluated against the requirements. Next steps include; staffing performance measurement positions and proposing the draft model to L1 stakeholders for extensive review and comment—while concurrently developing a pan-CAF Costing Model policy, and applying de-conflict measures to the various data streams.

It has been determined that three additional positions are required for the Performance Management cell. Director IT&E submitted documentation to create and staff these positions, essentially going from a two- to a five-person section comprised of one Major with OPI/ Point of Contact Costing Model expertise; one Lieutenant (Navy) with PMF Scorecard and Policy/Doctrine exposure; one programmer/developer at Computer Systems level O2; one Administrative Services, level 04, for analytics and coordination; and one Administrative Services, level 03 with a finance and business analyst background). This then links both PMF with the Costing Model to further develop economies and efficiencies in numerous areas. With these additions, it is envisioned that CDA will be able to carry out all aspects of steady state PMF reporting and cost modelling investigations. Contracted support will be applied to develop a coherent and comprehensive PMF and cost models and to ensure the integration of these data sets. It should be noted that the staff of Director Costing Services have advised that they would like an opportunity to review any model developed prior to its disposition. They have stated that they do not want to be a part of the working group, but will provide comments when required and requested.

OPI: CDA

Target Date: February 2016

[Note: For infrastructure, please see Management Action for CRS Recommendation 8.]

Discussion Regarding Sustainability of CDA Personnel Reductions

The reductions in CDA expenditures, notably on personnel, over the past five years, and the concurrent increase in student throughput is a laudable accomplishment as a clear increase in efficiency. Determining if the new model, which demands more from fewer resources, is sustainable over the long term is the next logical step.

CDA Personnel Administration staff have tracked manning and additional manpower requests over the past few years. In FY 2013/14, there were requests from CDA units for 33 additional military positions during an annual establishment review. Requests for additional FTEs were not entertained that year. In FY 2014/15, there were requests for 28 additional military positions and requests for 39 additional civilian FTEs and corresponding salary and wage envelope.

Over five years, CDA reduced its workforce by ten percent. In FY 2013/14, requests from CDA units signaled that there were valid requirements to return at least one percent of that total back to the units. In FY 2014/15, there were further requests for an additional two percent of that original reduction. CDA has achieved important economies of scale, but with the CAF strategic intake plan expected to increase in FY 2015/16 and subsequent years, it is fair to expect further requests for additional personnel.

Management Action

CDA leadership has implemented several strategies to address the pressures that have arisen as a result of personnel cuts. Online training delivery and distance learning solutions have been and continue to be the main LOOs to do more with less.

Through successive years, CDA has prioritized all military and civilian positions and rebalanced the workforce by reallocating military positions and civilian FTEs from lower priority endeavours to higher priority activities. The second approach has been to work with Director Military Careers to get as many of CDA’s vacant military positions filled as possible, irrespective of their Vice Chief of the Defence Staff priority manning number. Our latest efforts have been to seek additional FTEs and accompanying salary and wage envelope from CMP. Indications are that CMP is looking favourably at increasing our workforce by 35 FTEs.

In sum, between 2010 and 2016, the CDA workforce would have decreased initially by 10 percent but has actually slowly increased by 4 percent, for a net loss of 6 percent. At the same time, a certain amount of rebalancing of the workforce will have been accomplished.

OPI: CDA

Target Date: December 2016

Discussion Regarding Sustainability of CDA Budget Reductions

The CDA’s baseline is underfunded. Consequently, CDA is reliant on large amounts of in-year funding in order to meet its mandate. That said, this reliance has been reduced from $50 million to $19 million in just two years. Specifically, CDA spending has been reduced from $286 million in FY 2011/12 to $224 million only two years later. Much of this is due to reductions from the Strategic Review and the Deficit Reduction Action Plan, although some of it was accomplished through internal efficiencies. For example, since implementation of the Travel, Hospitality, Conference, and Event Expenditures policy, travel expenditures have been reduced by 75 percent ($13 million) from FYs 2011/12 to 2014/15.

In addition, CDA does not have sufficient capital funds. Specifically, since FY 2013/14 and up to Business Plan FY 2015/16, CDA budgets do not include re-capitalization of equipment. That said, depending on the availability of funds, the CDA has received investment opportunity funding to act as a stop-gap. However, due to the long lead times required to contract/spend capital funds, CDA has experienced difficulty and slippage with expenditures of these capital funds. Essentially, we are mortgaging our future.

Finally, it should be noted that the vast majority of CDA’s costs are fixed (In FY 2014/15, $135.5 million was fixed costs and $44.4 million was variable). As such, CDA has little room to absorb further cost reductions without jeopardising delivery of mandated programs.

Management Action

CDA will undergo a Vice Chief of the Defence Staff-directed Corporate Account Review in the fall of 2015. This Corporate Account Review is essentially a baseline review and recognizes that CDA has never been properly baseline-funded (because its growth, since its inception, was funded in large part with temporary in-year funding, which is no longer available).

The effort to regularize CDA’s baseline funding started with a Formation-wide review of programs and activities that started in July 2012, and included a presentation to the Program Management Board in March 2013. While this resulted in $29 million of in-year funding being provided for FY 2013/14, there was no change to CDA’s baseline.

An Investment Plan Change Proposal that was developed last winter was intended to identify CDA’s baseline shortfall to a high level of detail; however, it did not receive the desired traction given the Department’s challenging fiscal situation for FY 2014/15 and beyond.

All of this being said, the upcoming Corporate Account Review is an opportunity for CDA to be re-base-lined to a sufficient funding amount as to ensure the long-term sustainability of the delivery of CDA programs to the CAF.

OPI: CDA

Target Date: April 2016

Management Action

The evaluation team also noted that increased effectiveness and efficiency may result from the judicious regrouping of some of the responsibilities of our institutions. The Osside Institute, for example, has already been absorbed under the command of RMC Saint-Jean, effective August 21, 2014, thereby greatly improving the capacity utilization of that national institution. The Report on the Optimization of RMC Saint-Jean of April 14, 2014, has also examined the idea of moving the Chaplains’ School to the campus, but rejected the idea based on the lack of synergy between the programs of RMC Saint-Jean and the Chaplains’ School. Similarly, the optimization report contained a recommendation to increase the number of Officer Cadet students at the College by 150, thus fully maximizing the use of the infrastructure available in Saint-Jean. Any examination of organizations potentially moving to Saint-Jean should take place based on synergies that are possible.⁵⁷ Regrouping orphan programs under RMC Saint-Jean will not necessarily improve efficiency, and may adversely affect the College’s current operational effectiveness.

The importance of economical stewardship of the infrastructure resources allocated to CDA’s IT&E mandate is understood. As a continuation of CDA’s focus on “the 3 Es,” a study will be initiated with the objective of performing a holistic review of CDA’s educational footprint, to include the RMCC, RMC Saint-Jean, and CFC, taking into account the opportunities for synergy with other CDA organizations, as applicable. Based upon the recommendations resulting from this study, any supported changes to the allocation of personnel will be targeted to be implemented by August 2016. Similarly, associated infrastructure changes (e.g., updating, divestments, transfers of responsibility, new building, etc.) will be implemented in close coordination with ADM(IE).

OPI: CDA

Target Date: August 2016

Annex B—Evaluation Methodology and Limitations

1.0 Methodology

The evaluation team used multiple lines of evidence and complementary qualitative and quantitative research methods to help ensure the reliability of information and data to support evaluation findings. The methodology established a consistent approach in the collection and analysis of data to support findings, conclusions and recommendations. Based on the evidence from available sources, the evaluation reviewed the achievement of expected outcomes, and the CDA’s efficiency and economy, to develop a balanced picture of the relevance and performance of the CDA. Information and data were correlated to each evaluation question and corresponding indicators. The team used a data triangulation approach to ensure the validity of data captured through different methodologies.

1.1 Overview of Data Collection Methods

As a first step, the evaluation team consulted websites for information on the CDA and its formation units, and the IT&E and PD systems. This permitted the team to develop an interview questionnaire for the preliminary interviews with the CDA HQ and formation units’ personnel. The purpose of the interviews was to obtain general information on CDA activities, courses and programs offered, and documents and data that could be provided to us. Following the analysis of the interviews and the documentation, a logic model and an evaluation matrix were developed by the evaluation team. These two documents led to the evaluation questions, as well as to the identification of the information and data necessary for the Evaluation. The logic model and the evaluation matrix were presented to the evaluation advisory group. The group suggested a few modifications. These documents are presented in Annexes C and D.

The evaluation team undertook the collection and analysis of various quantitative and qualitative data in order to respond to the evaluation questions. The research methods were chosen in function of the indicators. The quantitative and qualitative data are composed of the following sources:

• document and file review;
• financial and administrative data;
• key informant interviews; and
• literature review.

1.1.1 Document and File Review

The review of the CDA and formation units’ documents and files was conducted using a customized template organized according to the evaluation questions and indicators.

The following documents and files were consulted:

• governmental documents, such as Speech from the Throne, Government Priorities, and the CFDS;
• departmental documents, such as Reports on Plans and Priorities, Departmental Performance Reports, Departmental Priorities, the Program Alignment Architecture, and the Corporate Risk Profile; and
• CDA and the formation units’ documents and files—such as strategic documents, planning documents, policy documents, business plans, validation reports, CFITES manuals, research reports, committee meeting minutes, statistical reports on training, historical reports, and activity reports.

1.1.2 Financial and Administrative Data

The evaluation also examined and analyzed financial, personnel, and student data for the economy and efficiency section. This data was retrieved from multiple sources. The CDA provided all expenditure, personnel, and student data for its formation units. Estimates of regular forces pay were provided by the Cost Factors Manuals 2008/09–2012/13, from Assistant Deputy Minister (Finance and Corporate Services). The FTE data for Ontario universities was retrieved from the Ontario Council for University Libraries, and the financial data was retrieved from the Canadian Association of University Business Officers.

1.1.3 Key Informant Interviews

Interviews were conducted with CDA HQ and formation units’ representatives, utilizing standardized questions in order to acquire consistent information. The interviewees were given an interview guide in advance. During interviews, the evaluators asked additional questions for clarification, when required. Most interviews were done in groups (of 3 to 10 people). Each interview was directed by a member of the evaluation team, while others asked additional questions when necessary. Other interviews with the principal stakeholders were conducted on an individual basis. Individual interviews were also conducted with members of the PD Council, in order to get their perspectives on the CDA and the IT&E and PD systems.

1.1.4 Literature Review

A literature review was undertaken about military college training in allied countries, such as England, Australia, and the United States, and regarding their websites, publications, and reports. A literature review on IT&E and PD systems was conducted.

2.0 Limitations

Like all evaluations, the CDA evaluation has limitations, although many have been mitigated. Certain limitations stem from the Treasury Board Policy on Evaluation, which requires that each department evaluate one hundred percent of their expenses over a five-year period. This has a particular impact on the design and scope of evaluations at DND. Evaluations must encompass numerous activities and programs into one. This could not be mitigated, as it is an integral part of evaluations within the federal government.

Due to this constraint, the evaluation examined the activities of the CDA HQ most attentively, but could not provide as in-depth evaluations of the formation units. Nevertheless, it was able to identify the principal findings and challenges of the CDA and its formation units.

Additionally, the Basic Military Qualification and Basic Military Officer Qualification were excluded from this evaluation’s scope, as they were recently evaluated by CRS. Given the important role these play in the MES, these should be included in future CDA evaluations.

Mitigation strategies have been implemented to reduce the negative impacts on the results, as demonstrated in Table B-1.

Table B-1. Evaluation Limitations and Mitigation Strategies. This table lists the evaluation limitations and the corresponding mitigation strategy.

Lastly, certain limitations could not be mitigated, as is the case with data that was simply unavailable. Some data necessary for the evaluation is not currently being compiled by the CDA and the formation units. Additionally, it is almost impossible to compile true training costs since the military members’ salaries while on training are not counted. The implementation of a Performance Measurement Strategy should help mitigate these limits.

Annex C—Logic Model

Figure C-1. Logic Model for the CDA. This logic model demonstrates the relations between CDA’s main activities, outputs, and expected outcomes.

Annex D—Evaluation Matrix

Table D-1. Evaluation Matrix—Relevance. This table indicates the data collection methods used to assess the evaluation issues/questions for determining the relevance of the CDA.

Table D-2. Evaluation Matrix—Performance (Effectiveness). This table indicates the data collection methods used to assess the evaluation issues/questions for determining the performance in terms of achievement of outcomes (effectiveness) of the CDA.

Table D-3. Evaluation Matrix—Performance (Efficiency and Economy). This table indicates the data collection methods used to assess the evaluation issues/questions for determining the performance in terms of efficiency and economy of the CDA.

Annex E—CDA HQ and Formation Units’ Expenditures

Table E-1. CFSTG Expenditures ($millions). This table demonstrates the trends of CFSTG expenditures by categories for FYs 2008/09 to 2012/13.

Table E-2. RMCC Expenditures ($millions). This table demonstrates the trends of RMCC expenditures by categories for FYs 2008/09 to 2012/13.

Table E-3. CDA HQ Expenditures ($millions). This table demonstrates the trends of CDA HQ expenditures by categories for FYs 2008/09 to 2012/13.

Table E-4. CFLRS Expenditures ($millions). This table demonstrates the trends of CFLRS expenditures by categories for FYs 2008/09 to 2012/13.

Table E-5. RMC Saint-Jean Expenditures ($millions). This table demonstrates the trends of RMC Saint-Jean expenditures by categories for FYs 2008/09 to 2012/13.

Table E-6. CFC Expenditures ($millions). This table demonstrates the trends of CFC expenditures by categories for FYs 2008/09 to 2012/13.

Table E-7. CFLS Expenditures ($millions). This table demonstrates the trends of CFLS expenditures by categories for FYs 2008/09 to 2012/13.

Table E-8. RMCC Research Expenditures ($millions). This table demonstrates the trends of RMCC research expenditures by categories for FYs 2008/09 to 2012/13.

Table E-9. CDA Total Expenditures ($millions). This table demonstrates the trends in the overall CDA expenditures by categories for FYs 2008/09 to 2012/13.

Annex F—Number of Courses and Sessions Delivered by CDA HQ and Formation Units

Table F-1. Number of Courses, Sessions, and Graduates Delivered by CDA HQ and Formation Units. This table demonstrates the trends of the CDA HQ and formation units over FYs 2008/09 to 2012/13.

___________________________________________________________________________________________________________________________

Footnote 1 Treasury Board of Canada Secretariat, Policy on Evaluation, April 1, 2009, http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15024&section=text. Last consulted December 2, 2013.

Footnote 2 DND, Individual Training and Education Modernization Strategy. A Strategic Advantage for Future Operations, CDA, June 2011. CDA, CDA Business Plan FYs 2010/11 to 2013/14, Version 1.0. August 31, 2009.

Footnote 3 Treasury Board of Canada Secretariat, Policy on Evaluation, April 1, 2009, http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15024&section=text. Last consulted December 2, 2013.

Footnote 4 Defence Administrative Orders and Directives 5031-2. CMP is the “Functional Authority …for all CF professional development, including all IT&E. The Functional Authority is accountable to the CDS for providing strategic direction on how the CF manages IT&E.”

Footnote 5 The CDA is the training authority for the following occupations: logistics officer, traffic technician, cook, supply technician, ammunition technician, mobile support equipment operator, postal clerk, resource management support clerk, court reporter, musician, imagery technician, meteorological technician, communications research, and fire fighter.

Footnote 6 CRS, Evaluation of Recruiting and Basic Military Training, November 2012. http://www.crs-csex.forces.gc.ca/reports-rapports/pdf/2012/p0936-eng.pdf. Last consulted August 27, 2014.

Footnote 7 Treasury Board Secretariat, Directive on the Evaluation Function, April 1, 2009, http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15681&section=text. Last consulted July 4, 2014.

Footnote 8 From the Production, Attrition, Retention and Recruiting Analysis Reports, FY 2013/14.

Footnote 9 Defence Administrative Orders and Directives 5031-2. CMP is the “Functional Authority… for all CF professional development, including all IT&E. The Functional Authority is accountable to the CDS for providing strategic direction on how the CF manages IT&E.”

Footnote 10 Ibid.

Footnote 11 IT&E Modernization Strategy, 2011. Multiyear Business Plan for FYs 2010/11 to 2013/14.

Footnote 12 National Defence Program Alignment Architecture—Final. July 18, 2013.

Footnote 13 Canada, Seizing Canada’s Moment. Property and Opportunity in an Uncertain World. Speech from the Throne to Open the Second Session of the Forty-First Parliament of Canada, October 16, 2013. http://speech.gc.ca/sites/sft/files/sft-en_2013_c.pdf. Last consulted August 20, 2014.

Footnote 14 DND, CFDS, 2008, page 16. http://www.forces.gc.ca/assets/FORCES_Internet/docs/en/about/CFDS-SDCD-eng.pdf. Last consulted August 20, 2014.

Footnote 15 CFDS, op. cit., page 16.

Footnote 16 DND/CAF, Defence Priority documents, http://vcds.mil.ca/sites/intranet-eng.aspx?page=4871.

Footnote 17 National Defence Program Alignment Architecture – Final – July 18, 2013.

Footnote 18 DND/CAF, Corporate Risk Profiles, http://vcds.mil.ca/sites/intranet-eng.aspx?page=4431.

Footnote 19 CAFPDS Study Tier 2, March 13, 2014.

Footnote 20 CAFPDS Study update, PowerPoint presentation delivered to the Armed Forces Council on September 13, 2013.

Footnote 21 CAFPDS Study Tier 1, December 9, 2013.

Footnote 22 This is a Commander CDA priority, as explained in this document: CDA Campaign Plan Refresh, Major General J.G.E. Tremblay, Commander CDA, October 9, 2014, page 19.

Footnote 23 E-mail from CDA HQ Detachment Borden, June 12, 2014.

Footnote 24 An interview conducted in Borden, March 3–4, 2014, indicated that a rough estimate of carrying 2000 PAT in Borden represents annual costs of approximately $200 million. In addition to the financial cost, this also poses a concern for morale, as many individuals remain as PAT for periods of over 12 months.

Footnote 25 CDA responses to interview questions, July 17, 2014.

Footnote 26 CDA, Performance Measurements—DRAFT June 2014–Version 8c, June 29, 2014.

Footnote 27 CAF, Manual of Individual Training and Education, Volume 8: Validation of Instructional Programmes, October 15, 1998, page 6.

Footnote 28 There is a difference between the year the validation reports were conducted and published, and the year the courses were delivered. For example, the validations listed as being published in 2009/2010 and 2010/2011 were validating courses run in 2008. They amalgamated data from the same series of courses run in multiple years.

Footnote 29 The Your Say Survey 2012 had a substantial number of neutral respondents, ranging between 16.4 and 33.2 percent.

Footnote 30 Although there was a substantial number of neutral respondents in this section of the survey, ranging between 9.9 percent and 31.5 percent, areas of concern can still be identified.

Footnote 31 Official Languages Strategy for the DND/CAF, draft, provided on May 29, 2014.

Footnote 32 Recently, the CDA submitted a request for additional funding for the CAF Campus, based on two scenarios. The first allows for basic CAF Campus Enterprise Engine functionality ($750,000), and the second allows for advanced CAF Campus Enterprise Engine functionality ($2.2 million).

Footnote 33 CDA Campaign Plan Refresh, May 21, 2014, PowerPoint presentation.

Footnote 34 The CDA Campaign Plan covers the following topics: CDS’ Intent; CMP’s Guidance; CDA’s Mission and Vision; Commander CDA’s Operational Approach; CDA’s Priorities and Ends; CDA’s Conditions for Success; CDA’s End State; and CDA HQ Organization.

Footnote 35 Information provided via e-mail.

Footnote 36 The interviews were conducted with key stakeholders from the formation units between January and March 2014.

Footnote 37 CFITES manuals are available on the DIN at http://cda.mil.ca/pub/lib-bib/cfites-eng.asp.

Footnote 38 E-mail from CDA Staff Officer Change Management, April 24, 2014.

Footnote 39 Performance Measurement Working Group—Terms of Reference, Volume 2, May 2013.

Footnote 40 CRS, Evaluation of Military Individual Training and Education, 2005. http://www.crs.forces.gc.ca/reports-rapports/pdf/2005/P0466-eng.pdf. See also Review of the e-Learning Component of the Military Individual Training and Education, 2005. http://www.crs-csex.forces.gc.ca/reports-rapports/pdf/2005/P0662-eng.pdf. Last consulted on December 2, 2014.

Footnote 41 CFITES, Validation: Foundation Document.

Footnote 42 Preliminary interviews with formation units and CDA representatives conducted between December 2013 and February 2014.

Footnote 43 Performance Measurement Working Group—Terms of Reference, Volume 2, May 2013.

Footnote 44 The evaluation reviewed the Performance Measurements—Draft June 2014–Version 8c, June 29, 2014.

Footnote 45 Performance Measurements—Draft June 2014–Version 8c, June 29, 2014.

Footnote 46 DND/CAF Global Engagement Annual Planning Guidance—FY 2014/2015, February 12, 2014.

Footnote 47 An estimate for Regular Forces salaries was calculated by multiplying the number of Regular Forces personnel, received from CDA, by the average salary for Regular Forces personnel from the Cost Factors Manual for a given year. Also, an estimated 20 percent of all salaries (civilian and military) is added for employee benefits.

Footnote 48 Consistent student data for the evaluation period was not available for RMCC.

Footnote 49 Commander CDA Update August 24, 2011, CFB Borden/CFSTG IT&E Modernization.

Footnote 50 Student numbers specific to CFLS were not available. The student numbers include all military members undergoing language training, not only the students at CFLS.

Footnote 51 Ontario Council of University Libraries, Student Populations. http://www.ocul.on.ca/node/21. Last consulted July 17, 2014.

Footnote 52 Robert Hill (Minister of Defence), New Agreement for Australia’s Premier Military Officer Training Establishment, media release, Senate, December 11, 2003. http://www.defence.gov.au/minister/13tpl.cfm?CurrentId=3345. Last consulted June 12, 2014.

Footnote 53 Hugh Smith and Anthony Bergin, Special Report: Educating for the Profession of Arms in Australia. Australian Strategic Policy Institute, number 48, 2012, page 16. https://www.aspi.org.au/publications/special-report-issue-48-educating-for-the-profession-of-arms-in-australia/SR48_military-education.pdf. Last consulted December 17, 2013.

Footnote 54 CAF Campus Framework, page 29.

Footnote 55 Chemical, biological, radiological, and nuclear.

Footnote 56 An employment category within Educational Services.

Footnote 57 It should also be noted that any additional infrastructure renewal for RMC Saint-Jean is not currently supported by higher headquarters/GC. Any move to further increase the scope of the mandate for RMC Saint-Jean would require significant changes within, and support from, higher levels within the Department and the government.