Evaluation of Naval Forces

December 2013

1258-201 (CRS)

Reviewed by CRS in accordance with the Access to Information Act (AIA). Information UNCLASSIFIED.

Acronyms and Abbreviations

Executive Summary

This report presents the findings and recommendations of the evaluation of the Naval Forces Program activities within the Department of National Defence (DND). The evaluation was conducted by Chief Review Services (CRS) between November 2012 and July 2013 as a component of the DND/Canadian Armed Forces (CAF) Five-Year Evaluation Plan (fiscal year (FY) 2012/13 to 2016/17), and in compliance with the Treasury Board Secretariat (TBS) Policy on Evaluation (2009). As per the TBS policy, the evaluation examines the relevance and the performance of the program over a five-year period (2008-2013).

Program Description

Naval forces are defined as those elements of the Royal Canadian Navy (RCN), as well as supporting and related components, which are responsible for the generation and sustainment of naval readiness for employment in domestic/continental and international operations as directed by the Government of Canada (GoC).

The program is administered principally by three organizations within the DND: the RCN, the Assistant Deputy Minister (Materiel) (ADM(Mat)), and the Chief of Military Personnel (CMP). This program falls primarily under section 2.1 of the DND Program Activity Architecture (PAA) 2009 and involves annual expenditures in excess of $2.3 billion.

Relevance and Performance

Relevance. The evaluation determined that the need for DND to conduct both the generation and employment of naval forces is of continuing relevance and is aligned with federal government and departmental roles, responsibilities and priorities.

While the expenditures for Maritime Readiness represent 11 percent of the DND annual spending, the impact of its contribution to overall DND strategic objectives and priorities has been significant. The RCN contributions can directly impact and improve the safety and security of Canadians, and provide public confidence in the Government’s ability to defend Canada and protect Canadian citizens at home and abroad. The RCN involvement in operations also draws significant positive media attention, earning recognition both nationally and internationally.

Continued Need. There is ample evidence of continued need for the RCN to generate and sustain globally deployable naval forces, up to and including combat operations, as directed by the GoC. In addition, the RCN works in close cooperation with other government departments to deal with a wide range of security threats to Canada, including terrorism, illegal resource exploitation, pollution violations, narcotics trafficking and illegal immigration.

During the period of the evaluation, the program had serious challenges in meeting readiness targets (sufficient numbers of ships available for operations) as promulgated in readiness directives. The principal factor for this shortcoming stems from a shortage of available ships. The root cause appears to have been a combination of the following factors:

• the impact of the HCM program, which is removing a significant number of ships from service;
• the high operational tempo during the evaluation period;
• under-funding for equipment maintenance; and
• a lack of capacity within both ADM(Mat) and the fleet maintenance facilities.

Various initiatives are under way to address these issues. For example, the HCM will be complete by FY 2017/18, and ADM(Mat) has significantly improved its ability to meet equipment maintenance demands to the extent that going forward funding allocations will be the main challenge in addressing maintenance issues. Furthermore, the Department is undergoing numerous other initiatives to improve efficiencies as a means to address the funding gap.

Despite shortcomings in meeting readiness targets during the evaluation period, the RCN has consistently met demands for the delivery of combat-capable forces for operations, both domestically and abroad. By and large, the forces employed have met DND mission requirements and expectations. As such, the RCN is highly regarded internationally, and among our allies, it is seen to be extremely capable, interoperable and reliable.

Efficiency and Economy. With respect to efficiency and economy, the program is seen to have demonstrated sound value. Numerous efficiency improvements were noted, including a 24-percent increase in maintenance services delivered despite a reduction of 23 percent in the cost of personnel who deliver those services. Infrastructure costs were kept in line despite inflation. While the overall cost of the program has increased by 6 percent annually, this is largely a reflection of the much-needed increase in RCN personnel numbers. Benchmark indicators with allied navies demonstrate that although RCN personnel numbers have grown, given the age and composition of the fleet in 2012, the RCN has a reasonable number of personnel. Furthermore, with respect to overall expenditures against the size and composition of the fleet, in a comparison to benchmark allies (New Zealand, Germany, Australia, Netherlands, United Kingdom, and Spain), the RCN appears at the lower end of the spectrum.

The Naval Forces Program has demonstrated operational effectiveness despite numerous challenges. During the evaluation period, the program has faced pressures due to fleet modernization, age of equipment and increased operational tempo. However, while the program has taken steps to be more efficient in order to address its challenges, a principal concern over the availability of financial resources for priority areas will remain and will need to be carefully addressed.

The findings and recommendations of the evaluation are presented as follows:

Findings and Recommendations

Findings—Relevance

Key Finding 1: There is evidence of continued need for the RCN to generate and sustain globally deployable naval forces to conduct domestic/continental and international operations, up to and including combat operations, as directed by the GoC.

Key Finding 2: There is alignment between the RCN’s generation and delivery of Naval Forces and departmental and federal roles and responsibilities.

Key Finding 3: The generation and delivery of combat-ready naval forces aligns directly with the government priority of “A Safe and Secure World through International Engagement.

Findings—Performance (Effectiveness)

Key Finding 4: During recent years there has been a steady decline in the RCN’s ability to achieve the required levels of readiness, to the point that it is currently challenged to meet some of its readiness requirements.

Key Finding 5: In order to meet readiness levels and force posture requirements, the RCN has had to rely more heavily on the use of Maritime Coastal Defence Vessels (MCDV) as a result of reduced frigate availability during the period of the HCM. This represents a reduction in force capability.

Key Finding 6: The HCM and Victoria-class Introduction (VCI) programs, essential to the long-term viability of the Navy, have strained Navy resources and created issues in meeting current force posture and readiness targets.

Key Finding 7: Among available ships, technical readiness and combat training are the primary challenges towards achieving prescribed readiness levels.

Key Finding 8: Taking into consideration available units and funding, the RCN will remain challenged to meet prescribed readiness and force posture requirements for some time in the future. The Navy will be obliged to do less with less.

Key Finding 9: Despite challenges with readiness targets, the RCN has consistently met GoC and DND expectations and demands for the conduct of operations.

Key Finding 10: The Chief of Maritime Staff (CMS) preliminary readiness direction to formations led to an unintended reduction in the required materiel state of ships¹ entering the HCM and constrained the Navy’s force generation (FG) capabilities. The direction was subsequently revised to address these issues.

Findings—Performance (Efficiency and Economy)

Key Finding 11: The significant increase in personnel over the evaluation period was in response to personnel shortages. In terms of overall personnel numbers, the RCN appears to be properly sized.

Key Finding 12: Significant improvements in the overall efficiency of fleet maintenance have occurred over the evaluation period.

Key Finding 13: Despite the efficiency improvements, there remains a funding gap in maintenance within the RCN.

Key Finding 14: Maritime Infrastructure Maintenance and Base Support costs have declined due to a reduction in personnel. However, these costs represent a larger proportion of the RCN’s Operations and Maintenance (O&M) budget, resulting in fewer resources being available for other O&M expenditures.

Key Finding 15: The Navy is delivering maritime training with increased efficiency. The overall cost of maritime training has decreased since FY 2008/09, whereas the number of people trained and the number of courses offered has increased.

Key Finding 16: On average, the percentage of personnel trained at optimal course capacity was 54 percent. The RCN Performance Measurement Framework (PMF) sets a target for the percentage of personnel trained at optimal course capacity at 90 percent.²

Key Finding 17: As a percentage of total readiness, Maritime Command and Control represents 7 percent of total costs on average. Over the evaluation period, the costs attributed to the Maritime Command and Control have increased significantly (40 percent) due to a large influx of personnel as a result of reductions or elimination of other coastal formation establishments and the transition of the Navy’s coastal maritime headquarters (HQ) to Regional Joint Headquarters.

Key Finding 18: The program expenditures for Maritime Readiness have increased during the evaluation period. However, they have not exceeded reasonable amounts for a fleet of the size and composition of the RCN.

Key Finding 19: Overall costing data for RCN should include several Level 1 (L1) accounts, for example the PAA 2.1 Maritime Readiness includes CMP, RCN and ADM(Mat). It was noted that the RCN did not assess all its program expenditures (nor use performance indicators) that include all costs such as labour and materiel. Therefore, a true indication of costs and performance is not available, which may impact the ability to assess overall performance.

Recommendations

Recommendation 1: It is recommended that the RCN review its current readiness program, enshrined in documents such as Readiness and Sustainment Direction (Canadian Forces Controlled Document (CFCD) 129) and Maritime Command Combat Readiness and Training Requirements (CFCD 102), to rationalize unit readiness requirements.

Recommendation 2: The RCN should review and identify budget savings in its Infrastructure Maintenance and Repair (IM&R) portfolio. In addition, it is recommended that the RCN review the expected IM&R targets for the RCN with Assistant Deputy Minister (Infrastructure and Environment) (ADM(IE)) to prevent any possible future reduction in ship readiness.

Recommendation 3: Further optimize training by more fully loading courses and/or by reducing frequency of course offerings. This will reduce training costs associated with travel and cost of operating trainers, such as Naval Combat Operations Trainer (NCOT), Operations Room Team Trainer, Navigation Trainer, and Submarine Control Trainer.

Recommendation 4: During the evaluation period, the RCN’s command and control function experienced significant personnel increases and associated costs. It is recommended that the RCN review these increase and identify efficiencies where possible to address emerging future manning priorities.

1.0 Introduction

1.1 Profile of Canada’s Naval Forces

1.1.1 Background

This report presents the results of the evaluation of the RCN generation of combat-capable Naval Forces. Naval forces are defined as those elements of the RCN, including supporting and related components, which are responsible for the generation and sustainment of naval readiness for employment in domestic/continental and international operations as directed by the GoC.

The evaluation was conducted by CRS between November 2012 and July 2013 as a component of the DND/CAF Five-Year Evaluation Plan (2012/13 to 2016/17) and in compliance with TBS Policy on Evaluation (2009). As per the TBS policy, the evaluation examines the relevance and the performance of the program over a five-year period (2008-2013).This evaluation, conducted on behalf of the Chief of the Defence Staff (CDS) and the Deputy Minister, will be used to inform future senior management decisions related to the development, generation and sustainment of Canada’s Naval Forces and their supporting activities.³

There have been no previous CRS evaluations of Canada’s Naval Forces. However, the following audits related to naval fleet maintenance and capital equipment acquisition projects were conducted by CRS’ Internal Audit division:

• CRS Review of the Submarine Acquisition/Capability Life-Extension Program, May 2003;
• CRS Audit of the Halifax-Class Modernization/Frigate Equipment Life Extension (HCM/FELEX) Project, March 2011; and
• CRS Internal Audit of Joint Support Ship (JSS) Project, November 2011.

In addition, elements of maintenance and materiel support essential to the technical readiness of the Navy were covered in the following Auditor General reports:

• Report of the Auditor General of Canada to the House of Commons, Chapter 5 Maintaining and Repairing Military Equipment – National Defence (Fall 2011); and

• Report of the Auditor General of Canada to the House of Commons, Chapter 5 Real Property – National Defence (Fall 2012).

1.1.2 Program Objectives

The program falls under the following program activity of the DND/CAF (PAA 2009):

• 2.1 Maritime Readiness and Sub-activities
• 3.2 Canadian Peace Stability and Security
• 3.3 Continental Peace Stability and Security
• 3.4 International Peace Stability and Security

The objectives of these program areas are to support DND in meeting GoC expectations with regard to the defence of Canada, partnering in the defence of North America, and contributing, when called upon, to international peace and security operations. More specifically, these expectations are articulated in the six missions of the Canada First Defence Strategy (CFDS).

The specific activities, outputs and outcomes involved in the RCN’s generation and delivery of combat-ready naval forces are illustrated in the program logic model at Annex C.

1.1.3 Program Description

While the expenditures for Maritime Readiness represent 11 percent of the DND annual spending, the impact of its contribution to overall DND strategic objectives and priorities has been significant. The RCN’s contributions can directly impact and improve the safety and security of Canadians, and provide public confidence in the Government’s ability to defend Canada and protect Canadian citizens at home and abroad.⁴ The RCN’s involvement in domestic/continental and international operations also draws significant positive media attention, earning recognition both nationally and internationally.

The RCN also works in close cooperation with other government departments to deal with a wide range of security threats to Canada, including terrorism, illegal resource exploitation, pollution violations, narcotics trafficking and illegal immigration.

The RCN is directed through Naval Staff Headquarters (NSHQ) by the Commander of the Royal Canadian Navy (CRCN), whose mission is to lead the strategic development and generation of combat-capable multipurpose naval forces and to provide advice in support of maritime operations.

The RCN fleet, roughly balanced between the Atlantic (Halifax, Nova Scotia) and Pacific (Esquimalt, British Columbia) coasts, is comprised of three destroyers, twelve frigates, two replenishment ships, four submarines and twelve MCDVs, plus minor auxiliary and support vessels. These formations, known as Maritime Forces Atlantic (MARLANT) and Maritime Forces Pacific (MARPAC), manage all aspects of fleet maintenance, training and manning, while ensuring the fleet is operationally ready for deployment, exercises and operations. Deployments can involve the assignment of a single ship to a multinational task force, or a larger essentially self-sufficient task group of complementary ships, submarines and aircraft tailored to meet mission specific requirements.⁵

To respond to these missions, the RCN generates and sustains relevant, responsive, combat-capable naval forces that can respond to a wide spectrum of tasks. The RCN generates maritime forces through the following program activities:

• Maritime Training. This activity provides training through five naval training establishments, and contributes to the sustainment of naval capabilities through the provision of training to personnel. This includes continuing occupation and environment training and appropriate professional development and education activities. Currently, at-sea training is supplemented with simulators and other tools such as e-learning and the Defence Learning Network, often used in lieu of classroom training.
• Maritime Infrastructure Maintenance and Base Support. This activity, provided by the Director Naval Infrastructure Requirements reporting to the ADM(IE), captures all activities dealing with the maintenance and repair of naval realty assets. This is achieved through the maintenance and repair of naval infrastructure and the provision of administrative and logistical support. Base support involves the human resource and material support provided to all units.
• Maritime Equipment Maintenance. This support is provided through the Director General Maritime Equipment Program Management (DGMEPM) who is responsible to the ADM(Mat) and responsive to the CRCN. DGMEPM is the materiel authority for all naval ships, submarines, auxiliary vessels and naval equipment for shore establishments. This activity ensures that RCN ships are maintained and repaired both domestically and abroad. The work comprises preventive and corrective maintenance, as well as some engineering changes (EC) and fitting of mission-specific equipment for a given operational activity or deployment.
• Maritime Command and Control. This activity ensures effective governance, processes and headquarters organizations are in place for the application of command and control. This includes strategic, resource and capability plans, human resource and financial management, and logistics and resource allocation functions. Naval Readiness and Sustainment is managed by the RCN. Documents issued by the CRCN include Commander’s Planning Guidance and, more recently, Readiness Direction to Formations, which provide direction for the generation of naval readiness requirements. The role of the Navy’s governance structure⁶ is to provide advice and recommendations on emerging issues affecting the Navy and DND/CAF. The senior executive council for the Commander RCN is the Naval Board.
• Operations. This activity ensures suitable support is provided to naval forces, whether they are being force-generated or employed on operations. This includes five-year plans, operational schedules, policies and briefs as well as providing services such as intelligence, harbour services and electronic warfare for the smooth manning and provision of ships to support various naval task groups.⁷ While the generation of combat-ready forces leading to operations is an RCN activity, the planning and conduct of all CAF operations is a Canadian Joint Operations Command (CJOC) responsibility.

Once generated, the maritime force capability is considered operational and maintained at a designated level of readiness for operational employment.⁸ Readiness consists of two components—operational capability and response time—and is a measure of the ability of an element of the CAF⁹ to undertake an approved task.¹⁰

1.1.4 Stakeholders and Partners

For the purposes of this evaluation, the term “stakeholder” refers to individuals, groups or organizations within DND that may be accountable for the generation and employment of maritime readiness and the conduct of CAF operations. These specifically include the force generator (RCN) and force employers (CJOC), as well as other allied/partner Nations’ Navies).

Partners, vice stakeholders, also include other federal government departments and agencies including Public Safety Canada, Department of Fisheries and Oceans, Canada Border Services Agency and the provincial and territorial governments.

Canadians ultimately benefit from maritime readiness as the DND/CAF has the relevant capability to meet maritime defence and security missions as may be tasked by the Government.

1.2 Evaluation Scope

1.2.1 Coverage and Responsibilities

The evaluation focused on the relevance and performance of Maritime Readiness, DND/CF PAA (2009) Activity 2.1, specifically, the generation and sustainment of relevant, responsive, combat-capable multipurpose naval forces. In order to validate the RCN’s capability to respond and perform in government-directed operations, the evaluation also studied the RCN’s involvement in CAF-directed domestic/continental and international operations as per DND/CAF PAA Activities 3.2 to 3.4.

The evaluation does not cover initial training activities prior to advanced maritime training (PAA 2.1.5.1) nor Maritime Equipment Acquisition and Disposal (1.3.1.0) or Maritime Real Property Acquisition or Disposal (1.4.1.1).

1.2.2 Resources

Table 1 describes the costs attributed to the Maritime Readiness Program (PAA 2.1), the costs attributed to naval operations conducted by CJOC (Force Employment), and the number of military (Regular Force (Reg F) and Primary Reserve (P Res)) and civilian full-time equivalents (FTE) employed by the RCN.

Table 1. Generation and Delivery of Canada’s Naval Forces.

(1)Source: PAA 2.1 Maritime Readiness expenditures

(2)Source: MARCOM RCN Mil Civ_FY 200809 to FY 2012/13

Note: Military population represented at effective strength, including Reg Force and P Res; FY 2012/13 values are preliminary.

For FY 2012/13, the combined costs for Maritime Readiness and naval operations (Force Employment) are at $2.40 billion. During the evaluation period, costs for Maritime Readiness have increased by 5.9 percent per year (more than DND’s budget of approximately 2.1 percent per year). The costs attributed to naval operations (Force Employment) have decreased by $50.4 million, which led to an average annual decrease of $12.6 million in combined costs. The total number of FTEs has decreased by 5 percent.

1.2.3 Issues and Questions

In accordance with the core evaluation issues noted in the Treasury Board’s Directive on the Evaluation Function,¹¹ this evaluation has considered the following:

2.0 Findings and Recommendations

The following sections evaluate the relevance and performance of Canada’s Naval Forces. The evaluation examined the extent to which the generation and delivery of combat-ready naval forces addresses an actual and ongoing need, is aligned with GoC priorities and DND strategic outcomes, is appropriate to the role of the federal government, achieves the intended outcomes, and demonstrates efficiency and economy.

2.1 Continued Need

Does the generation and delivery of combat-capable naval forces address an actual and ongoing need?

To determine whether the generation and delivery of combat-ready naval forces continues to address a demonstrable need, two key indicators were used:

• the likelihood of future need (threats); and
• the number and type of actual operational employments of Canada’s Naval Forces over the past five years.

The following findings are based on evidence from document reviews and key informant interviews with senior CJOC officers.

Key Finding 1: There is evidence of continued need for the RCN to generate and sustain globally deployable naval forces to conduct domestic/continental and international operations, up to and including combat operations, as directed by the GoC.

While Canada's immediate security environment remains relatively stable and secure, Canada continues to face a number of significant security concerns.¹² In 2008, by issuing the CFDS, the Government committed to making sure that Canada has the tools it needs to deal with the full range of threats and challenges to Canada and Canadians.¹³

Over the past five years, the RCN was deployed both domestically and internationally in response to a wide range of crisis and threats as directed by the GoC.

In domestic operations, the RCN was employed in an integrated, whole-of-government approach¹⁴ to maritime security in a variety of operations (6 operations, 25 ships). Internationally, the RCN has been conducting counter-terrorism interdiction operations with key allies and regional partners in the seaward approaches to the Arabian Peninsula (15 operations, 20 ships). These forward deployments were also used for counter-piracy operations in the area of the Horn of Africa. The RCN was also required to respond quickly to unfolding crises, to prevent and contain conflict elsewhere.

In North American/Continental Operations, following the GoC’s Americas Strategy in 2007, the Canadian Navy contributed to government priorities by taking part in defence diplomacy and counter-narcotics operations throughout the Caribbean and Latin America (4 operations, 36 ships, 1 submarine).

Interviews with CJOC senior officers underlined the rising global tensions and confirmed that Canada’s Naval Forces have been highly relevant to international, domestic and North American defence and have been critical to the force employer’s operational success. Senior external informants confirmed that the requirement for naval combat capability continued, despite the fact that over the past decade the Navy was called upon more often than before for humanitarian and disaster relief missions and constabulatory activities.

Most informants, including the external ones, emphasized the need to maintain or increase the Navy’s inter-operability with foreign navies and its combat capabilities, and improve its fleet readiness in the future, due to increasing global tensions and maritime security threats both at home and abroad.

2.2 Alignment with Federal Roles and Responsibilities

Is the generation and delivery of combat-ready naval forces consistent with the roles and responsibilities of the federal government and more specifically the roles and responsibilities if the DND/CAF?

This section examines the extent to which the generation and delivery of naval forces by the RCN aligns with departmental and federal roles and responsibilities. The findings in this section are based on documents reviewed and key informant interviews, including senior NSHQ and CJOC officers.

The following indicators were used in the assessment of alignment with federal roles and responsibilities:

• alignment with government Acts, legislation and policies; and
• the extent to which Canada’s Naval Forces conduct activities that are the responsibilities of other government departments, other levels of government or the private sector.

Key Finding 2: There is alignment between the RCN’s generation and delivery of Naval Forces and departmental and federal roles and responsibilities.

Defence is a core federal government responsibility as articulated in the Constitution Act,¹⁵ which defines and outlines the responsibilities and duties of the federal government, including the armed forces and defence.

Furthermore, the National Defence Act, Article 17 establishes DND and the CAF as separate entities, operating within an integrated National Defence Headquarters (NDHQ), as they pursue their primary responsibility of providing defence for Canada and Canadians. Moreover, the Act assigns the Minister as the authority in charge with all matters relating to defence, including the Land, Naval and Air Services of Canada.

In addition, within the Whole-of-Government Framework,¹⁶ the generation and delivery of combat-ready naval forces fall within the “Safe and Secure World through International Engagement” outcome area. Accordingly, it is a priority of the Government to promote peace and security, freedom, democracy, human rights and the rule of law throughout the world, and the RCN (as an FG component of the CAF) has the relevant and credible capacity to meet defence and security commitments.

2.3 Alignment with Government Priorities

Are the priorities of this program consistent with DND strategic objectives and federal government priorities?

This section examines the extent to the generation and delivery of combat-ready naval forces is consistent with DND strategic objectives and federal government priorities. The findings in this section are based on evidence from documents reviewed for the evaluation, which include Federal Budget Plans (2008-2012), CFDS and Reports on Plans and Priorities.

Key Finding 3: The generation and delivery of combat-ready naval forces aligns directly with the government priority of “A Safe and Secure World through International Engagement.”¹⁷

The following indicators were used to make this determination:

• alignment between priorities of the Canada’s Naval Forces and federal government priorities; and
• alignment between Canada’s Naval Forces priorities and DND/CAF priorities.

Budget 2008 supported the Government's “leadership abroad” agenda under item “Protecting Canadians and Canada.” Moreover, by issuing the CFDS (2008), the GoC provided the CAF with stable and predictable funding to permit long-term planning, setting up clear priorities to guide future actions, including the capacity to project leadership abroad by making meaningful contributions to operations overseas.

Document review suggests that the delivery of combat-ready naval forces by the RCN is directly aligned with the DND/CAF priority of “Ensuring Operational Excellence Both at Home and Abroad.” Through its program activities leading to generation of deployable, combat-ready naval forces, this program provides direct support to CAF combat-ready forces in support of the six core missions¹⁸ of the CFDS. Therefore, it is deemed to be a high priority for the DND/CAF.

Finally, the Report on Plans and Priorities for the Department (FYs 2008/09 to 2011/12) placed specific emphasis on increasing the RCN’s operational capability. These reports consistently included the priority of “…ensuring operational success at sea by maintaining a highly effective but ageing fleet, while preparing to meet the challenges of a renewed three-ocean readiness.”

2.4 Achievement of Expected Outcomes (Effectiveness)

This section evaluates the achievement of Naval Forces’ expected outcomes, with a focus on the immediate outcome: the RCN’s ability to generate combat-capable multipurpose forces. As detailed in the Departmental Performance Report 2011-12 for Program Activity (PA) 2.1 Maritime Readiness,

Intermediate outcomes are considered in this evaluation insofar as they support the assessment of the RCN’s effectiveness in generating combat- capable multipurpose forces.

Findings with respect to effectiveness have been based on program readiness data; RCN and departmental documents, including the Chief of Maritime Staff (CMS) / CRCN assessments and directives, as well as operations reports; interviews with key staffs, and responses to a questionnaire administered to Commanding Officers (CO) of commissioned ships, submarines and Fleet Diving Units.

2.4.1 Immediate Outcome

To what extent have Canada’s Naval Forces achieved required levels of readiness?

Broadly speaking, readiness refers to the capacity (number), responsiveness and capability of Naval Forces to conduct operations or tasks as directed. The levels of readiness for Naval Forces are directed in the Canadian Armed Forces Force Posture and Readiness.

Key Finding 4: During recent years there has been a steady decline in the RCN’s ability to achieve the required levels of readiness, to the point that it is currently challenged to meet some of its readiness requirements.

The evaluation used the following indicators to make this determination:

• Capacity—the number of units which have achieved prescribed personnel, materiel/ equipment, and training readiness levels for HR and Standard Readiness (SR).
• Responsiveness—NTM capacity – number of units available at prescribed NTM in accordance with strategic direction.
• Capability—Alignment between what naval forces are trained and equipped to do, and what tasks and missions they are required to perform on operations.

Capacity and Responsiveness

A review of Navy readiness reports over the period of the evaluation revealed the RCN has been challenged to achieve and maintain prescribed readiness levels for several years. This was found at both SR and HR levels.

In accordance with CFCD 129 Readiness and Sustainment (2009), SR represents the normal level of readiness for all maritime operational capability across the Navy. Units at SR comprise a broad zone of capability that is employed for the purposes of conducting core naval training and executing assigned CAF continental and expeditionary missions that do not entail the possibility of high-intensity, full-spectrum combat. SR ships will be crewed as a minimum, to a level between 75-85 percent of full complement. These ships are to be available at 10 days’ NTM to deploy at SR, or 90 days to generate to HR if required. Should an SR unit be deployed on a continental or expeditionary mission, its crewing and other task specific needs will be determined against mission requirements.

HR units shall be capable of conducting the full spectrum of combat operations and are to be available at 10 days’ NTM. HR units will have undergone additional levels of training based on both the mission and the intensity requirements of full-combat operations. HR units will be earmarked for inclusion in the Composite Contingency Task Group but may deploy independently depending on mission requirements. While a ship may meet technical and personnel requirements at the HR level, a mission work-up will determine achieving HR for mission employment. HR units will be fully crewed and may, in addition, include other specialized personnel based on mission requirements. HR ships shall also have successfully completed and maintained validity of 90 percent of team and combat training requirements in accordance with CFCD 102 Maritime Command Combat Readiness Training Requirements. The materiel state requirements for HR units will be determined by the type of mission and resource availability; however, an HR ship shall not have any significant operational defects.

A review of RCN documents highlights the challenges it faces with respect to meeting readiness targets. From FY 2009/10 to FY 2012/13, the ability to generate required numbers of frigates and destroyers at both HR and SR levels, as outlined by the CAF Force Readiness Posture, was extremely limited and in fact declined over the evaluation period. The fleet capacity was under pressure due to the number of frigates entering the HCM program and delays in the completion of the VCI program. Figure 1 depicts the decline in fleet readiness and associated risk, as the Halifax-class frigates go through the HCM/FELEX program, the Victoria-class submarines achieve full operational capability, Iroquois-class destroyers and replenishment ships are retired from service, and new shipbuilding programs begin to deliver product. Note that timelines were approximate and the timelines for new ship program deliveries and ship disposals have been delayed since 2010. As such, the readiness targets will continue to be under pressure and will not be met going forward well past 2020.

Figure 1. Fleet Readiness versus Risk.

Source: MARCOM Strategic Assessment – (2010) Amendment, Figure 2 Fleet Readiness

During the five-year evaluation period, it was noted that while the numbers of ships designated at HR and SR have essentially been maintained, the RCN has only been able to do this by relying on the use of MCDV as SR units in place of the Halifax-class frigates as they undergo modernization, and there has been reduced availability of the older destroyers and auxiliary oil replenishment (AOR) ships as reflected in Table 2. The relatively stable percentage of operational sea days attributed to frigates reflects the continuous deployment of a HR frigate to international operations since 2011 and regular deployments of SR frigates to Operation CARIBBE, even while the number of frigates available for operations has diminished.

The concern is the limited operational capability of the MCDV (a 950-ton ship) compared to a frigate (4,700 tons). As a result, the overall capability of Naval Forces has been diminished, which can translate into impacts on operational effectiveness. For example, a shortage of HR capacity meant that non-modernized SR frigates with reduced capabilities in accordance with the CRCN direction to formations were utilized in conjunction with MCDVs in Operation CARIBBE. Several issues were noted with these units which had an impact on operational effectiveness (see section 2.4.2).

Table 2. Distribution of Operations Sea Days by Ship Class (Less Submarines).

Source: Naval Staff Headquarters

Key Finding 5: In order to meet readiness levels and force posture requirements, the RCN has to rely more heavily on the use of MCDVs as a result of reduced frigate availability during the period of the HCM. This represents a reduction in force capability.

Another major concern was serious challenges for HR-designated frigates and destroyers to achieve their prescribed readiness levels. Challenges to achieve the target readiness levels were related to the following:

• technical readiness;
• effective manning strength; and
• combat readiness training.

Technical readiness was not an issue during the first three years of the evaluation period (FY 2009/10–FY 2011/12). However, issues have increased in this area. As of FY 2012/13, it has become the leading problem with respect to the program’s ability to meet prescribed readiness levels. The majority of stakeholders interviewed by the evaluation team, including the Commander of the RCN and the Commanders of both coastal formations, noted that the ageing fleet and equipment have created challenges for materiel readiness and sustainment. A lack of sufficient resources to meet maintenance demands has led to reduced ship technical readiness levels, with the restriction on funding and Fleet Maintenance Facility resources available to support the older fleet. This has been anticipated in CRCN direction to formations that has been intended to mitigate the impact on the HCM and Victoria-class programs as well as maintain funds required for FG. Between FY 2008/09 and 2011/12, DND/CAF spending on Maritime Maintenance has averaged $880.7 million per year, with an average annual increase of 2.9 percent. Increases in the cost of maritime equipment maintenance were largely driven by ADM(Mat) expenditures in National Procurement (NP),¹⁹ which increased by $129 million over the evaluation period (average of 7.5 percent annually). A more detailed explanation of the rise in NP expenditures is provided in the Efficiency and Economy Section of this report.

Key Finding 6: The HCM and VCI programs, essential to the long-term viability of the Navy, have strained Navy resources and created issues in meeting current force posture and readiness targets.

Key stakeholders, including the CRCN, commented that maintenance for older ships (see Table 3) is increasingly expensive and difficult due to the lack of availability of spare parts from original equipment manufacturers for obsolescent equipment. As a result, there is an increasing reliance on maintenance facilities to repair and overhaul broken equipment when possible, or even manufacture new replacement parts.

Table 3. Average Age of RCN Warships.

Source: IHS Jane’s Fighting Ships on line (DND Canada Group User) at https://janes.ihs.com/CustomPages/Janes/Home.aspx. Consulted October 11, 2013

Examples of obsolescent equipment that is difficult to maintain include the steam propulsion and power generation systems in the AORs, and the solar gas turbine generators in both the AORs and destroyers. Without these components, these vessels cannot function and the only manufacturer that can support these units will cease to have components available in FY 2014/15—meaning that the fleet must procure enough spares in advance to ensure operation beyond those years. Table 4 compares the average age of fleets of various allied navies. While age does not equate to capability due to the initial performance and modernization, it is a reflection of difficulty in maintenance and support.

Table 4. Average Age of RCN and Allied Warships and Number of Units.

Source: IHS Jane’s Fighting Ships on line (DND Canada Group User) at https://janes.ihs.com/CustomPages/Janes/Home.aspx. Consulted October 11, 2013

With respect to personnel availability, key stakeholders interviewed and 40 percent (23/57) of respondents to the CO questionnaire noted manning shortages in technical trades (in particular, a shortage of qualified engineering watchkeepers), as affecting ships’ readiness levels. Interviewees and CO questionnaire respondents also cited falling levels of experience, reduced “sea time” and early promotion as a source of growing concern, as personnel being promoted early in some instances lack the requisite trade qualifications for that rank, and more often lack the experience required for that rank level’s responsibilities. That being said, the analysis performed by the evaluation team has indicated that only rarely (2 percent of all causes) has a lack of effective manning strength resulted in an inability to meet a readiness target Figure 2 demonstrates the RCN’s manning shortages from FY 2008/09 to FY 2012/13.

Figure 2. Naval Forces Personnel Manning. This graph compares the authorized Planned Manning Level (PML) strength with the actual TES of Naval Forces between FY 2008/09 to 2012/13. The data is summarized in Table 5.

Source: Naval Personnel Report FY 2012-13, February 1, 2013

Table 5. Naval Forces Personnel Manning.

From a ship level or task group level training perspective, the Navy has been challenged by its reduced platform availability and financial resources to undertake more complex and challenging warfare training critical to its readiness requirements. During the evaluation period, the RCN had approximately 40 percent of the fleet undergoing refit or modernization.

The RCN has been able to maintain the total number of days ships have spent at sea (sea days) over the evaluation period; however, it was noted that a sizeable portion (17 percent) of sea days were on actual operations and much of the remaining sea time was single ship activity for FG with a limited ability to conduct combat readiness training exercises. While training can occur in such circumstances, often the nature of the operations undertaken does not offer opportunities for specific types of combat training, such as anti-submarine warfare or anti-air warfare exercises. Further compounding this problem was the fact that many of the major warships (frigates, destroyers, submarines and AORs) were the ones undergoing refit or modernization during this period, limiting the RCN’s ability to conduct the required multi-ship combat readiness training. To offset these challenges, the RCN has undertaken various mitigation measures, including the use of simulators, posting of personnel with other navies and participation in major North Atlantic Treaty Organisation (NATO) or United States-led multinational exercises. For example, it has taken advantage of opportunities to participate in the United States Navy (USN) Rim of the Pacific (RIMPAC) exercise that permit both individual units and task groups to train across the full spectrum of operations with the additional benefit of reduced cost to the Navy and the CAF.

Despite these mitigation attempts, over the evaluation period it was noted that a lack of combat-readiness training has in fact been the leading factor of readiness shortfalls (only recently overtaken by technical readiness issues), responsible for almost 70 percent of readiness deficiencies.

Meanwhile, the return of operational submarines to the RCN has begun to benefit the Navy, providing invaluable anti-submarine warfare readiness training for RCN surface combatants and the Royal Canadian Air Force (RCAF)’s maritime air assets. As an aside, the RCN’s diesel submarines are considered high-value training assets by the USN.

Key Finding 7: Among available ships, technical readiness and combat training are the primary challenges towards achieving prescribed readiness levels.

Ship COs have reported that challenges exist with regards to the quality of naval training and the experience levels of their subordinates. When asked to rate personnel training and experience, the most cited comment (by 34 percent of COs) was the shortage of qualified and experienced personnel across certain trades and departments. COs also felt that personnel had been promoted too quickly and noted that their crew’s lack of sea time was a challenge. This has been noted as particularly significant in Navy reports pertaining to submarine crew generation, as seen in Figure 3, as a result of a the lack of submarines available to conduct crew training.

Figure 3. Naval Forces Personnel Manning (Submarines). This graph describes actual and projected TES and the PML, i.e., the required number of trained submarine personnel. The data is summarized in Table 6.

Source: Naval Personnel Report FY 2012/13, February 1, 2013

Table 6. Naval Forces Personnel Manning (Submarines).

As a result of these challenges, the RCN has had to review its ability to achieve prescribed readiness levels and force posture. Since early 2010, the CMS/CRCN has issued annual readiness direction to formations in response to these challenges and provided direction to mitigate them, acknowledging that this would lead to a decline in the fleet’s capabilities. In June 2010,²¹ the CMS noted,

Later, in June 2012,²³ the CRCN noted in his readiness direction that, “Operational Commanders are to emphasize a ‘come as you are’ approach, and accept that there will be some missions that may not be possible. When absolutely necessary, deviations from this direction are permissible.”

Key Finding 8: Taking into consideration available units and funding, the RCN will remain challenged to meet prescribed readiness and force posture requirements for some time in the future. The Navy will be obliged to do less with less.

To summarize, the current RCN readiness level and force posture challenges are the result of several coincidental causes:

• the availability of ships during the HCM and VCI programs; and
• reduced budgets, specifically O&M and NP funding required to operate and maintain Naval units.

The impact of reduced O&M and NP funding is currently exacerbated by the challenges of maintaining older ships with obsolescent equipment. When the older destroyers and AORs are retired from service, because they are no longer sustainable, this will reduce the burden on these budgets. However, those ships will be withdrawn from service prior to delivery of replacement ships, which will extend the Navy’s deficit of naval combatants and key operational capabilities.

2.4.2 Intermediate Outcome

Evaluation Question: To what extent have Canada’s Naval Forces been able to meet expectations when assigned to operational missions?

While the purpose of the immediate outcome was to make ready forces available for maritime operations, once those forces are generated they must be capable of conducting their assigned tasks. Essentially, they must meet the expectations of the DND and the GoC with respect to their assigned roles and missions.

The following indicators were used to make this determination:

• the ability of Naval Forces to arrive in the area of responsibility on time in accordance with an operational directive;
• the ability of Naval Forces to be deployed for the duration of an assigned mission in accordance with operational directives;
• Operational Commanders’ opinion that assigned responsibilities, tasks, and other duties are completed on operations;
• demonstration of flexibility of Naval Forces to be re-tasked when required; and
• demonstration that Canadian Naval Forces are interoperable with allied and coalition forces.

Key Finding 9: Despite challenges with readiness targets, the RCN has consistently met GoC and DND expectations and demands for the conduct of operations.

Through a review of operational reports and directives, the evaluation team noted that ships tasked on operations have consistently arrived in their assigned area of responsibility within the required timeframe, and have been operationally available for the planned duration of their assigned missions in accordance with operational directives.

There have been some instances where significant materiel defects have required ships on station to return to port for repair, or Canadian ships have been re-assigned to cover a task or patrol area for another Task Group vessel from an allied or coalition navy as a result of maintenance issues with that vessel. However, the frequency of this occurrence is low. Based on key stakeholder interviews, this is a normal occurrence with all navies and does not appear to occur more often for the RCN than with our allies.

Based upon key stakeholder opinions (ship CO) questionnaires, allied commander attestations, and interviews with senior operational commanders), the evaluation found that in the majority of instances the RCN has met operational expectations. For example, the CJOC Deputy Commander (Expeditionary) noted that, by and large, Canadian ships and task groups are lauded by our allies. Further, the Royal Australian Navy Commander of CTF 150 (Arabian Gulf) recognized two RCN ships as being the go-to ships to get things done in the coalition Task Force.

There are, however, some concerns. While SR Frigates are routinely assigned to missions such as Operation CARRIBE, based on the CRCN readiness direction that includes restrictions on maintenance and operation of various ship’s sensors, it has been noted in after-action reports that the sensors that were not being maintained or authorized for use diminished the ship’s effectiveness during the operation. Concerning the MCDVs, their effectiveness is very limited on such operations as they do not possess the command, control, computers, communications intelligence, surveillance and reconnaissance systems, speed, endurance, or other operational capabilities, including a helicopter, that are required to fully meet mission requirements.

On a positive note, the evaluation confirmed that Naval Forces have repeatedly demonstrated their responsiveness and flexibility to be re-tasked to different missions when already deployed. Domestic examples include transition from a Fisheries Patrol, in support of the Department of Fisheries and Oceans, to a humanitarian assistance disaster relief mission in support of provincial authorities (Operation LAMA, 2010) or counter-narcotics mission in support of the Royal Canadian Mounted Police (Operation CHABANEL, 2006). International operations examples have included ships that have transitioned from an initial tasking to support a possible non-combatant evacuation operation of Canadian nationals in Libya in 2011, to a complex NATO operation (Operation MOBILE, 2011). In yet another example, an RCN ship was initially deployed to Operation SEXTANT, assigned to the Standing NATO Maritime Group 1 operating in the Mediterranean, and was subsequently re-tasked to escort World Food Program shipping in the piracy danger area off the Horn of Africa under Operation ALTAIR.

Concerning interoperability, in every instance reviewed, training, equipment and experience has enabled the RCN to seamlessly integrate with a variety of allied or coalition forces, allowing them to effectively contribute to each multinational mission. This has been most apparent when units have assisted in the capture of illegal narcotics in the Persian Gulf, where proceeds may support terrorist activities, and in the Caribbean, through Operation CARIBBE, to prevent narcotics from entering into the North American drug market. It is noteworthy that all senior stakeholders interviewed by the evaluation team agreed that Canada’s Naval Forces are highly interoperable with allied and coalition forces, and that the ability to seamlessly integrate Canadian ships and operational staffs into USN naval battle groups is seen as a major strength of the RCN.²⁴ Moreover, many interviewees noted that the Navy’s participation in major joint and combined exercises with allied forces and partner nations such as the RIMPAC²⁵ series of exercises, was one of the principle reasons for the RCN’s interoperability. Furthermore, the RCN’s participation in these multinational exercises has enabled the RCN to establish beneficial working relationships with other non-traditional naval partners around the globe.²⁶

Key Finding 10: CMS preliminary readiness direction to formations led to an unintended reduction in the required materiel state of ships entering the HCM and constrained the Navy’s FG capabilities. The direction was subsequently revised to address these issues.

Initial CMS readiness direction to formations restricting maintenance and operations of some ships' systems created issues. The intent of the direction was to focus effort and limited resources on the HCM and VCI programs, but it had an undesired effect on the materiel state of ships entering HCM program. As ships enter the program, legacy systems are required to be operational in order to meet contractual obligations and avoid large re-activation costs. Additionally, the CMS’ direction impeded some naval operator and maintainer trades’ training that was required by the Navy to achieve sustainable TES in those occupations. The CMS subsequently revised formation readiness direction in April 2011²⁷ to address these issues.

2.5 Demonstration of Efficiency and Economy

The following section examines the extent to which the Naval Forces Program use the most appropriate and efficient means for their activities.

Under the 2009 Evaluation Policy, “efficiency” is defined as “maximizing the outputs produced with a fixed level of inputs” or, in other words, minimizing the inputs used to produce a fixed level of outputs. “Economy” is defined as “minimizing the use of resources to achieve expected outcomes.”²⁸ For the purposes of the Evaluation Policy, these elements of performance are demonstrated under the following circumstances:

• outputs are produced at minimum cost (efficiency); and
• outcomes are produced at minimum cost (economy/cost effectiveness).

Accordingly, the Evaluation of Naval Forces considered “efficiency” in the following areas:

• Personnel,
• Maritime Equipment Maintenance,
• Maritime Training,
• Maritime Infrastructure Maintenance and Base Support, and
• Command and Control.

“Economy” was considered as follows:

• Are resources allocated to the program reasonable, economical, and sustainable?

2.5.1 Demonstration of Efficiency

Are resources allocated and utilized efficiently in the generation and delivery of combat-ready naval forces?

Data available to the evaluation team provided information on the RCN’s financial allocations for generating and delivering combat-ready naval forces and the proportion spent on each program activity. To assess the efficiency of the program, the evaluation focused on expenditure trends and, where possible, benchmarks to similar navies of other countries.

2.5.1.1 Personnel Costs²⁹

Key Finding 11: The significant increase in personnel over the evaluation period was in response to personnel shortages. In terms of overall personnel numbers, the RCN appears to be properly sized.

The following criteria were used to assess personnel costs:

• overall personnel costs; and
• benchmarking to similar allied countries.

Personnel costs represent approximately half of all Naval Forces Program expenditures. The RCN attributes 8,919 military personnel to the Naval Forces Program, supported by approximately 5,000 civilians. As demonstrated in Table 7, expenditures for military personnel attributed to the RCN grew by approximately 18 percent (adjusting for inflation), while the number of civilian FTE declined by approximately 12 percent as a result of Department-wide Strategic Review and Deficit Reduction Plan measures.

Table 7. Naval Forces Personnel Expenditures.

Source: Defence Resource Management Information System

*Military numbers represent those assigned to the RCN and do not include P Res, MILPERSCOM and NDHQ values.

To assess whether the appropriate number of personnel resources was assigned to the Naval Forces Program, the evaluation conducted a benchmark analysis with similar allied navies, including that of Germany, the Netherlands, Spain and Australia.³⁰ Consideration was given to the size, composition, and age of the fleet and operations. Benchmarks utilised data for FY 2012/13 that included

• total personnel per crew requirements (assuming one crew per vessel);
• overall fleet tonnage (vessels >500 tons) per total personnel; and
• non-crew personnel per overall fleet tonnage (vessels >500 tons).

Table 8 illustrates the results of the benchmark analysis. Due to a higher proportion of major vessels (frigates, destroyers and Victoria-class submarines) and fewer coastal vessels, the proportion of personnel that can be attributed to crew demands is higher for Canada than its benchmark nations. This is further compounded by the age of the fleet; for example, the Protecteur-class AOR vessels require a complement of 365 people largely due to its obsolescent steam propulsion system. The proposed AOR replacement will likely have a crew half that size.

With respect to overall personnel numbers, the benchmarks indicate that RCN personnel numbers are reasonable and in line with other similar navies. Despite the higher maintenance demands of an older fleet, the RCN appears to have slightly fewer non-crew personnel available. Overall, the total personnel per fleet tonnage is slightly higher than average—likely a reflection of higher crew demands due to RCN fleet composition. As such, the growth in expenditure on military personnel, despite the stability in fleet size, was not determined to reflect decreasing efficiency; rather, it demonstrates a trend from under-staffing to a more reasonable number of RCN personnel.

Table 8. RCN Personnel Benchmark Comparisons.

Source: IHS Jane’s, 2013³¹

2.5.1.2 Maritime Equipment Maintenance Costs

The following criteria were used for the assessment of maritime equipment and maintenance costs:

• overall cost of equipment maintenance; and
• NP costs.

Key Finding 12: Significant improvements in the overall efficiency of fleet maintenance have occurred over the evaluation period.

Maintenance expenditures include the delivery of materiel and services required to perform preventive maintenance, repairs, and minor upgrades (betterments) to the fleet. These activities are performed by both RCN personnel (military and civilian) and third-party contractors. ADM(Mat), through the Maritime Equipment Program Management (MEPM), is responsible for the materiel supply chain and procurement of both parts and contractor services. Excluding personnel costs, maintenance represented 30 percent of the program expenditures in FY 2011/12. When the personnel associated with maintenance are factored into the grouping, maintenance represents 40 percent of total program expenditures.

As per Table 9, the overall costs associated with the delivery of maintenance were seen to increase by 9 percent between FY 2008/09 and FY 2011/12, which is in line with inflation.³² However, the program was determined to have delivered more value since the number of managers was significantly reduced (12 percent) reduction in civilian and military pay) and increased the expenditures on goods and services (NP) by 24 percent. As a result, approximately $130 million more was spent on materiel and services in FY 2011/12 while the overall budget only increased by $75 million.

Table 9. Breakdown of Maritime Equipment Expenditures.

*Note: The O&M funds may include expenditures from financial coding Line 115, Line 116, Line 119.

Source: PAA

This can be directly translated into increased support to the operation fleet. As observed in Table 10, the FY 2011/12 NP expenditure per ship was $20.1 million, which has also increased 6 percent annually on average.

Table 10. Trend in Cost of Output Equipment Maintenance.

Source: PAA

In past years, maintenance requirements (spare parts, engineering changes, etc.) could not be met due to both a lack funding allocated and an inability to execute the demand due to capacity issues within ADM(Mat) and among the fleet maintenance facilities.

Recently, MEPM’s ability to spend NP funding has significantly increased due to recent strategic initiatives, permitting MEPM staff to more effectively carry out functions related to procurement and maintenance support. As reported by ADM(Mat) and the MEPM organization, these strategic initiatives include a reorganization of MEPM, increased use of In-Service Support Contracts (ISSC), and a review of naval platform maintenance practices and cycles.

MEPM staff reported that MEPM’s bundling of maintenance work into ISSCs³³ to manage third-line maintenance work for MCDV and Victoria-class submarines will produce efficiencies by freeing up staff from transactional activities to work on higher-level strategic functions. The move to ISSCs is in keeping with the Department's policy as well as recommendations by the Auditor General.³⁴

It was reported that no metrics currently exist to monitor efficiencies resulting from ISSCs; however, it has been suggested that Victoria-class ISSC has allowed for significant capacity to deliver and support submarine capability. Some interviewees felt that ISSCs may initially be more expensive and efficiencies will only be realized in subsequent years. It is also reported that the Department will continue to move non-ISSC maintenance contracts to an ISSC format where reasonable to do so.

Submarine Maintenance and Operational Cycle Alignment and Rationalization (SUBMOAR) ³⁵ and Halifax Maintenance and Operational Cycle Alignment and Rationalization³⁶ projects were also cited as measures by which the RCN has sought to improve efficiency. A review of MEPM documentation found that one outcome of Halifax Maintenance and Operational Cycle Alignment and Rationalization has been that Halifax-class operational cycles have been extended from 48 to 60 months and that routine maintenance has been reduced by more than 50 percent.³⁷ SUBMOAR will also extend the operational cycle of the Victoria class from 3.6 years to 6 years between Extended Docking Work Periods (EDWP).

The implementation of SUBMOAR will also substantially reduce the preventive maintenance work loads for Fleet Maintenance Facilities and ships’ staff, and will lower spares usage. There will also be fewer defects from servicing equipment that does not require repairs, as well as other savings.³⁸ This should result in further overall cost savings.

Documentation reviews and staff interviews have further suggested that the RCN has responded to resource constraints by prioritizing support to HR ships, HCM and VCI programs, and restricting materiel support to other ships’ systems.

Key Finding 13: Despite the efficiency improvements, there remains a funding gap in maintenance within the RCN.

Despite these initiatives, however, there are still significant concerns going forward. As discussed in section 2.4.1, maintenance issues have become a leading cause of challenges to meeting readiness targets. While the capacity to execute maintenance demands appears to be aligned to the needs, there is still a shortfall in funding allocations. The gap between executable capacity and allocation has consistently been greater than $100 million annually, and as the fleet ages the demand is growing. An overall comparison of maintenance expenditures per vessel with similar allied navies demonstrates that Canada spends significantly less per ship, despite the greater age of the fleet. As per Table 11, Canadian maintenance expenditures per vessel and per fleet tonnage are significantly lower than those of the Germany or Australia, and generally in line with the Netherlands —which has a much more modern fleet. Although these are very high-level indicators and should not be used as targets, in principle they support ADM(Mat) and RCN estimates of a funding gap.

Table 11. Comparison of Maintenance Expenditures.

Source: IHS Jane’s (June 2013)

One means to address the funding gap may be to seek further efficiencies in the program. In 2013, DND announced the Defence Renewal initiative which seeks to reallocate resources to high-priority areas, and to find those resources through numerous efficiency initiatives. Included in the plan are initiatives to improve inventory management, warehousing and distribution, maintenance execution and departmental procurement. The initiatives will seek to increase productivity of maintenance personnel and reduce costs of goods and services through the use of improved procurement strategies. Although not broken down by command, the savings targets for these initiatives across the Department are anticipated to be $370 million annually and have been given a four-year timeline to be implemented.

2.5.1.3 Maritime Infrastructure Maintenance and Base Support Costs

The following criteria were used for the assessment of Maritime Infrastructure and Base Support activity costs:

• total cost of providing infrastructure and base support; and
• cost per personnel.

Key Finding 14: Maritime Infrastructure Maintenance and Base Support costs have declined due to a reduction in personnel. However, these costs represent a larger proportion of the RCN’s O&M budget, resulting in fewer resources being available for other O&M expenditures such as training.

Between FY 2008/09 and 2011/12, DND/CAF spending on Maritime Infrastructure Maintenance and Base Support has averaged $433.6 million per year. Over this period, the overall cost has decreased by $15 million, at an average annual rate of 0.5 percent.

As reflected in the Equipment Maintenance area, infrastructure maintenance and base support personnel costs have decreased, which is attributable to a reduction in personnel assigned to these activities. However, as shown in Table 12, the RCN O&M costs have increased by 5.3 percent on average annually,³⁹ a majority of which represents increases in Infrastructure Maintenance and Base Support spending.

Table 12. Cost of Maritime Infrastructure Maintenance and Base Control.

*Note: The O&M funds may include expenditures from financial coding Line 115, Line 116, Line 119.

IM&R is a significant challenge for the RCN and Maritime Readiness. Between FY 2008/09 and FY 2011/12, the RCN invested, on average, 52.5 percent of its O&M funds on IM&R. This represents an average of 0.8 percent of the RCN’s real property replacement cost (RPRC) over the period (ranging from 0.5 percent to 1.1 percent).

Based on departmental direction, the RCN was instructed to increase IM&R funding to 1.4 percent of RPRC by FY 2013/14, which would require an additional investment of $30 million for FY 2013/14 (based on RPRC of $3.7 billion). This poses a significant challenge for the RCN as increased spending on IM&R will place increasing pressure on the Navy’s limited O&M budget. Furthermore, the high valuation of the RCN waterfront property in an industrialized urban area drives more rapid inflation of the RCN’s RPRC and results in higher IM&R funding requirements. This could potentially be diverting resources away from other areas of greater significance to the Navy, such as O&M funding available for “sea days.” It should be noted, however, that while the Navy has suggested that the IM&R target of 1.4 percent RPRC may be too high, it made significant efforts to achieve its lesser IM&R targets in FYs 2010/11, 2011/12 and 2012/13 and the investments in fact exceeded the IM&R targets.

2.5.1.4 Maritime Training Costs

The following criteria were used for the assessment of maritime training costs and its outputs:

• cost of delivering maritime training;
• cost per trainee; and
• cost per course.

Key Finding 15: The Navy is delivering maritime training with increased efficiency. The overall cost of maritime training has decreased from FY 2008/09 to FY 2012/13, whereas the number of people trained and the number of courses offered has increased.

As shown in Table 13, and Figure 4, between FY 2008/09 and 2011/12, DND/CAF spending on Maritime Training has averaged $229 million per year. Annual maritime training costs have decreased over the evaluation period by $57 million (an average annual decrease of 8 percent). The largest decrease was observed between FY 2010/11 and FY 2011/12 at almost 16 percent.

Simultaneously, the number of persons trained and the number of courses offered have increased over the evaluation period by an annual average of 4.2 percent and 5.2 percent respectively.

Table 13. Cost of Maritime Training.

Source: PAA for training costs, Director Naval Training and Education for persons trained and number of courses (excludes NOTC as costing for this course is attributed to PAA 1.2.2.1 and 1.2.2.2 – Initial Qualifications and Occupations Training ), PMF for percentage of personnel trained at optimal course capacity.

Figure 4. Maritime Training—Cost per Person and Cost per Course. This graph shows the relation between cost per person and cost per course. The data is summarized in Table 14.

Source: PAA for training costs, Director Naval Training and Education for persons trained and number of courses (excludes NOTC as costing for this course is attributed to PAA 1.2.2.1 and 1.2.2.2 – Initial Qualifications and Occupations Training), PMF for percentage of personnel trained at optimal course capacity.

Table 14. Cost per Trainee and Cost per Course.

As shown in Table 15, the overall cost of providing maritime training has been decreasing, and the average number of persons trained and courses offered have increased, which demonstrates improved efficiency. The average annual cost per trainee was $17,800 and the average cost per course was $267,600. The cost per person trained has decreased at an average annual rate of 11.2 percent. Similarly, the cost per course has continued to decrease, on average by 12.4 percent annually. When compared to FY 2008/09 levels, by FY 2011/12, both the cost per person trained and the cost per course had decreased by 31 percent and 34 percent, respectively.

Table 15. Training Costs per Personnel in Years.

Source: Expenses reported according to PAA structure for costs; Director Naval Training and Education for persons trained and number of courses (excludes NOTC as costing for this attributed to PAA 1.2.2.1 and 1.2.2.2 – Initial Qualifications and Occupations Training).

Key Finding 16: On average, the percentage of personnel trained at optimal course capacity was 54 percent. The RCN PMF sets a target for the percentage of personnel trained at optimal course capacity at 90 percent.⁴⁰

However, one area for improvement is with respect to course loading. On average, the percentage of personnel trained at optimum course capacity was 54 percent .There has been an upward trend over the evaluation period (44 percent in 09/10, 55 percent in FY 2010/11 and 64 percent in FY 2011/12); however, this suggests that more work is required to further optimize training loads. Furthermore, the RCN’s current PMF sets a target for the percentage of personnel trained at optimal course capacity at 90 percent. Optimizing course loading and reducing frequency of course offerings may help further reduce training costs, including the cost of operating trainers, further increasing efficiency.

Document review and interviews have indicated that the RCN focused on identifying innovative ways to be more efficient in what they do, and to privilege readiness and operational output. For example, the harnessing of technology in the creation of virtual classrooms will allow linking west and east coast naval training, creating synergies that should reduce annual training costs.⁴¹

In addition, it was reported by many interviewees that thanks to the RCN’s transformation initiatives such as the Naval Strategic Analysis Team’s⁴² work— particularly the realignment of naval training organizations under a single authority—the Naval Training System has gone a long way to clarifying roles and responsibilities within the RCN, reducing duplication between the two coasts, and streamlining processes. It was noted by some interviewees that as these transformations are still under way, there are still areas which need to be delved into further.

Other significant transformation initiatives⁴³ of the RCN have been in the areas of establishment of Personnel Coordination Centers and coordination of fleet scheduling, a “Single Fleet Scheme of Manoeuver.”⁴⁴ Documentation reviews and interviews with RCN staff have also highlighted that the Navy has increased the use of more economical purpose-built e-training to support Maritime Surface and Sub-Surface officer training. Furthermore, the Navy aims to increase availability of on-line military occupation and operational training, and use of operational simulation/trainers to complement and augment more expensive ships’ training currently being conducted.

2.5.1.5 Maritime Command and Control Costs

Key Finding 17: As a percentage of total readiness, Maritime Command and Control represents 7 percent of total costs on average. Over the evaluation period, the costs attributed to the Maritime Command and Control have increased significantly (40 percent) due to a large influx of personnel as a result of reductions or elimination of other coastal formation establishments and the transition of the Navy’s coastal maritime Headquarters to Regional Joint Headquarters.

The following criteria were used for the assessment of maritime Command and Control costs:

• overall cost of Command and Control;
• percentage change in dollar expenditures; and
• percentage share of cost of maritime readiness.

As shown in Table 16, between FY 2008/09 and 2011/12, spending on RCN command and control has averaged $144.2 million per year. Over this period, the cost of command and control increased overall by $107 million, an average annual increase of 33 percent. The increase in the total cost of command and control was principally driven by increases in personnel costs. Reg F Pay (which represented on average almost 70 percent of the total command and control costs) increased overall by $73 million, an average annual increase of 42 percent. The most significant increases were observed in FY 2009/10 and FY 2011/12, where Reg F Pay increased by 67 percent and 31 percent, respectively.

Table 16. Cost of Output—Command and Control.

Source: Expenses reported according to PAA structure

Note: O&M and Capital allocations removed and applied towards Operations

On average, command and control represented only 6.7 percent of the total cost of Maritime Readiness, and its share of the total Maritime Readiness cost has continued to increase over the period of the evaluation, as shown at Table 17. This is also reflected in the fluctuations in dollar expenditures. A notable increase is observed in FY 2009/10; which as indicated above, was largely driven by increases in personnel costs.

Table 17. Share of Total Cost of Maritime Readiness—Command and Control.

Source: Expenses reported according to PAA structure

To support the PAA financial information, the RCN Reg F personnel numbers associated with the command and control establishments were further analyzed. The data indicated that the number of military personnel (Reg Force and PRes) attributed to command and control has increased considerably, largely as a result of significant increases in Reg F personnel assigned to MARPAC HQ and MARLANT HQ.⁴⁵ The person years for these increases were drawn from other MARPAC and MARLANT units to meet post-2006 CAF C2 changes and emerging Navy C2 and FG issues as the RCN transitions to the future fleet. The main C2 increases were as follows:

• MARPAC HQ
  • FY 2008/09, creation of an integrated Regional Joint Operations Centre organization to address MARPAC’s new “dual-hatted” role following MARPAC HQ’s additional tasking in February 2006 as the Regional Joint Task Force (Pacific) HQ.
  • FY 2011/12, Director Canadian Submarine Force/Commander Submarine Force organization was established in MARPAC HQ as the pan-Navy submarine authority and to ensure smooth introduction of the Victoria-class submarines. Submarine technical support was also augmented.
  • FY 2011/12, MARPAC N36 (Operations and Intelligence) staff was further augmented.
• MARLANT HQ
  • FY 2009/10, MARLANT HQ N6 (Formation Communications and Information Systems) organization, which had not previously existed, was created with person years acquired from closure of the Canadian Forces Station Mill Cove radio station, when its control was remoted to MARLANT HQ, as well as person years from closure of the Canadian Forces Base Halifax information services and message centre.
  • FY 2011/12, Submarine Support Cell was established in the HQ to address MARLANT submarine support requirements.
• MARLANT and MARPAC HQs
  • FY 2011/12, both Formation HQs established Personnel Coordination Centres to “manage the limited skill-sets available [in certain sea trades] against a prioritized list of demands,”⁴⁶ as well as personnel training requirements.

The increases in formation HQ staffs, when considered against other current and future manning requirements, suggests that there may be a need for the Navy to rationalize its Headquarters’ manning and consider measures to reduce its command and control costs.

2.5.2 Economy of Naval Forces Program

Is the level of funding for the program appropriate, sustainable, and affordable?

To assess the ongoing economy of the Naval Forces Program, the evaluation examined trends in expenditures over the period of the evaluation, and analyzed the trends based on information provided by key stakeholder interviews and external benchmarks where possible.

Key Finding 18: The program expenditures for Maritime Readiness have increased during the evaluation period. However, they have not exceeded reasonable amounts for a fleet of the size and composition of the RCN.

Between FY 2008/09 and 2011/12, DND/CAF spending on generating and delivering combat-ready naval forces (PAA 2.1 Maritime Readiness) has averaged $2.2 billion per year. Average annual increase in this activity spending has been 6 percent (more than the annual DND budget increase of approximately 2.1 percent per year). In comparison, the average annual percentage change has decreased for both Land (-2.5 percent) and Aerospace Readiness (-3.2 percent), respectively (Table 18).

Table 18. Annual Change (%) in Total Readiness Cost.

Source: PAA

Table 19 demonstrates the changes in overall costs for the program over the evaluation period, as well as corresponding sea days and commissioned ships.

Table 19. Maritime Readiness Program Costs.

Source: PAA (for financials) and Director Naval Strategic Operations (for sea days). CRS estimated force employment (the operation portion) of sea days through review of data logs.

Overall, the largest share of expenditures were on personnel (52 percent), followed by NP (31 percent) and O&M (13 percent) and Infrastructure (7 percent). Resources expended were about equally distributed between ADM(Mat), which is responsible for NP and 3rd line maintenance/refit activities, CMP, which is responsible for Reg F Pay, and the CRCN who is responsible for navy readiness and sustainment activities.

Naval Forces Expenditures FY 2011/12 (Personnel Cost Separated)

Figure 5. Naval Forces Personnel Costs for FY 2011/12. This figure demonstrates RCN personnel costs associated with different program activities. The data shown in Figure 5 is summarized in Table 20.

Source: PAA

Table 20. Naval Forces Personnel Costs for FY 2011/12.

Preliminary total program costs for FY 2012/13 are $2.38 billion, an increase of 3.6 percent ($83 million) from FY 2011/12. This growth is less than the average increase in costs over the period analyzed (5.9 percent), indicating that the rate of increase in expenditures on generating and delivering Maritime Readiness is starting to slow.⁴⁷

To assess the overall economy of these expenditures, the costs of generating and delivering Canadian Naval Forces were compared to those of other allied/similar countries’ navies, taking into consideration two factors that could significantly affect differences in outcomes (i.e., naval readiness): the number of in-service commissioned ships and the number of Reg F personnel in the Navy.

As Table 21 illustrates, the costs were also calculated using Navy O&M expenditures, and total expenditures supporting navy readiness. While these benchmarks are very general and high-level, they were found to give a good overall sense of value delivered as well as support to observations made by program managers and other key stakeholders. The evaluation team found the calculations with the O&M expenditures to be more relevant, as the support costs categories (i.e., military personnel, procurement, research and development, training and equipment O&M) likely differ for each country.

Table 21. Canada’s Naval Expenditures versus Allied Nations.

Source: IHS Jane’s (February 2013) for expenditures and IHS Jane’s (June 2013) for numbers of Reg F and in-service commissioned ships (greater than 500 tons).

Note: All values in constant 2013 United States dollars with the exception of Australia and New Zealand which are in constant 2012 United States dollars. Total expenditures supporting the Navy include cost of military personnel, procurement, research and development, training and equipment, O&M, plus all other attributable indirect costs.

The data demonstrates that as of 2012, Canada falls on the lower end of the range of other countries on funds spent per ship or per Reg F member, based upon both Navy O&M and overall expenditures. Benchmark countries that spent more than Canada are Australia and United Kingdom, while the Netherlands, which has a similar budget and fleet composition (although newer), spent relatively the same amounts. It should be noted, however, that Canada spends a significant amount of its sea days and operations far from its home ports, which increases operating costs and maintenance. Part of the higher costs for the United Kingdom can be attributed to differences in fleet composition, which includes large amphibious assault vessels and nuclear submarines.

While the Naval Forces Program expenditures have grown during the evaluation period, they have not exceeded reasonable amounts for a fleet of the size and composition of the RCN. Going forward, despite various initiatives under way, the program will continue to face resource allocation pressures.

Key Finding 19: Overall costing data for RCN should include several L1 accounts, for example the PAA 2.1 includes CMP, RCN and ADM(Mat). It was noted that the RCN did not assess all its program expenditures (nor use performance indicators) that include all costs such as labour and material. Therefore, a true indication of costs and performance is not available, which may impact the RCN’s ability to assess overall performance.

One key area of concern appears to be the current program structure, which divides resource allocations among three organizations: CMP, CMS and ADM(Mat). This model restricts the ability to prioritize funding over key areas and, as a result, can severely impact the overall effectiveness of the program. As overall costs of generating and delivering Naval Forces by the RCN are spread among these “L1” (separate organizational) accounts, the program does not collectively assess its program expenditures in a manner that includes labour, materiel and overhead costs. Therefore, a true indication of costs and performance is not provided, which may impact the ability to assess overall performance and prioritize decisions.

For example, although the combined budget is $2.3 billion, the stove-piped program structure is extremely hard pressed to find less than $20 million annual funding to procure critical components to keep the destroyers and AOR vessels in service beyond FY 2014/15. Similarly, the demand to increase infrastructure and base support by $30 million annually severely impacts the RCN’s ability to fund readiness training. As a result, a support infrastructure may exist for a non-functional fleet.

Another concern related to the program structure is with performance measurement. The Navy collects a vast amount of program data using such systems as Application for the Reporting and Measurement of Operational Readiness and the Defence Resource Management Information System. These are used to capture and monitor the RCN performance by some organizations, such as NSHQ and DGMEPM. However, measurements on financial data, in areas such as unit costs (cost per ship, cost per training day) are not currently being tracked and there are no established benchmarks to determine whether the costs reported are reasonable, fall within projected targets, or whether differences in the costs of delivery fall within acceptable limits.

The 2011 Report of the Office of the Auditor General on Maintaining and Repairing Military Equipment had recommended development and implementation of performance measures on maintenance and repair activities, their effectiveness, and their efficiency using common performance measures where possible. DGMEPM has followed through on these recommendations and has started the development process for improve performance measures. However, this is still in its early stages.

Annex A—Management Action Plan

The RCN is extremely pleased to have worked closely with CRS on the development of this Evaluation of Naval Forces Report, as a component of the DND/CAF Five-Year Evaluation Plan (2012-2017). We welcome the report and its timeliness as we embark on a series of internal initiatives that will improve the way we manage the “business of our business.” While there were some predictable findings that are already being addressed, the report also highlighted some additional observations which have garnered our focused attention. We were particularly pleased to note the enterprise perspective that CRS has articulated, which anchors the point that L1s cannot be looked at individually when discussing management of an L1 program. For the RCN, the importance of understanding these interdependencies, in particular with CMP, ADM(Mat) and RCAF, is critical to the successful delivery of the Naval Forces Program.

The RCN is in the midst of the most comprehensive period of fleet renewal ever in peacetime. Limited resources, financial and human, and competing priorities continue to test our ability to most effectively and efficiently deliver the Naval Forces Program. In acting decisively, the RCN has developed a comprehensive plan to address these challenges—one that is fully aligned with the intent, principles and initiatives associated with Defence Renewal and other key departmental/corporate initiatives including, but not limited to, the new PAA and the Force Posture and Readiness Directive. Launched in December 2013, the RCN Executive Plan details Navy priorities over a four- year planning period and seeks to do things smarter in order to privilege maritime operational capability today and into the future. We are evolving our strategic, operational and business planning approach to the Force Management, Force Development, FG, Force Support and Force Employment of Naval Forces to incorporate a comprehensive multi-year, integrated “One Navy” perspective.

Management Action

There are many aspects of the Naval readiness program not within the control of the RCN, directly involving Level 0s and various L1s. We are working closely with external staff to improve our collective understanding of these interdependencies and their implications. Within our own lines of responsibility, and in support of evolving our approach to managing our business, the RCN readiness policy and requirements key documents are undergoing review with an expected re-promulgation in the spring of 2014. Each document will include updates to keep pace with the evolving nature of naval readiness, warfare and platforms, as well as reflect the latest Force Posture and Readiness policy for the CAF. In parallel with this comprehensive review, the RCN continues to manage and assess the short- to medium-term readiness impacts of HCM on fleet availability and experience levels of sailors, as well as implications for the transition to the future fleet (Arctic Offshore Patrol Ship, Joint Support Ship and Canadian Surface Combatant) and the divestment plan of the Iroquois and Protecteur classes.

OPI: RCN

Target Date: May 2014

Management Action

The overall departmental level of IM&R investment has been well below established targets as indicated in Chapter 5 of the Office of the Auditor General Fall 2012 Report. IM&R funding (L119) has been fenced to restrict funding from being removed for other purposes. However, funds can—and have been—added from RCN baseline levels on an as-available basis to help attain the departmental target levels. Unfortunately, the resource reductions associated with SR and the Deficit Reduction Action Plan have severely limited the RCN's ability to add to the L119 envelope, without an unacceptable impact to operations and training. Accordingly, the RCN will continue to risk-manage O&M budget apportionments between operations/training and IM&R-related life-safety, health and regulatory compliance requirements.

RCN staff is participating in several initiatives (Defence Renewal Team initiatives 4.2 and 4.3) to reduce IM&R requirements including divestments of real property, consolidation of facilities and real property service delivery optimization based on the Departmental Real Property Centralization currently under way. Due to the breadth of the RCN Real Property Portfolio (approx $3.7 billion of the $25 billion DND Portfolio) significant and meaningful changes must be measured over an extended period to reduce annual fluctuation errors.

OPI: RCN, ADM(IE)

Target Date: January 2019

Management Action

Enable synchronized aggregation of data for planning and forecasting of individual training and collective training through the following initiatives:

3.a) Conduct research on industry and defence best practice to determine which training management solutions and training technologies are best suited to optimize Navy training delivery through the Navy Training System Transformation Project.

Target Date: July 2014

OPI: RCN

3.b) Acquire and enable a corporate training resource management system, including a scheduling system, compliant with the Defence Learning Network (DLN) and Naval Information Management System, to control Regular and Reserve personnel training progression at the five Naval Training System training points:

1. recruit—Strategic Intake Plan,
2. occupational training,
3. collective training, and
4. refresher and readiness training.

Target Date: December 2014

OPI: RCN

3.c) Enable training technology like the Universal Classroom, and DLN to deliver this training on both coasts in the most efficient manner using synchronous (in class) or asynchronous (at time of need) delivery in home port locations;

Target Date: December 2018

OPI: RCN

3.d) Update training management policy including Gold, Silver and Blue training demand books to reflect how training management systems and training technology and should be used to optimize training throughput efficiency.

Target Date: December 2014

OPI: RCN

Aggregate Target Date: 2019

To note, all initiatives related to optimizing RCN training will be aligned with Defence Renewal – DRT initiative 5.1: Modernize CAF Individual Training and Education.

Management Action

The RCN continues to examine the root causes of the identified personnel and cost increases to the Maritime Command and Control function. While part of the increase can be explained by organizational misattribution, analysis to date indicates that the majority is related to two specific instances: 1) 2010/11 – amalgamation of the Formation N6 organization in MARLANT HQ, parts of which had previously been attributed to the base organization and Mill Cove Detachment and 2) 2012/2013 – the stand-up of the Director Canadian Submarine Force in MARPAC HQ and the submarine support staffs located on both coasts, which have been attributed to the Formation HQs.

As part of the RCN Executive Plan, a series of strategic objectives have been established to seek efficiencies in Maritime Command and Control in the areas of Force Management and Force Support at the HQ and Base level, in order to support FG and Force Employment activities as a priority. The review examines issues of organizational construct, attribution, taxonomy, and activities alignment. In addition, the RCN is in the process of developing an integrated regular, reserve and civilian HR model to determine the most effective and efficient personnel balance to deliver the Naval Forces Program. Sensitive to the current and projected fiscal environment, it aims to identify the right amount people, with the right skill sets, to do the right jobs – in the right place. The following key initiatives support the RCN plan to identify in this regard:

4.a) Map and prioritize all RCN positions against the 5F model. Ensure associated costs are properly attributed to the new PAA structure.

Target Date: June 2014

OPI: RCN

4.b) Conduct a functional review to ensure that positions and roles are properly aligned. Clearly identify roles for strategic, operational and tactical levels and develop supporting terms of reference. Ensure associated costs are properly attributed to the new PAA structure.

Target Date: June 2015

OPI: RCN

4.c) Adjust RCN Organization and Establishment (O&E) based on analysis. Ensure associated costs are properly attributed to new PAA structure.

OPI: RCN

Target Date: October 2015

Aggregate Target Date: December 2015

Annex B—Evaluation Methodology and Limitations

1.0 Methodology

1.1 Overview of Data Collection Methods

The evaluation scope and methodology were established in an evaluation work plan developed during the planning phase (November 2012 to January 2013).

A program logic model (Annex C) and an Evaluation Matrix (Annex D) were developed based on information gathered through initial, select stakeholder meetings. An advisory group of key stakeholders was formed to review and provide feedback on various evaluation planning documents. The committee was initially requested to provide feedback on the draft logic model and evaluation matrix.

The evaluation included the use of multiple lines of evidence and complementary research methods as a means to help ensure the reliability of information and data collected. The following methods were used to gather qualitative and quantitative data:

• interviews,
• document review,
• analysis of administrative and financial data,
• benchmarking, and
• questionnaire.

Each of these methods is described in more detail in the following sections.

1.1.1 Key Informant Interviews

To help scope the evaluation, initial consultations were conducted with CRS auditors and evaluators to discuss past audits and evaluations, and with financial analysts from Assistant Deputy Minister (Finance and Corporate Services (ADM(Fin CS)), Chief of Programme, RCN and CRS, to clarify attributions to the PAA elements.

Stakeholder interviews were an important source of information for gathering in depth, qualitative data. Preliminary consultations were conducted with program OPIs, and evaluation stakeholders from the Director Maritime Infrastructure, the Naval Staff Headquarters, including the Director Naval Personnel and the Director Civilian Personnel. A total of 41 interviews were conducted.

Further interviews were conducted with the RCN senior leadership and NSHQ staff, ADM(Mat)/DGMEPM, RCN force generators (East/West Coast staffs: MARLANT, MARPAC) and force employers (CJOC staff).

Almost all interviews were conducted in person. Most interviews lasted 45 to 60 minutes. Separate interview guides were developed for each of the interview groups and the interview questions were aligned with the evaluation questions identified in the evaluation matrix. While the interviewers took separate notes, all interviews were also recorded for verification and then summarized by evaluation question and indicator.

1.1.2 Document Review

A preliminary document review was conducted as part of the planning phase of the evaluation to help gain an understanding of how the RCN-generated Naval Forces. A more comprehensive document review was later conducted as part of the conduct phase of the evaluation with focus on the relevance and performance of the program, including benchmarking with similar programs delivered in comparable ally militaries. The following types of documentation and literature were reviewed during the evaluation:

Program Documents. Operational reports, capability-based planning, RCN strategic objectives, CFCD 129 Readiness and Sustainment, Leadmark: The Navy’s Strategy for 2020.

Accountability Documents. National Defence Act, Report on Plans and Priorities, CFDS, Speech from the Throne 2008, etc.

The document review was conducted using a customized spreadsheet that was organized according to evaluation questions and indicators.

1.1.3 Analysis of Administrative and Financial Data

The main source of program administrative data was from operational records, post-operational records, minutes of Naval Board Meetings, Maritime Readiness Quarterly Reports (2007/08-2011/12) and MEPM program records.

Financial data was analyzed based on attributed allocations to PAA 2.1 (Maritime Readiness). It was limited to a four-year period (2008/09-2011/12).

1.1.4 Benchmarking

A comparative analysis of costs with similar, allied countries’ navies was performed. The source for the data was IHS Jane’s (2013). This analysis assisted in determining whether or not resources used were reasonable in the achievement of outcomes.

1.1.5 Questionnaire

Electronic questionnaires were administered to address the effectiveness issue (i.e., whether the RCN has been able to meet expectations when assigned to operations and if sufficient readiness levels were achieved). The questionnaire population was relatively small (number = 106), since only the COs of ships, who were deployed in operations within the last 10 years were targeted. The questionnaire was administered in both official languages.

The questionnaire contained mostly close-ended, with a small number of open-ended questions. The response rate to the questionnaire was high (just over 60 percent). The questionnaire was completed within a three-week period.

The questionnaire findings supported the results of other lines of evidence (i.e., stakeholder interviews).

2.0 Limitations

Table B-1 shows the limitations related to this evaluation and the mitigation strategies applied to them.

Table B-1. Evaluation Limitations and Mitigation Strategies.

Annex C—Logic Model

Figure C-1. Naval Forces Logic Model

Annex D—Evaluation Matrix

Table D-1. Evaluation Matrix—Relevance.

Table D-2. Evaluation Matrix—Performance (Effectiveness).

Table D-3. Evaluation Matrix—Performance (Efficiency and Economy).

___________________________________________________________________________________________________________________________

Footnote 1 The repair/maintenance state of the ship’s hull and equipment is referred to as the “material state of the ship.”

Footnote 2 SO 2.1 PMF (2010) sets performance target of 90 percent for Maritime Training indicator “% personnel trained at optimal course capacity.”

Footnote 3 The efficiency and economy of the program (Section 2.5) is assessed over FY 2008/09 to FY 2011/12.

Footnote 4 As explained in more detail in section 2.4, during recent years, the RCN has conducted numerous successful counter-narcotic operations in domestic/continental context and successful counter-terrorist and counter-piracy activities in the international context.

Footnote 5 Source: http://www.navy.forces.gc.ca/cms/12/12_eng.asp (consulted on December 5, 2012).

Footnote 6 The Navy’s governance structure is comprised of three tiers: (1) the Naval Board, (2) the Maritime Strategic Planning Board, (3) the Maritime Capability Development Board and Fleet Sustainment Program Board, Maritime Command, http://marcom-comar.mil.ca/exco-reex/default-eng.asp.

Footnote 7 The primary operational task group is a single Composite Contingency Task Group, derived from high readiness (HR) ships and submarines, able to deploy within prescribed notice to move (NTM) anywhere in the world in support of Contingency Operations. The components of the task group include an HR destroyer, replenishment ship and up to three combatants, either frigates and/or a submarine. Other readiness elements include:

• Naval Task Group: as a follow-on separately employable entity to the Contingency Task Group and is comprised of one HR destroyer, one HR replenishment ship and up to three HR frigates and/or submarines and a Helicopter Air Detachment onboard.
• Single Ship International Deployment: a single HR combatant usually conducted by a frigate, but on occasion a destroyer, submarine or a replenishment ship may conduct this mission.
• Domestic Naval Readiness: includes remaining frigates, destroyer(s), and submarines and 12 mine countermeasure vessels as well as port security, guidance to civilian shipping and two Fleet Diving Units.

Footnote 8 CFCD 129, Readiness and Sustainment, 2009.

Footnote 9 CAF elements consist of the Royal Canadian Air Force, RCN and the Army.

Footnote 10 CFCD 129, Readiness and Sustainment, 2009.

Footnote 11 http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15681&section=text.

Footnote 12 Report on Plans and Priorities, DND, 2009/10.

Footnote 13 The 2008 CFDS identifies security threats as globalization, ethnic and border conflicts, fragile states, resurgent nationalism and global criminal networks that continue to threaten international stability. Other challenges to domestic security include possible terrorist attacks, human and drug trafficking and protecting arctic sovereignty and security.

Footnote 14 The role of the Canadian Government in the Oceans Sector, www.dfo-mpo.gc.ca/oceans/publications/cg-gc/pdf/cg-gc-eng.pdf.

Footnote 15 1867 Constitution Act, section 91.

Footnote 16 The federal Whole-of-Government Framework maps the financial and non-financial contributions of federal organizations to a set of high-level outcome areas defined for the government as a whole. http://www.tbs-sct.gc.ca/ppg-cpr/frame-cadre-eng.aspx.

Footnote 17 Departmental Link to Government of Canada Outcome Areas, RPP 2011/12, http://www.tbs-sct.gc.ca/rpp/2011-2012/inst/dnd/dnd02-eng.asp.

Footnote 18 CFDS six core missions are: (1) conduct daily domestic and continental operations, including in the Arctic and through NORAD; (2) support a major event in Canada, such as the 2010 Olympics; (3) respond to a major terrorist attack; (4) support civilian authorities during a crisis in Canada such as a natural disaster; (5) lead and/or conduct a major international operation for an extended period; and (6) deploy forces in response to crisis elsewhere in the world for shorter periods, DND, http://www.dfo-mpo.gc.ca/ae-ve/evaluations/12-13/Evaluation_SCH_Final_Rpt_Mar13-eng.html#ch4.1, consulted July 26, 2013.

Footnote 19 Maritime maintenance refers to work conducted by the RCN and contractors. NP refers to the acquisition of materiel and services to support the maintenance and betterments of the fleet. It does not include major capital upgrades or platform replacements.

Footnote 20 The RNLN has commissioned three Holland-class Offshore Patrol Vessels, included here as frigates given their size, sensors, weapons, and endurance.

Footnote 21 3371-3000-1(CMS/RDIMS #195998), June 4, 2010, REVISED READINESS DIRECTION TO FORMATIONS.

Footnote 22 Ibid.

Footnote 23 3371-3000-1 (CMS/RDIMS #240652), June 13, 2012, READINESS DIRECTION TO FORMATIONS.

Footnote 24 RCN units integrating to a USN Expeditionary Strike Group is categorized as a routine mission for a RCN HR unit in accordance with CFCD 129 article 405.2.B(2).

Footnote 25 RIMPAC 2012 included twenty-two nations, 40 surface ships, 6 submarines, more than 200 aircraft and 25,000 personnel. Participating units were from Australia, Canada, Chile, France, Japan, Mexico, New Zealand, Republic of Korea, Russia, Singapore, and the United States and military personnel from Colombia, India, Indonesia, Malaysia, Netherlands, Norway, Peru, Republic of Philippines, Thailand, Tonga and the United Kingdom will also participate. Rear-Admiral Ron Lloyd, now the Deputy Commander RCN, was the Deputy Commander Combined Task Force during the exercise. http://www.cpf.navy.mil/rimpac/2012/about/ consulted September 25, 2013.

Footnote 26 As reported in the RCN Business Plan (3371-1948-1 (DNBP / RDIMS # 229404) 13 January 2012 RCN BP 2012-2013) and REGULUS training records, to maintain at-sea experience and progress training and qualifications during the HCM program, RCN personnel have been deployed under the REGULUS program to non-traditional naval partners including the Japanese Maritime Self-Defence and the navies of India, Ireland, Chile, and Singapore.

Footnote 27 3371-3000-1(CMS/RDIMS #193344) dated April 23, 2011, READINESS DIRECTION TO FORMATIONS.

Footnote 28 Treasury Board of Canada Secretariat’s Policy on Evaluation, April 1, 2009.

Footnote 29 Personnel costs are first analyzed as overall personnel costs, followed by personnel costs attributed to each program activity. As per the program logic model (Annex C), the program activities are as follows: Maritime Equipment Maintenance, Maritime Infrastructure Maintenance and Base Support, Maritime Training, Maritime Command and Control, and Operations.

Footnote 30 IHS Jane’s, June 2013.

Footnote 31 For this benchmarking, only the countries with overall fleet tonnage (vessels >500 tons) were selected.

Footnote 32 DND Inflation based on DND Economic Model.

Footnote 33 The ISSC framework intends to reduce the Department’s maintenance contract management activities by bundling hundreds of short-term maintenance contracts into a few longer-term contracts that are awarded to industry.

Footnote 34 Report of the Auditor General of Canada to the House of Commons, Chapter 5 Maintaining and Repairing Military Equipment – National Defence, Fall 2011.

Footnote 35 A review of submarine maintenance practices conducted by DGMEPM.

Footnote 36 MEPM study to review and rationalize Victoria- and Halifax-class maintenance requirements and operational cycles.

Footnote 37 Halifax-class Maintenance Way Ahead Presentation, October 2012.

Footnote 38 12th Shipbuilding Quadrilateral Meeting Presentation: Modifying a Submarine Maintenance Profile to Meet Operational Requirements, April 2012.

Footnote 39 This cost increase is above the 3 percent inflation rate for that period.

Footnote 40 Performance Management Framework (PMF) (2010) Strategic Outcome 2.1 sets performance target of 90 percent for Maritime Training indicator “% personnel trained at optimal course capacity.”

Footnote 41 Vice-Admiral Paul Maddison, previous commander RCN responding to questions during the Standing Senate Committee on National Security and Defence (February 2013).

Footnote 42 The Naval Strategic Analysis Team report concluded with a series of recommendations and follow-on actions that led to the implementation of a functional model to a) balance efficiency and effectiveness; b) enhance force development; c) identify personnel efficiencies; d) improve governance across Maritime Command; and e) minimize complexity and redundancy, bearing in mind the realities of geography where there is a need for parallel structures when it makes sense to do so.

Footnote 43 Naval Transformation Board II Transformation Initiatives Planning for 2012/13.

Footnote 44 (1) Readiness Management/Fleet Scheduling – Built upon the premise of a single pan-Navy Fleet Scheduling Authority, this initiative will increase coordination of the scheduling functions through the creation and management of a single (national) fleet scheduling mechanism. (2) Personnel Coordination Centres – the creation of four Personnel Coordination Centres, in Ottawa, Quebec City, Halifax and Esquimalt (controlled by a single national authority) has occurred to better manage the employment and training of naval personnel. (3) Naval Training System – consolidation of the five naval schools, through the creation of a more efficient and effective integrated Naval Training System, under a single command, will continue.

Footnote 45 Reg F manning in Maritime Command and Control (i.e., HQs including Fleet Commanders' staffs and the Maritime Operational Groups) increased from 668 to 947 between 2008 and 2013, with most significant increases in 2009 and 2010, with all the major increases occurring in the MARLANT and MARPAC HQs.

Footnote 46 Item X Force Generation Lines of Operation, Naval Board Record of Discussion, 1/10, April 22, 2010.

Footnote 47 More specifically, in FY 2012/13, RCN expenditures declined by 5.6 percent (or -$36 million) due to reductions in O&M of 8 percent (-$25 million) and capital of 22.3 percent (-$6 million), and decreases in civilian pay of 3.7 percent (-$10 million) and Reserve pay of 4.8 percent (-$4 million). ADM(Mat) expenditures increased by 13.2 percent due to equipment maintenance activities ($101 million).

Footnote 48 Total costs for Maritime Air were estimated to be small (< 7 percent of total costs for PAA 2.1 in FY 2010/11) (source: ADM(Fin CS)). Attributed allocations from Director General Aerospace Equipment Program Management / Director Aerospace Equipment Program Management and RCAF were insignificant (each < 1 percent of total costs for PAA 2.1 in FY 2010/11).

Footnote 49 Source: Leeuw, Frans and Jos Vaessen. ‘‘Impact Evaluations and Development – NONIE Guidance on Impact Evaluation 2009.’’ NONIE Network of Networks on Impact Evaluation. (2009): 31. http://www.worldbank.org/ieg/nonie/index.html (July 16, 2012).