Evaluation of the Maritime Equipment Program

June 2016

1258-224 (ADM(RS))

Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

Acronyms and Abbreviations

Executive Summary

This report presents the findings and recommendations of the Evaluation of the Maritime Equipment Program (MEP) within the Department of National Defence (DND). The evaluation was conducted by Assistant Deputy Minister (Review Services) (ADM(RS)) between January 2015 and June 2016, as a component of the DND / Canadian Armed Forces (CAF) Five-Year Evaluation Plan for fiscal years (FY) 2012/13 to 2016/17 and in compliance with the Treasury Board Secretariat (TBS) Policy on Evaluation (2009). As per the TBS policy, the evaluation examined the relevance and performance of the program over a five-year period (FY 2009/10 to FY 2014/15).

Program Description

The MEP supports the life cycle management of all Royal Canadian Navy (RCN) vessels through the provision of maintenance, repair, engineering, inspection and testing, and disposal services. These vessels include 12 Frigates, one Destroyer, four Submarines, 12 Maritime Coastal Defence Vessels, eight Orca-class Training Vessels and 74 Auxiliaries. MEP services are delivered by RCN Fleet Maintenance Facilities (FMF) located on each coast as well as by private service contractors. While the majority of support is delivered within the two home ports of Esquimalt BC and Halifax NS, deployed maintenance and repair teams can be dispatched to support operations undertaken by the globally deployable RCN.

Over the five years covered in the evaluation period, maritime equipment expenditures averaged $1.1 billion per year and encompassed approximately 3,000 individual contracts. In FY 2014/15, the program expenditure addressed by this evaluation, which excludes capability acquisition, upgrade and insertion, 1st and 2nd line maintenance and national warehousing and storage, was $592 million. The DND Materiel Group, under the Assistant Deputy Minister (Materiel) (ADM(Mat)), is responsible for this program.

Relevance and Performance

The provision of a maintenance and in-service support (ISS) program for the RCN is aligned with federal government and departmental roles, responsibilities and priorities. The MEP plays an important role in contributing to the RCN by ensuring that equipment is available in the right quantity, mix and condition. Accordingly, the program helps enable the readiness and employment of multi-purpose combat-capable forces and fulfill the obligations of Defence services.

During the five-year period examined by the evaluation (2010-2015), the MEP was challenged by the high operational tempo of the RCN. This included operations in support of the mission in Afghanistan, security for the Vancouver 2010 Winter Olympics, anti-trafficking and anti-piracy missions in the Caribbean and the Indian Ocean, humanitarian and disaster relief in Haiti, and support to NATO¹ combat missions and exercises in the Mediterranean and the Black Sea. This tempo was compounded by a reduction in the availability of major vessels. These included the Halifax-class modernization, which removed on average two to three frigates from operational service throughout the period as these vessels were modernized, as well as the decommissioning of two Iroquois-class destroyers and two auxiliary/refueling ships. All of this impacted the MEP in a significant manner as the remaining major vessels were deployed more frequently, which posed significant challenges with respect to scheduling and conducting maintenance in port. Further, the decommissioning of the four vessels has actually increased workload due to disposal activity. In addition, during this period the four Victoria-class submarines were re-introduced into service.

Funding and the availability of personnel also continued to be an issue. The actual amount of maintenance provided continues to fall short of targets (developed in conjunction with RCN/ADM(Mat) operational experience and manufacturer/supplier recommendations) due to issues of not only ship scheduling, but also the amount of funding provided and the ability of ADM(Mat) personnel to execute.

Despite these challenges, the program for the most part has been able to meet the needs of the RCN. Ships have been available in a state that has met operational requirements. Although readiness targets have been challenged, this can be seen to be a short-term issue which, as the Halifax-class modernization becomes complete in 2017, should be addressed.

With respect to the economy of the program, the evaluation was satisfied with the business processes that are in place to maximize the value for money and minimize unnecessary work. Expenditure demand is determined through a sound challenge function of both RCN and ADM(Mat) experts, which enables the department to act as a smart customer. The age of the fleet poses challenges, but it also provides the advantage of many years of operational experience which enables the program to effectively prioritize work requested. The RCN/ADM(Mat) also have sound processes in place to manage contractors to ensure value for money once contracts are in place. Furthermore, the RCN is well advanced with respect to the establishment of strategic partnerships with suppliers, as evidenced by the number of ISS contracts (ISSC).

With respect to efficiency, both the amount of work per full time equivalent (FTE) and the amount of expenditures per vessel have essentially kept pace with inflation, but for the most part, processes have remained unchanged. Opportunities may be present in improving the efficiency of the maintenance process. These are being examined through numerous initiatives, including an extensive performance management system which will provide significant business intelligence to make resource decisions in the near future.

Key Findings and Recommendations

Key Finding 1: The MEP, a major component of DND management of materiel, is a critical support element contributing to force readiness, allowing the CAF to respond to the needs of the Government of Canada (GC).

Key Finding 2: The MEP aligns with the roles and responsibilities set by the federal government.

Key Finding 3: Through the rigorous management of materiel, the MEP is aligned with the government’s priorities of the defence of Canada and the security of Canadians.

Key Finding 4: DGMEPM directly supported ADM(Mat) and RCN priorities through alignment of DGMEPM strategic objectives and level of effort monitoring.

Key Finding 5: The progression of the Halifax-Class Modernization / Frigate Life Extension (HCM/FELEX) project and the achievement of steady state for the Victoria-class submarines together contributed to increased operational readiness for the RCN.

Key Finding 6: DGMEPM has been in a state of fundamental transformation throughout the evaluation period.

Key Finding 7: DGMEPM lacked sufficient HR to adequately support ongoing operations and change initiatives simultaneously.

Key Finding 8: DGMEPM and the RCN coordinated their efforts in the utilization of the Defence Resource Management Information System (DRMIS) and developed extensive performance data collection capabilities.

Key Finding 9: The backlog of outstanding DRMIS enhancement requests trended upwards.

Key Finding 10: ADM(IM) implementation of DGMEPM DRMIS enhancements is inefficient.

Key Finding 11: DGMEPM has proper oversight of FMF/contractor ship disposal activities.

Key Finding 12: DGMEPM lacked formal tracking of outputs of all stages of disposal process.

Key Finding 13: The implementation of the FMF Integrated Management System enhanced performance data collection and ability to measure 3rd line maintenance performance.

Key Finding 14: The 3rd line maintenance conducted by the FMFs has decreased during the evaluation period.

Key Finding 15: The reduction in 3rd line maintenance funding to FMFs contributed to an increase in outstanding 3rd line maintenance during the evaluation period.

Key Finding 16: The linking of Repair and Overhaul (R&O) to FMFs fund transfers limits FMFs and DGMEPM efficiency.

Key Finding 17: Multiple maintenance contracts tracking tools inhibit proactive management.

Key Finding 18: Further progress on Naval Materiel Assurance (NMA) process transformation is required.

Key Finding 19: ADM(Mat) and DGMEPM have inventory governance including inventory management planning and oversight.

Key Finding 20: DGMEPM consistently underspent its allocation for spares purchases.

Key Finding 21: The MEP is supported by an integrated RCN/ADM(Mat) governance framework.

Key Finding 22: While lacking resources, DGMEPM progressed selected change initiatives enhancing strategic coordination and transforming governance.

Key Finding 23: DGMEPM has the base of a sound HR framework.

Key Finding 24: DGMEPM has initiated multiple tactical level performance data collection initiatives, but lacked a Performance Measurement Framework.

Key Finding 25: DGMEPM demonstrated efficient use of resources during the evaluation period.

Key Finding 26: DGMEPM requires additional funding and HR to meet the maintenance demand.

Key Finding 27: The DGMEPM strategic HR governance structure and the introduction of decision support tools improved DGMEPM efficiency.

Key Finding 28: A common RCN/DGMEPM availability performance data structure for fleet availability data would improve efficiency.

Recommendations

ADM(RS) Recommendation 1: ADM(Mat) investigate methods to increase support to DGMEPM change initiatives.

ADM(RS) Recommendation 2: ADM(Mat) investigate leveraging the DGMEPM DRMIS performance data collection capabilities across the organization.

ADM(RS) Recommendation 3: ADM(Mat) liaise with ADM(IM) to investigate and implement methods to reduce the number of open DRMIS incidents and reduce the duplication of effort in implementing DRMIS enhancements.

ADM(RS) Recommendation 4: ADM(Mat) investigate a method to reduce amount of outstanding 3rd line maintenance.

ADM(RS) Recommendation 5: ADM(Mat) and the RCN investigate methods to remove financial limitations on R&O work conducted by FMFs.

ADM(RS) Recommendation 6: DGMEPM investigate the utilization of a single contract management tool.

ADM(RS) Recommendation 7: DGMEPM investigate methods to increase NMA support.

ADM(RS) Recommendation 8: DGMEPM complete the Performance Measurement Framework to mitigate findings 5, 6, 12, 20 and 22 in this report.

ADM(RS) Recommendation 9: ADM(Mat) investigate the feasibility of utilizing the DGMEPM HR tools across the organization to enhance ADM(Mat) and other EPM performance measurement.

ADM(RS) Recommendation 10: ADM(Mat) and the RCN generate a common availability performance data structure for their respective performance frameworks and investigate the potential for use of DRMIS to automate performance reporting.

Note: Please refer to Annex A—Management Action Plan for the management responses to the ADM(RS) recommendations.

Back to Table of Contents

1.0 Introduction

1.1 Context for the Evaluation

This report presents the results of the evaluation of the DND MEP. The evaluation examined the relevance and performance of this program over the period of 2010 to 2015, and was conducted in accordance with the 2009 TBS Policy on Evaluation. This program has not been previously evaluated.

In the conduct of the evaluation, an Advisory Panel, comprised of representatives from ADM(Mat) and the RCN, supported ADM(RS). The Advisory Panel was consulted at key intervals throughout the evaluation, specifically when defining the project scope, developing the logic model, identifying key performance indicators, and reviewing preliminary findings. The findings and recommendations in this evaluation may be used to inform management decisions related to program delivery and resource allocation, and will serve as a baseline for future evaluations.

1.2 Program Profile

1.2.1 Program Description

The MEP encompasses the Materiel Acquisition and Support (MA&S) activities for maritime equipment. This includes the life cycle management of equipment from initial acquisition to final disposal. The governance and administration of the resources and activities required to execute the MEP are shared between ADM(Mat) and Commander RCN. Commander RCN, as a force generator, is accountable to the Chief of the Defence Staff for defining the requirements for naval assets and systems to be acquired. Additionally, the Commander RCN is responsible for custody and operational use of materiel commissioned into service, as the operational authority.

The Materiel Group, under ADM(Mat), is the single central service provider and program authority for materiel for the CAF and the DND. The ADM(Mat) is accountable to the Deputy Minister for materiel's full life cycle – from acquisition, through maintenance and support, to disposal.² DGMEPM, as the Naval Materiel Authority responsible to ADM(Mat), provides naval materiel program management services for the maintenance and support of all naval materiel. Additionally, DGMEPM is Commander RCN’s Senior Naval Engineering Advisor responsible for providing engineering and materiel support to the existing fleet and shore establishments.³

1.2.2 Program Objectives

The objective of the MEP, as the maritime component of the materiel life cycle program, is to ensure that maritime materiel capability elements are available in the quantity, mix and condition to meet the readiness requirements of the RCN with respect to the employment of multi-purpose combat-capable naval forces.⁴

The specific outcomes of the program in support of this objective are depicted in the MEP Logic Model (Annex C).

1.2.3 Stakeholders

The life cycle management of defence materiel is a whole-of-government activity with multiple stakeholders and complex processes for which DND is not the sole owner.⁵

Stakeholders include, but are not limited to, the following:

• RCN;
• Canadian Army (client common maritime equipment);
• Royal Canadian Air Force (client common equipment);
• Canadian Special Operations Forces Command (client common maritime equipment);
• Other DND/CAF organizations;
• Other government departments such as: Public Services and Procurement Canada (PSPC), TBS, and Innovation, Science and Economic Development Canada; and
• Canadian defence industry.

1.3 Evaluation Scope

1.3.1 Coverage and Responsibilities

The MEP is linked to the DND/CAF Program Alignment Architecture (PAA)⁶ under the strategic outcome: Defence Remains Continually Prepared to Deliver National Defence and Defence Services in Alignment with Canadian Interests and Values. Specifically this evaluation includes portions of the following program and sub-programs:

• 4.0 Defence Capability Element Production
  • 4.2 Materiel Lifecycle
    • 4.2.1 Materiel - Portfolio Management
    • 4.2.2 Materiel - Acquisition
    • 4.2.3 Materiel - Equipment Upgrade and Insertion
    • 4.2.4 Materiel - Divestment and Disposal
    • 4.2.5 Materiel - Engineering, Test, Production and Maintenance
    • 4.2.6 Materiel - Inventory Management and Distribution
    • 4.2.7 Materiel - Strategic Coordination, Development and Control

The evaluation examines the life cycle materiel management activities conducted by the MEP to assess the achievement of: the expected outcomes of the program; relevance; and efficiency and economy. The evaluation is primarily focused on a horizontal examination of the Life Cycle Materiel Manager (LCMM) functions across all DGMEPM directorates with respect to the defined maintenance outcomes for the program.

1.3.2 Exclusions

The evaluation excluded MEP activities that have been previously evaluated or audited by ADM(RS) or by the Office of the Auditor General of Canada. Additionally, activities that will be the subject of future evaluation(s) are not covered in this evaluation. The evaluation scope excluded the following components of the MEP:

• 4.2.2 Materiel - Acquisition;⁷
• 4.2.3 Materiel - Equipment Upgrade and Insertion;⁸
• First and second line components of 4.2.5 Materiel - Engineering, Test, Production and Maintenance;⁹
• National warehousing and storage;¹⁰ and

• Activities related to readiness covered by other evaluations.¹¹

1.3.3 Resources

1.3.3.1 Financial

Over the five fiscal years covered by the evaluation, maritime equipment expenditures averaged $1.1 billion. This figure includes capability acquisition, upgrade and insertion which are outside of the scope of the evaluation. Table 1 contains the DGMEPM funding within the PAAs evaluated in thousands of dollars. The MEP expenditures averaged approximately $540 million per year. The reduction of materiel funding in 2014 was primarily a result of Deficit Reduction Action Plan (DRAP) implementation.

Table 1. DGMEPM Funding. This table contains the DGMEPM funding for each year of the evaluation in thousands of dollars.

1.3.3.2 Personnel

DGMEPM is staffed with both civilian and military personnel. Table 2 presents the distribution of the staff for the period covered by the evaluation (FY 2010/11 to FY 2014/15). The data is based on extracts from the Human Resource Management System. These figures are population,¹² not FTEs, and exclude personnel in positions dedicated to capability acquisition. DGMEPM military staffing has been consistent but civilian staff decreased 7 percent from FY 2010/11 to FY 2014/15.

Table 2. Distribution of DGMEPM Staff. This table summarizes the distribution of DGMEPM personnel for each year of the evaluation period.¹⁴

1.3.4 Issues and Questions

In accordance with the TBS Directive on the Evaluation Function (2009),¹⁵ the evaluation addresses the five core issues related to relevance and performance. An evaluation matrix listing each of the evaluation questions, with associated indicators and data sources, is provided at Annex D. The methodology used to gather evidence in support of the evaluation questions can be found at Annex B.

Back to Table of Contents

2.0 Findings and Recommendations

The following sections examine the extent to which the MEP addresses a demonstrable need; is appropriate to the role of the federal government; is aligned with DND/CAF priorities; and demonstrates efficiency and economy. To make this determination, the evaluation analyzed program documents, empirical data and qualitative evidence.

2.1 Relevance—Continued Need

This section examines whether the sustainment of maritime defence materiel continues to address an ongoing demand for the outcomes and objectives of the program. The findings in this section are based on documents reviewed and key informant interviews, with representatives from ADM(Mat) and the RCN. The following indicator was used in the assessment of alignment with federal roles and responsibilities:

• evidence of current and future need for the program.

Key Finding 1: The MEP, a major component of DND management of materiel, is a critical support element contributing to force readiness, allowing the CAF to respond to the needs of the GC.

The maintenance of equipment and the supply of materiel are directly linked to and essential to the readiness of the CAF; when called upon by the GC, DND must be capable of rapidly responding to the life cycle management of materiel requirements in order to deploy and maintain the equipment in the right quantity, mix and condition.¹⁶

The Government has committed to and continues to renew the CAF’s core equipment platform, to improve and to replace existing equipment as part of the Canadian defence policy.¹⁷

The MEP is the only program existing within DND which develops, leads and executes a materiel acquisition and sustainment program on behalf of ADM(Mat) for the RCN.¹⁸ DGMEPM, through the MEP, supports the life cycle management of 123 vessels.

In addition, the program manages 184,354 Stock Codes excluding the items provided by the Land Equipment Program and the Aerospace Equipment Program¹⁹ such as ammunition and maritime patrol aircraft. DGMEPM’s overall notional budget allocation for FY 2015/16 is approximately $1.08 billion across all fund types (Vote 1 and Vote 5) with approximately 3,000 contracts. Vote 1 National Procurement (NP) (C113 NP) funds which are used to maintain all classes of RCN ships have the largest allocation: $730 million with more than 2,800 contracts.²⁰

The demands on the MEP have increased and are expected to continue to increase. Canada is currently undergoing one of its largest fleet renewal programs while at the same time supporting the existing fleet, undertaking submarine design authority responsibilities and Halifax-Class Modernization implementation in addition to its maritime procurement responsibilities, and with limited human resource capacity and competencies.²¹ The Defence Renewal Committee recently stated: "There is also a growing expectation that readiness levels of fleets must be maximized in order to meet operational demands in a security environment that continues to be unpredictable and volatile."²²

2.2 Relevance—Alignment with Federal Roles and Responsibilities

This section examines the extent to which the program aligns with departmental and federal roles and responsibilities. The following indicators were used in the assessment of alignment with federal roles and responsibilities:

• alignment of materiel management with government acts and legislation; and
• alignment of materiel management with government policies and strategies.

Key Finding 2: The MEP aligns with the roles and responsibilities set by the federal government.

The MEP clearly aligns with the roles and responsibilities of DND. According to TBS Policy on the Management of Materiel (2006), federal ministers are accountable for the management of materiel and for the sound stewardship of the materiel entrusted to their organization. All government departments are required to manage materiel over the entire life cycle to ensure that both operational requirements and value for money are met when planning, acquiring, sustaining, assets.²³

The Defence Production Act assigns the Minister of PSPC the responsibility for the management of defence materiel.²⁴ PSPC and DND have agreed, in principle, to a division of responsibilities between the two departments for the quality assurance of materiel and services acquired. Subsequently, the Minister of PSPC delegated the authority to purchase defence materiel and services, under conditions listed in the Procurement Administration Manual,²⁵ to the Minister of National Defence.²⁶

In fulfilling the delegated responsibilities to procure and sustain defence materiel, DND is guided by the TBS Policy on the Management of Materiel. This policy requires all government departments to manage defence materiel over the entire life cycle of the materiel to ensure that it meets the departmental operational requirements of an effective program delivery and to ensure value for money when planning, acquiring, sustaining, and disposing of materiel assets.²⁷

2.3 Relevance—Alignment with Government Priorities

This section examines whether the objectives of the MEP are consistent with current GC and DND/CAF priorities. The following indicators were used in the assessment of alignment with federal priorities:

• alignment to GC defence priorities; and
• alignment to DND/CAF priorities.

Key Finding 3: Through the rigorous management of materiel, the MEP is aligned with the government’s priorities of the defence of Canada and the security of Canadians.

As stated in the Speech from the Throne (2013), the defence of Canada and the security of Canadians continue to be a top priority for the GC. The GC is committed to ensuring that the CAF has the tools it needs to deal with the full range of threats and challenges to Canada and Canadians.²⁸

Through defence policy, the GC is committed to providing the CAF with the adequate resources for training, spare parts, and the equipment needed to provide effective military support, and to modernize and upgrade existing fleets in order to support CAF readiness and training requirements.²⁹ This is accomplished by maintaining rigorous stewardship over all assets, including materiel.³⁰ The following objectives of the MEP contribute directly to this achievement:

• Sustain operational priorities in accordance with agreed ADM(Mat) and Commander RCN plans;
• Ensure that the ships and submarines of the RCN and their crews remain as safe as practicable within the context of the operational environment in which they operate;
• Deliver new capabilities in accordance with the Navy’s future program; and
• Enhance the Materiel Group’s ethical stewardship of public funds and resources.”³¹

Key Finding 4: DGMEPM directly supported ADM(Mat) and RCN priorities through alignment of DGMEPM strategic objectives and level of effort monitoring.

ADM(Mat) and the RCN generate level 1 strategic priorities based upon DND priorities. The Report on Plans and Priorities (RPP) 2014-2015 states that one of the key priorities for DND is that: “The CAF must ensure resources are aligned and available to support determined readiness levels and therefore able to posture military capability to meet planned and anticipated requirements of the Government of Canada as articulated in defence policy.”³² The RCN identified four priorities that flowed directly from the Chief of Defence priorities.³³ ADM(Mat) also identified four priorities for FY 2015/16.³⁴

DGMEPM level 2 priorities and level of effort align with both ADM(Mat) and RCN level 1 priorities. DGMEPM is responsible to ADM(Mat) for compliance with the materiel acquisition and support policies and guidelines for the life cycle support of all naval platforms, systems and equipment. As well, DGMEPM is responsible to Commander RCN for all materiel and logistics support to the existing fleet and shore establishments.³⁵ DGMEPM measures the level of effort³⁶ expended on DGMEPM, ADM(Mat) and RCN priorities to ensure alignment. Figure 1 illustrates the weekly average of the percentage of DGMEPM hours expended from October 20, 2014 to March 29, 2015. The DGMEPM level of effort was balanced with 64 percent of the effort on ensuring excellence in operations at sea and 29 percent on enabling the RCN’s transition to the future fleet.

Figure 1. DGMEPM Level of Effort on DGMEPM/ADM(Mat) and RCN Priorities. This figure illustrates the level of effort expended by DGMEPM on identified DGMEPM, ADM(Mat) and RCN priorities.³⁷

DGMEPM and the RCN further refine the above priorities into DGMEPM and RCN 5F³⁸ strategic objectives. DGMEPM identified, in order of priority, the following ten DGMEPM Strategic Objectives (SO):

• SO1: Build a more robust Naval Materiel Assurance program;
• SO2: Position DGMEPM “total workforce” to deliver and support future requirements;
• SO3: Define future naval ISS exploiting and particularizing ISS contracts;
• SO4: Maximize potential and usage of DRMIS;
• SO5: Improve planning and management of a realistic MEP;
• SO6: Increase rigour in use of RCN and DGMEPM governance structure;
• SO7: Improve ability to manage MEP by class and capability;
• SO8: Exploit and particularize horizontal capability development;
• SO9: Establish the Naval Materiel Management System (NaMMS) as the singular materiel management system within the ”One Navy” concept; and
• SO10: Leverage and influence higher-level change initiatives.”³⁹

DGMEPM strategic objectives align with the RCN strategic objectives. Figure 2 is the result of a DGMEPM exercise to ensure alignment of the RCN 5F strategic objectives with the 10 DGMEPM strategic objectives. There are four 5F objectives that have no identified linkage to DGMEPM strategic objectives. RCN objectives 20, 21 and 23 have an internal RCN focus and minimal MEP support requirements. RCN objective 12 operations in the Arctic is a deliverable of the AOPS project being managed by Director General Major Project Development (Sea).

Figure 2. Map of DGMEPM to RCN Strategic Objectives.⁴⁰ This table denotes the mapping of DGMEPM strategic objectives to the RCN strategic objectives.

2.4 Performance—Achievement of Expected Outcomes (Effectiveness)

This section evaluates the achievement of the MEP expected outcomes, with a focus on PAA 4.2 Materiel Lifecycle Program. PAA 4.2 aims to provide the defence materiel products and services required to ready defence force elements and fulfill obligations of Defence Combat and Support Operations, Defence Services and Contributions to Government, and Defence Capability Development and Research activities. It ensures that defence materiel capability elements are available in the quantity, mix and condition that enable the production of ready force elements, the employment of multi-purpose combat-capable forces, and Defence services.⁴¹

Accordingly, an assessment of the MEP was conducted based on the following immediate outcomes:

• Materiel - Portfolio Management PAA 4.2.1 - Ability to ensure MEP materiel, equipment, and equipment fleets are available in the quantity, mix and condition to meet Defence needs;
• Materiel - Divestment and Disposal PAA 4.2.4 - Ability to ensure MEP materiel, equipment and equipment fleets are reduced in a safe, responsible and timely manner to meet Defence needs;
• Materiel - Engineering, Test, Production and Maintenance PAA 4.2.5 - Ability to ensure that defence materials, equipment and services satisfy operational and performance requirements and are in a condition that allows for their safe use at the inception and throughout their life cycle;
• Materiel - Inventory Management and Distribution PAA 4.2.6 - Ability to ensure MEP materiel, equipment and fleets are supplied and made available in a timely manner at the location to meet the needs of Defence; and

• Materiel - Strategic Coordination, Development and Control PAA 4.2.7 - Extent that the strategic coordination, development and control of MEP materiel meets Defence needs.⁴²

2.4.1 Immediate Outcome – Portfolio Management PAA 4.2.1

The Materiel Portfolio Management Program seeks to ensure that defence materiel, equipment, fleets, and supporting elements, the materiel elements of Defence capabilities, are managed throughout their life cycle and made available to the production of ready force elements, capability development and research, and Defence services. The Program is primarily focused on ensuring that sufficient types, quantities and mixes of equipment, fleets and associated materials are available and can be delivered to enable the readiness training and the employment of multi-purpose combat-capable forces as well as other Defence services. This is accomplished through the delivery of defence materiel portfolio management, coordination, and project planning services. This Program oversees and prompts the suite of services which, in turn, ensure that defence materiel portfolio elements are available and in a condition that allows for their use in military readiness training, operations and Defence services.⁴³

The indicators used to assess this immediate outcome are as follows:

• extent of key fleets availability to meet operational and force development tasks in accordance with defence policy;
• extent that DGMEPM supports the management of the materiel portfolio; and
• extent that the enterprise supports the management of the materiel portfolio.

Key Finding 5: The progression of the Halifax-Class Modernization / Frigate Life Extension (HCM/FELEX) project and the achievement of steady state for the Victoria-class submarines together contributed to increased operational readiness for the RCN.

The RCN requires vessels to be either at sea or available to sail in order to meet the RCN and DGMEPM outcomes. Vessel availability is a key deliverable required to meet DGMEPM’s mission to deliver safe and capable fleets today⁴⁴ and the RCN mission to generate and maintain combat-effective, general-purpose maritime forces for employment on operations.⁴⁵ DGMEPM achieves its mission by delivering a safe, modern, technically ready, fully capable, and well-supported Canadian Naval Fleet to the Commander RCN and Operational Commanders.⁴⁶

The RCN conducts readiness sustainment under PAA 3.1.1. The MEP component of the RCN performance indicator is the materiel availability to meet the RCN’s fleet readiness requirements. Table 3 denotes the PAA 3.1.1 indicators and thresholds for the evaluation period; the variance in the targets used over the evaluation period reflects the ongoing efforts to better assess fleet availability. The thresholds lacked the fidelity for analysis for the three years where the readiness requirements were met. With respect to the two years where measured readiness was below threshold levels, the RCN stated the principal cause was the refit/upgrade of the Halifax-class vessels and the progression of the submarines to operational status.⁴⁷ The Halifax-class vessels were undergoing planned mid-life refits under the auspices of the HCM/FELEX program, which led to prolonged periods of unavailability while maintenance and upgrades were progressed to extend the life and increase the operational effectiveness of the platform.

Table 3. RCN Fleet Availability. This table denotes the PAA 3.1.1 indicators and thresholds for the evaluation period.

Under the new PAA structure approved in April 2014, ADM(Mat) reports consolidated materiel availability. DGMEPM is required to report the MEP component of this PAA to ADM(Mat). With the new PAA structure, under PAA 4.2.1, DGMEPM assesses the availability of each vessel within a key fleet against the RCN Managed Readiness Plan expressed as a percent of the sailing requirements met.⁴⁸ As the PAA was approved in April 2014, only one year of DGMEPM data was available for the evaluation period. Table 4 demonstrates the availability by quarter for 2015. The results demonstrate the continuing improvement through the year.

Table 4. DGMEPM PAA Performance. This table denotes the indicators for PAA 4.2.1 and 4.2.5, thresholds and results by quarter for 2015.

The performance measurement of PAA 4.2.5 lacked accuracy. For 2015, ADM(Mat) required DGMEPM to provide the status of eight selected projects out of the over 350 DGMEPM maintenance projects.⁴⁹ A sample size of less than 3 percent of the total number of maintenance projects places an increased importance on a small number of projects. Table 4 demonstrates an increase between quarter 1 and quarter 2 in DGMEPM performance score from 63 to 75 as the result of one out of the eight projects returning to schedule. The evaluation noted that DGMEPM has begun investigating the utilization of DRMIS to automate PAA reporting. This Performance Indicator is in an early stage of development and its validity and relevance will increase as more projects are incorporated into the assessment.

Key Finding 6: DGMEPM has been in a state of fundamental transformation throughout the evaluation period.

In 2010, MEPM began a fundamental shift in how the RCN fleet would be supported. DGMEPM began transforming from a “maintenance” management to a “materiel” management system with the publication of the NaMMS in 2011.⁵⁰ The DGMEPM Strategic Plan 2012-2017 identified the delivery of DGMEPM as being at risk due to upcoming constraints of managing fleet renewal challenges and managing the current and aging fleet. As a result of the constraints, a mission statement was formulated and communicated to balance and sustain execution whilst undertaking the necessary changes for them to remain relevant: “to develop, lead, and execute an effective and efficient materiel acquisition and support program on behalf of ADM(Mat) for the Royal Canadian Navy” with four key objectives:

• Sustain operational priorities in accordance with agreed ADM(Mat) and Commander RCN plans;
• Ensure that the ships and submarines of the RCN and their crews remain as safe as practicable within the context of the operational environment in which they operate;
• Deliver new capabilities in accordance with the Navy’s future program; and
• Enhance the Materiel Group's ethical stewardship of public funds and resources.

DGMEPM further identified that in order to adapt to its new “fluid” environment it would require decisions, initiatives and actions that would encompass two concurrent approaches: continued delivery of the MEP and change to adapt to the new environment. To address these risks and challenges, DGMEPM created the MEPM Strategic Initiative (MSI) in 2010⁵¹ by which it would ensure that all strategic initiatives/changes, deemed necessary to ensure successful implementation of DGMEPM performance and strategic objectives, were coordinated, managed and implemented, thereby “transforming its major business processes and creating a lean and efficient organization.” The DGMEPM transformation was conducted as a component of the Strategic Coordination, Development and Control immediate outcome (PAA 4.2.7).

MEPM commenced the change management process execution phase without completing the preparation phase. The outputs of the change management preparation phase are: root causes of the current problems, a vision of the ideal state, and a baseline of the current state.⁵² DGMEPM stated the root cause of limited human resource capacities and competencies resources, and the end state of, by 2017, transforming DGMEPM to achieve a class-focused effective program management system focused to support the current and future fleets.⁵³ DGMEPM is lacking the establishment of a performance baseline. The performance baseline requires DGMEPM to map and document its “as is” processes and procedures and develop key performance metrics for resource inputs and outputs.

Key Finding 7: DGMEPM lacked sufficient HR to adequately support ongoing operations and change initiatives simultaneously.

DGMEPM directed the concurrent objectives of continuing delivery of the MEP, and transforming DGMEPM. The DGMEPM transformation was to be accomplished via the MSI. The MSI was to be completed in 2017 with DGMEPM transformed to a division that will have achieved an effective program management system that is class-focused to better support the current and future fleet.⁵⁴

The implementation of the change initiatives did not come with any additional DGMEPM personnel resources. DGMEPM staff was directed to “free up” the human resource capacity required to focus support on high-priority taskings such as the MSI. DRAP reductions in DGMEPM personnel created an additional third course of action of maintaining core competencies and capabilities while reducing personnel. The support of current and future RCN fleets and DRAP limited the DGMEPM staff availability to support the MSI. In 2010, the DGMEPM level 3 directors were assigned the responsibility for the generation of the 17 DGMEPM business processes identified in the logic model (Annex C). As of January 13, 2016, none of the processes have been completed. DGMEPM also lacked the funding required to implement the MSI. Although initiated in 2010, the MSI received no dedicated funding until 2013. The lack of funding resulted in limiting the support available in managing the initiative and in supporting the DGMEPM staff in generating the MSI deliverables. The MSI deliverable dates were altered as the deadlines were not met, without tracking the original dates or the linkage to other deliverables. The creation of a project management schedule in 2015 with milestone deadlines and linkages should improve MSI management and the measurement of the effect of human resource limitations.

Key Finding 8: DGMEPM and the RCN coordinated their efforts in the utilization of DRMIS and developed extensive performance data collection capabilities.

The DGMEPM data collection capability enabled the conduct of an in-depth and detailed evaluation. This approach supported more focused and fulsome findings and recommendations to be included in this report. The findings and recommendations in this report support ADM(RS) acknowledgement of DGMEPM’s data collection and analysis capabilities.

MEPM and the RCN have been heavily involved in DRMIS since its inception. DGMEPM was the ADM(Mat) division selected for the development of the Materiel Acquisition and Support Information System (MASIS) in 1998. MASIS evolved into DRMIS in 2010 with the inclusion of functionality previously in the department financial and materiel management system. There has been consistent support from the consecutive DGMEPMs on enhancing data collection capabilities. This senior support focused on the aim of DRMIS as the single repository of performance data to enable information to be written once and used many times.⁵⁵ The data collection capability incorporates enterprise-wide systems, such as DRMIS, and DGMEPM specific data tools such as Level of Effort recording and the Human Resources/Capability Management tool.

DRMIS support is synchronized between the RCN and DGMEPM. The DRMIS management relationship between DGMEPM and the RCN is structured under the Maritime DRMIS Integration Authority,⁵⁶ co-chaired between COS MEPM and D Nav Log.⁵⁷ DGMEPM provides an MDIA project manager (Technical Authority for all Maritime DRMIS-related matters) and a deputy project manager (RCN/MEPM DRMIS Solution Manager). MDIA consists of three DRMIS Centres of Excellence (West, East and Central). Collectively, all RCN and DGMEPM initiatives and incidents are assessed and priorities assigned and staffed to senior management, primarily through ADM(Mat) Director General Materiel Systems and Supply Chain (DGMSSC). While the RCN/MEPM DRMIS structure exists and is in operation, the completion of the documentation, such as terms of reference, roles and authorities, would formalize the structure. Note that the MDIA is being replaced by the NaMMS Support Tools Working Group.

Benchmarking DGMEPM performance with Land Equipment Program Management⁵⁸ was not possible. LEPM and DGMEPM support benchmarking activities divisions by having the same materiel management responsibilities and operating under the same government processes. However, DGMEPM has data collection and DRMIS utilization capabilities that DGLEPM has assessed will take until March 2018 to attain.⁵⁹ There is potential gain in ADM(Mat) by leveraging the DGMEPM data collection capability to generate a single standard across the ADM(Mat) organization to support performance management and benchmarking.

Key Finding 9: The backlog of outstanding DRMIS enhancement requests trended upwards.

The MEP requires the support of the enterprise systems, primarily DRMIS, to convert performance data into actionable information. This is accomplished through decision support capabilities, such as queries, embedded in the enterprise tools. DRMIS is managed utilizing incident reports. Incidents have three main categories: service requests; change requests; and problem reports. The number of unresolved DRMIS incidents has trended upwards during the evaluation period. The incident backlog was 1,800, in November 2012 it exceeded 2,000, and by October 2013 had accumulated over 4,000. An RCN/DGMEPM incident prioritization and rationalization effort was conducted to remove duplication and address the outstanding incidents and in December 2014 over 3,000 remained.⁶⁰ Figure 3 shows the percentage of the backlog for three categories of incidents at the end of December 2014. Note that as of December 13, 2014, 11 percent of the 3,000 outstanding incidents were still unresolved after two years. The resolution of the outstanding incidents would enhance DRMIS by reducing the requirement to utilize other methods and standalone systems to support the MEP.

Figure 3. Outstanding Incident Reports. This figure illustrates the initiation dates of outstanding incident reports from 2010 to December 2014.

DRMIS support is provided by ADM(IM) under Directorate of DRMIS.⁶¹ DRMIS ISS is provided by DND personnel supported by contracted personnel. One of the identified DRMIS challenges is insufficient ISS funding to support the necessary number of resources to begin to tackle the outstanding backlog.⁶²

Key Finding 10: ADM(IM) implementation of DGMEPM DRMIS enhancements is inefficient.

MEPM has embedded DRMIS experience. The Director Maritime Management and Support (Management Information Systems) has the responsibility to enable and facilitate the realization of MA&S processes within the corporate naval management information system of record. The naval system of record includes DRMIS, and those data systems that either interface with DRMIS, or are used to support DRMIS in the execution of the MA&S task. Under the director is a dedicated team of DRMIS experts who provide DRMIS support to DGMEPM.

DGMEPM dedicated DRMIS experts have the rights and authority to create temporary queries. DGMEPM identifies a potential enhancement and the DGMEPM DRMIS staff generates the necessary code for the enhancement. The enhancement is then tested to ensure that it meets the requirement, and as a Quality Assurance function. The temporary query can be identified to be converted to a permanent query by DRMIS staff.

The current DRMIS staff process requires the query requests to be coded from scratch in Development and to be migrated through Quality Assurance before finally being brought to Production. DGMEPM staff consider this process to be consuming precious DRMIS resources.

2.4.2 Immediate Outcome – Materiel Divestment and Disposal PAA 4.2.4

Disposal is the final step of life cycle materiel management. When assets are no longer required by the DND/CAF, they are declared surplus and disposed of through sales, trade-in, transfer, donation or destruction in accordance with GC and departmental policies and procedures.⁶³ The mandate for the DGMEPM Capability Divestment and Disposal process is to ensure that naval materiel assets are disposed of when the capability is no longer required. The DGMEPM Capability Divestment and Disposal Business Process includes divestment, which is defined as the removal of a capability leading to:

• disposal⁶⁴ of an asset (equipment or vessel) through
  • sell-off,
  • gratuitous transfer or donation, or
  • destruction;⁶⁵
• ceasing supporting training;
• removing supporting billets;
• logistics changed/ceased, plus parts disposal; and
• possible change to doctrine.

The indicators used to assess this immediate outcome are as follows:

• evidence of DGMEPM governance on disposal;
• percentage of materiel required/due for disposal;
• percentage of materiel disposed of (five years); and
• percentage of disposals on schedule.

Key Finding 11: DGMEPM has proper oversight of FMF/contractor ship disposal activities.

DGMEPM has proper oversight of coastal ship disposal via several channels. First, the Maritime Disposal Working Group⁶⁶ was created to bridge the gaps between different authorities by developing terms of reference, and to streamline DGMEPM’s governance of ship disposal. Second, a master equipment database was created by DGMEPM to survey and denote each system and compartment on the ship as well as identify controlled goods and environmental assessments which ensure disposals are conducted in a proper manner. Third, an experienced disposal manager conducts monthly disposal telecoms at the coastal level with the various organizations involved, from logistics to maintenance facilities to receive updates and provide direction. This includes a working relationship with the Commanding Officer, who manages the actual disposal which includes, but is not limited to: determining artefacts and relics, managing the safe handling of hazardous materials, complying with controlled goods policy and environmental assessments.

Key Finding 12: DGMEPM lacked formal tracking of outputs of all stages of disposal process.

The DGMEPM process on divestment and disposal has evolved during the evaluation period. Divestment and disposal being managed by multiple organizations involved at different stages has caused lack of oversight. The Directorate of Disposal, Sales, Artefacts and Loans (DDSAL) in DGMSSC is the functional authority responsible for the disposal of surplus assets on behalf of DND.⁶⁷ In 2009, DDSAL initiated a review of the disposal program, stating that “the Disposal Program was fragmented and decentralized, delivered by multiple organizations with many different responsibilities and approaches, based on policies dating from 1986.”⁶⁸ This ongoing transformation included the issue of new Defence Administrative Orders and Directives (DAOD)⁶⁹ updating departmental policies on disposal and the publication of the Disposal of Surplus Materiel Guidance⁷⁰ detailing the application of the new policies.

To continue the transformation, the Director of Minor War Vessels and Auxiliaries (MWVA) was tasked as the process owner for the DGMEPM Strategic Initiative (MSI) Divestment and Disposal Business Process Redesign. The Business Process Management owner, MWVA, has accountability and ownership for the design and performance of the associated process and their impact on other DGMEPM processes. A cross-functional Business Process Redesign Working Group was identified to document and analyze current maritime divestment and disposal processes in an effort to operationalize the DGMSSC-led disposal business process mapping exercise that has been ongoing since 2010. The Business Process Redesign Working Group published a very thorough report mapping the as-is process to the to-be, and concluded with a series of recommendations in August 2013 on: governance, training and development, change management, communication strategy, and stakeholder engagement.

The evaluation could find no documented evidence to demonstrate that DGMEPM is aware of the status of disposal for each system being disposed. The lack of data makes it difficult to control and govern the process. A platform equipment database was created in MWVA which surveys the ship and identifies each system and compartment, catalogues controlled goods and environmental assessments, and tracks the progression of disposal. While it does provide a snapshot of major disposal, it is not used to measure outputs and performance indicators. As for systems, no such database exists. The disposal process is initiated by a formal Declaration of Surplus letter; between that time and the final disposal, the evaluation noted no formal monitoring or reporting is done on the status of disposal. In an interview, MWVA concurred that this is an issue.

2.4.3 Immediate Outcome – Materiel Engineering, Test, Production and Maintenance PAA 4.2.5

The MEP provides 3rd line maintenance in support of the management of the fleets of the RCN. Third line maintenance is defined as maintenance performed under the authority, sponsorship, and funding of DGMEPM, and is typically maintenance that can be performed by industry or by qualified FMFs with specialized tools, skill sets, equipment and facilities.⁷¹ DGMEPM manages the 3rd line maintenance activities utilizing either internal DND or contracted resources.

The indicators used to assess this immediate outcome are as follows:

• FMF performance in the conduct of 3rd line maintenance is achieving its expected outcomes;
• ISSC performance in the conduct of 3rd line maintenance is achieving its expected outcomes; and
• effectiveness of fleet naval materiel assurance.

Key Finding 13: The implementation of the FMF Integrated Management System enhanced performance data collection and ability to measure 3rd line maintenance performance.

The principal DND resources for 3rd line maintenance activities are provided by the RCN resources. They are the FMFs, one in Halifax, Nova Scotia (FMF Cape Scott), and another in Esquimalt, British Columbia (FMF Cape Breton). The FMFs’ mission is, as a strategic asset, to provide timely and effective engineering and maintenance services to the Navy and the Canadian Forces.⁷² The FMFs provide a full range of Naval Engineering Maintenance and Repair capabilities to support Naval Marine and Combat Systems in Formation warships and submarines, auxiliary vessels, and other Formation units.⁷³ The FMF expertise in the conduct of 1st and 2nd line maintenance is leveraged by DGMEPM to support 3rd line maintenance requirements.

As an RCN strategic asset, the overall FMF performance was previously evaluated.⁷⁴ The MEP evaluation scope focused on the FMF 3rd line maintenance activities conducted in support of the MEP. However, the evaluation noted the significant progress achieved in the standardization of FMF activities and processes under a single Integrated Management System.⁷⁵ The coordination of FMF Cape Breton and FMF Cape Scott activities supported the collection of detailed 3rd line performance measurement data from both FMFs. This capability was invaluable in providing the data requested in support of this evaluation.

Key Finding 14: The 3rd line maintenance conducted by the FMFs has decreased during the evaluation period.

FMF maintenance activities conducted for DGMEPM are resourced though funds provided from DGMEPM to the FMF conducting the required work. The DGMEPM NP funds are converted into salary funds (SWE).⁷⁶ These 3rd line maintenance activities enable the RCN to both employ a larger FMF workforce than it could otherwise with its L111 SWE allocation, and provide a consistent level of work to the FMFs.⁷⁷

Funding provided to the FMFs for the conduct of 3rd line maintenance has decreased during the evaluation period. Historically, each FMF received $36 million and $38 million, Cape Breton and Cape Scott respectively, in 3rd line maintenance contract work from ADM(Mat). This reduction in 3rd line maintenance conducted at the FMFs is largely the result of decisions to reallocate work to industry service providers and to ensure that both FMFs can fully commit to their 2nd line maintenance commitments. The most noteworthy example of this practice is the 3rd line maintenance support to the Victoria-class submarines provided under the Victoria In-Service Support Contract.

In 2015, the Commander RCN expected 3rd line maintenance activities would continue to decrease in accordance with the planned investment of $24 million per FMF, a trend that was forecasted to continue for the next four years as ADM(Mat) gradually reduces funding levels for 3rd line activities to settle at a steady state level of approximately $18 million per FMF in 2018. However, as part of 3rd line discussions between ADM(Mat) and the RCN, the FMF funding was subsequently lowered to $15 million beginning immediately in 2015.⁷⁸ This reduction in funding reflects the intent to focus the FMFs role as a strategic asset centred on providing 2nd line support to the fleet with industry playing a greater part in the execution of 3rd line activity.

The DRAP reductions at FMF have been concentrated at 3rd line maintenance support to the MEP. Table 5 in the resources section denotes the total work conducted by the FMFs during the evaluation period. The reduction in funding has led to a significant decline in the maintenance conducted by the FMFs during the evaluation period. Table 5 also denotes that while FMF overall maintenance hours were reduced by 36 percent, 3rd line maintenance hours were reduced by 60 percent. The percentage of FMF work hours expended on 3rd line compared to the total hours worked decreased from 51.22 percent in 2011 to 32.50 percent in 2015, an 18.72 percent reduction. Of the 695,000-hour reduction in FMF activities, 586,000 hours have come from 3rd line maintenance support.

Table 5. Total Work Assigned to FMFs. This table illustrates NP work completed by FMFs as ratios to the total number of work hours (all numbers are in thousands).

Key Finding 15: The reduction in 3rd line maintenance funding to FMFs contributed to an increase in outstanding 3rd line maintenance during the evaluation period.

DGMEPM identifies the work required to be conducted by the FMFs. The work is then assigned in DRMIS. A DRMIS extraction of the work assigned to FMFs but not commenced for the evaluation period is denoted in Table 6.⁷⁹ With the exception of 2012, the amount of outstanding FMF maintenance hours has doubled every year. As previously noted, the evaluation period encompasses a period of transition wherein the FMF was refocused on 2nd line support and industry service providers assumed a greater role in 3rd line work. The increase in outstanding FMF hours includes work that will be subsequently undertaken by industry as a result of this work reallocation.

Table 6. Outstanding Work Assigned to FMFs. This table contains the number of hours of work assigned to FMFs but not completed.

Key Finding 16: The linking of R&O to FMF fund transfers limits FMF and DGMEPM efficiency.

Repair and Overhaul⁸⁰ is defined as “the act of returning an item to a serviceable condition by disassembly, repair or replacement of damaged or deteriorated parts, reassembly, adjustment, examination and testing to specified standards. Whereas repair normally entails the correction of specific defects only, overhaul entails not only the replacement of worn and damaged parts but also of parts whose service life has expired or is about to expire, in order to return the item to its original performance and an acceptable life expectancy.”⁸¹ R&O work conducted by FMFs is conducted as a component to the total FMF 3rd line maintenance budget provided by ADM(Mat). Once the DGMEPM support budget limit is reached, R&O work is no longer authorized.The conduct of R&O enhances FMF efficiency. The level of FMF support by the RCN is fluid as activity levels are dependent upon the number of ships available for maintenance in home port. R&O work is conducted by FMFs as “opportunity” work that enhances efficiency by allowing FMFs to conduct R&O activities when RCN requirements are lower (load leveling).⁸²

The linkage of R&O work to the DGMEPM 3rd line maintenance funding ceiling limits ADM(Mat) and RCN efficiency. When R&O work is ceased, FMF is unable to leverage its excess workforce capacity when ships are deployed and DGMEPM loses opportunities for increased R&O. The outstanding FMF R&O work assigned but not commenced rose from 1,262 hours in 2014 to 2,579 hours in 2015.

FMF staff stated that at times they continue to conduct R&O work to ensure that their staff are employed efficiently, but the work is not tracked in DRMIS due to the financial linkage to work conducted. As DRMIS is the base for FMF performance measurement, this work also negatively affects the measurement of FMF efficiency.

Key Finding 17: Multiple maintenance contracts tracking tools inhibit proactive management.

DGMEPM conducts a portion of the 3^rd line maintenance activities utilizing contracted support. DGMEPM maintains multiple support contracts utilizing multiple Supply Managers. The support can be provided at multiple levels such as: complete capabilities (Lockheed Martin Canada for the Combat Management Systems); original equipment manufacturers (such as Raytheon Canada for the Close-In Weapon System); and for specific activities such as vessel refits.

DGMEPM maintenance contracts are managed by the Directorate of Maritime Procurement (D Mar P). Members of D Mar P manage multiple maintenance contracts. Members have their own methods and tools to manage their respective contracts. When information is requested on contract status, the members extract the data from their databases and format the data into a common structure. This creates an additional level of effort for contract managers and limits the ability of the Director to supervise the contact management and prioritize contract renewal efforts.

Key Finding 18: Further progress on NMA process transformation is required.

ADM(Mat) is responsible for formulating and approving MA&S policy, and DGMEPM as the ADM(Mat) naval Director General is responsible for formulating and issuing Navy MA&S instructions and directives.⁸³ This responsibility is supported though the Naval Materiel Management System and the NMA program. Accountability for naval materiel management is shared between the ADM(Mat), who is the Materiel Authority (acquisition, ISS and disposal), and the Commander RCN, who is the Operational Authority (custody and operation).⁸⁴

NMA was identified as DGMEPM’s highest priority strategic objective.⁸⁵ NMA provides adequate confidence, from acquisition to disposal, that ships, as integrated platform systems, are fit for service, safe, and compliant with regulations for the protection of the environment.⁸⁶ The implementation of NMA within DGMEPM and the RCN leverages the best practices of Canada’s closest allies and includes the engagement of maritime classification societies, a structured system of certification, revised policy, governance and the oversight needed to assure effectiveness, safety and environmental compliance. As an overarching framework, NMA requirements cross multiple DGMEPM strategic objectives. NMA encompasses 40 percent of total MSI requirements (89 out of 221) and spanned nine out of ten of DGMEPM’s strategic objectives and seven of the eight MSI projects.

The NMA program is progressing; however, elements have been delayed due to priority and resource issues. Interviewees have stated that the progress completed was largely due to the dedication of staff members to improve NMA. Out of the 89 MNA requirements, 32 percent were yet to commence and 76 percent were less than half completed.⁸⁷ Out of the 89 MNA requirements, 79 percent are identified as medium risk or higher.⁸⁸ The NaMMS Management Board provides governance and oversight of the NMA program.

2.4.4 Immediate Outcome – Inventory Management and Distribution PAA 4.2.6

Inventory management and distribution is a key supporting activity of life cycle management. ADM(Mat) has the responsibility for CAF inventory management and distribution. DGMEPM is responsible to ADM(Mat) for MEP inventory management. Inventory management includes the R&O of defective parts and the acquisition of replacement spare parts. The evaluation of the R&O component of inventory management is in the 3rd line maintenance section of the evaluation.

The indicators used to assess this immediate outcome are as follows:

• evidence that DGMEPM is managing, monitoring, reporting and sustaining its stock level;
• trend of out of stock items;
• trend of funds expended on dormant buys;
• trend of funds expended on overbuys; and
• trend of funds expended on purchases of repairable items.

Key Finding 19: ADM(Mat) and DGMEPM have inventory governance including inventory management planning and oversight.

MEP Inventory management is a DGMEPM division-wide activity. Specific inventory management responsibilities are held by DGMEPM, Class Program Mangers, Equipment Group Program Managers, LCMMs and Supply Managers.

The DGMEPM governance provides the framework MEP inventory management. The coordination/management of MEP inventory management activities is the responsibility of the Director of Maritime Procurement. D Mar P manages the inventory management process utilizing the draft DGMEPM materiel management process. The process synchronizes inventory management activities and deliverables to the budget planning cycle and RCN priorities. The LCMMs and Site Authorities are responsible for conducting the inventory management activities under the control of the Equipment Group Program Managers. The strategic control of inventory management is exercised by the DGMEPM Board of Directors.

ADM(Mat) is transforming materiel management at the organizational level. One of the aims of this transformation is to enable LCMMs and Supply Managers to make well-informed decisions with respect to procurement, maintenance, distribution and disposal activities.⁸⁹ ADM(Mat) has developed an organizational level materiel forecasting and planning tool called the distribution resource planning (DRP) application. ADM(Mat) mandated the use of the DRP application in June 2009.⁹⁰ The Directorate of Supply Chain Operations (DSCO) is responsible for the support of the tool and provision of data to the level 2 ADM(Mat) divisions.

The ADM(Mat) DRP information required additional processing to be utilized by the level 2 divisions. The processing is conducted as a component of the DGMEPM materiel management annual review process. Specific MEP criteria are applied to the DRP data to provide the actionable MEP information. For example, out of stock items for Protecteur-class vessels are included in the Figure 4 DRP stock out report. The DGMEPM process removes the Protecteur-class items when calculating the MEP out of stock items.⁹¹ A DGMEPM analysis of the 1,592 items identified as out of stock resulted in approximately 100 items that required immediate action.⁹²

Figure 4. MEP Stock Out Items During Evaluation Period.⁹³ This figure illustrates the number of out of stock items identified by the DRP application during the reporting period.

DSCO has improved the accuracy of the DRP data for organization-wide issues. DRP initially included demands being met by projects as out of stock items. This increased the number of items identified as out of stock because the DRP tool did not recognize stock provided by the project as a source to fill the demand. DSCO enhanced the tool to separate project stock from actual stock out rate.⁹⁴ The continued support of the DRP will improve efficiency by decreasing the additional processing required by DGMEPM.

Key Finding 20: DGMEPM consistently underspent its allocation for spares purchases.

DGMEPM consistently underspent its final allocation for spares purchases. Table 7 contains the percent of allocation not utilized for the purchase of spares. DGMEPM underspent the final allocation by an average of 21.5 percent during the evaluation period.

The purchase of spares within DGMEPM is influenced by a number of factors, and it is used as one of the key mechanisms in managing budget fluctuations in the complex financial construct that DGMEPM operates. The comparatively short approval and acquisition time typical for many spares purchases enables their use as a tool to optimize in-year spending as other maintenance and acquisitions cannot be readily deferred or procured at short notice. For FY 2014/15, the initial L1 allocation was provided by the Deputy Minister on April 22, 2013,⁹⁵ vice the standard date of March 30, 2013.⁹⁶ DGMEPM was also subsequently directed by senior leadership to significantly reduce expenditures at the second quarter. The reduction was later removed, but with insufficient time in the fiscal year to complete longer lead time spare purchases. These two occurrences in combination were the key contributors to the 29 percent unspent allocation. The flexibility afforded by the ability to defer spares purchases to the next FY prevented more significant disruption to other less flexible elements of the MEP. Excluding 2014, the average unspent spare parts allocation is 14 percent.

Table 7. DGMEPM Inventory Expenditures. This table contains the percent of allocation not utilized for the purchase of spares during the reporting period.

There is a process to acquire replacement parts to correct equipment failures. When equipment fails, the initial action is to request a replacement part from the supply system. If the part is not in the supply system or cannot be delivered to meet the required sailing schedule, the part will be removed from another platform to correct the issue on the higher priority vessel. The removal of the part is documented using a Transfer Request⁹⁷ message. Table 8 indicates the number of TRANREQ messages sent by the RCN during the evaluation period excluding equipment being replaced by the HCM/FELEX project. The number of TRANREQ messages sent averaged 207 messages per year. The decreasing then increasing trend of the TRANREQs corresponds with the decreasing then increasing availability of ships as the HCM/FELEX project conducted the mid-life refit of the Halifax-class vessels.

Table 8. Number of RCN TRANREQs During Evaluation Period.⁹⁸ This table denotes the number of RCN equipment TRANREQs sent during the evaluation period.

TRANREQs are inefficient, and decrease RCN availability. A TRANREQ requires three times the labour compared to when the part is in the supply system. The part must be removed from the losing ship and then the new part reinstalled when the part arrives. The losing unit has lost the capability that the part supported until the spare part arrives. There is also the additional risk that the part may be damaged during the removal process resulting in two vessels requiring the part.

DGMEPM implemented measures to reduce the impact of funding challenges on spares procurement. DGMEPM requested that spare parts acquisitions be assessed for delivery times. The purchases of items with long delivery times were to be commenced at the beginning of the fiscal year with the short term items being postponed where possible to react to allocation changes. DGMEPM has incorporated these measures into the periodic financial reviews conducted by the DGMEPM Board of Directors. The unspent allocation for spares was reduced from 29 percent in 2014 to 8 percent in 2015. DGMEPM is also exploring and implementing ways to expand the scope within the Victoria In-Service Support Contract⁹⁹ to allow for more responsive parts purchasing for in-service submarines.

2.4.5 Immediate Outcome – Materiel Strategic Coordination, Development and Control PAA 4.2.7

The Materiel Strategic Coordination, Development and Control program aims to ensure that defence materiel, equipment, equipment fleets and all supporting elements are managed, coordinated and overseen so that they are available for the production of ready force elements and the employment of multi-purpose combat-capable forces as well as other Defence services. Results are achieved through the delivery of planning, design, development, implementation, coordination, control, governance, performance management, reporting, relationship and partnership management, and advice services as they relate to defence materiel in order to meet Defence readiness needs.¹⁰⁰

This section examines the DGMEPM governance and planning and coordination activities using the following indicators:

• evidence of a governance framework;
• extent that the MSI has changed DGMEPM strategic coordination, development and control;
• evidence of DGMEPM human resource management mechanisms, structures and frameworks; and
• evidence of DGMEPM financial management mechanisms/structures/frameworks.

Key Finding 21: The MEP is supported by an integrated RCN/ADM(Mat) governance framework.

The RCN and DGMEPM have an integrated structure of governance boards, committees and councils to support the MEP. Commander RCN is supported by the Naval Board and ADM(Mat) by the Materiel Group Management Committee. There is a clear reporting framework from the level 3 working groups to both level 1 boards. The working groups have RCN and DGMEPM members. The dual ADM(Mat)/RCN reporting responsibilities of the working groups enhance the ability of ADM(Mat) and the RCN to make informed and coordinated decisions.

Key Finding 22: While lacking resources, DGMEPM progressed selected change initiatives enhancing strategic coordination and transforming governance.

The MSI has transformed the strategic governance of the MEP. The vision for the DGMEPM end-state included a class-focused organization.¹⁰¹ This was completed on July 2, 2013, with an organizational change to create class organizations assigned/identified as the design authority.¹⁰² The class organizations include one for major surface combatant, one for submarines and a third for minor war vessels and auxiliaries.¹⁰³ Funding provided to the LCMMs is coordinated by the class organizations.

The class organizations have improved the strategic coordination. DGMEPM created an integrated program with linkages from the strategic (MEP Plan) to the tactical to DRMIS work packages. Three key components of the integrated program are the Class Program Plan, the Equipment and Functional Group Program Plan, and the Through Life Management Plan. The completion of these plans will significantly improve the DGMEPM coordination and monitoring capability.

DGMEPM generated a logic model in support of the MSI. ADM(RS) noted that DGMEPM is the sole organization to create and utilize a logic model prior to the commencement of an evaluation. The initial version of the logic model included outcomes, outputs and an initial set of nine key performance indicators. Through successive iterations, the logic model evolved into the Annex C version.

The current logic model is not in accordance with TBS guidance. The DGMEPM logic model lacks clear identification of activities, outputs, and immediate to ultimate outcomes.¹⁰⁴ Discussions were held with DGMEPM staff to convert DGMEPM “processes” equating to logic model “activities” for the purpose of the evaluation. However, no clear linkages existed between the processes and the PAA sub-sub programs.

Key Finding 23: DGMEPM has the base of a sound HR framework.

DGMEPM identified Strategic Human Resource Management (SHRM) as a core component of the MSI.¹⁰⁵ The aim of the SHRM project was to develop and implement a robust approach to addressing competency and knowledge management requirements, and to establish a framework for assuring the acquisition and development of the DGMEPM capacity and competency necessary to meet current and future demands by 2017.¹⁰⁶ DGMEPM created the Strategic Human Resource Board responsible for developing, monitoring and executing the Strategic Human Resource Management Plan in 2010. The DGMEPM Strategic Human Resource Board is supported by the human resource coordinators working group providing support, advice and recommendations. The DGMEPM formalized the strategic human resource governance structure in 2014.¹⁰⁷

The SHRM project is creating a competency-based management framework. The framework will be enabled by a collection of decision support tools collectively called the Human Resource Management Database.¹⁰⁸ The Human Resource Management Database is comprised of the Human Resources/Capability Management, Standard Management, Human Resource Management and Level of Effort tools.

2.5 Performance—Demonstration of Efficiency and Economy

The following section examines the extent to which the MEP uses the most appropriate and efficient means for its activities.

Under the 2009 Evaluation Policy, efficiency is defined as “maximizing the outputs produced with a fixed level of inputs.” In other words, minimizing the inputs used to produce a fixed level of outputs. Economy is defined as “minimizing the use of resources to achieve expected outcomes.”¹⁰⁹ For the purposes of the Evaluation Policy, these elements of performance are demonstrated under the following circumstances:

• outputs are produced at minimum cost (efficiency); and
• outcomes are produced at minimum cost (economy / cost effectiveness).

Accordingly, the Evaluation of the MEP considered efficiency in materiel management. The question of economy considered whether resources that are allocated to the program are reasonable, economical and sustainable. The following indicators were used to assess the economy and efficiency:

• the extent to which DGMEPM has implemented a performance measurement framework;
• demonstration of efficient use of HR;
• demonstration of efficient use of financial resources; and
• evidence that alternative processes / delivery arrangements are considered.

Key Finding 24: DGMEPM has initiated multiple tactical level performance data collection initiatives, but lacked a Performance Measurement Framework.

DGMEPM has initiated multiple tactical level performance measurement activities under the MSI. These initiatives enable DGMEPM to rapidly generate performance data upon demand. The capabilities such as the DGMEPM Logic Model, Level of Effort reporting, extensive use of DRMIS, and improving collection class performance data enabled ADM(RS) to conduct a more thorough evaluation than was possible for the Land Equipment Program. The lack of a Performance Measurement Framework limits the ability of the performance data collection initiatives to identify efficiencies and economies.

The performance framework will validate MSI collection initiatives, such as Level of Effort, ensuring collection and analysis resources are optimized. A MEP performance framework would provide the linkage of the individual performance measurement activities initiated by DGMEPM. This linkage would enable all DGMEPM members to identify the indicators and decisions that their respective data is supporting. The lack of performance indicators, thresholds and periodic reviews also limits DGMEPM ability to conduct proactive versus reactive performance measurement.

The completion of a DGMEPM performance framework would resolve or mitigate several findings in this report. The performance framework would affect the following report findings:

• Finding 5: by ADM(Mat) increasing the maintenance project sample size to improve accuracy and considering automating reporting process;
• Finding 6: by creating a performance baseline to measure the current state;
• Finding 12: by creating methods to track outputs and measure performance at all stages of disposals;
• Finding 20: by including impact reduction measures in the planning and process documentation; and
• Finding 22: by completing the plans for class, equipment group and through life management and updating the DGMEPM logic model.

Key Finding 25: DGMEPM demonstrated efficient use of resources during the evaluation period.

DGMEPM efficiently utilized the available resources during the evaluation period. Table 9 denotes that, when calculated in 2011 dollars, the DGMEPM expenditures per person increased by 11 percent during the evaluation period. DGMEPM expended an average of 95 percent of its allocation for the evaluation period. As noted earlier in the report, DGMEPM altered the funding for spares purchases to respond to funding changes during the year. While this enhanced stability in other maintenance areas, the resulting lack of available spares had a detrimental effect on the efficiency of corrective maintenance.

Table 9. DGMEPM Demand to Expenditures. This table compares the DGMEPM maintenance demand to the executable demand and the actual expenditures during the reporting period.

DGMEPM’s rigorous processes have also improved efficiency. DGMEPM incorporated the executable demand which allows for initial maintenance planning and prioritization. Monthly senior reviews including actual expenditures, expenditure trends, risks and decisions have improved DGMEPM resource managing. The focus on capability management and the creation of the SHRM tools have enabled DGMEPM to focus and prioritize the limited HR staffing resources to maintain vital capabilities. These efforts have resulted in a knowledgeable and capable staff who ensured efficient use of resources.

Key Finding 26: DGMEPM requires additional funding and HR to meet the maintenance demand.

DGMEPM was underfunded to meet the maintenance demand during the evaluation period. Table 10 compares the DGMEPM maintenance demand to the executable demand and the actual expenditures during the reporting period. The maintenance demand averaged $937 million per year with DGMEPM executing an average of $665 million or 72 percent of the maintenance demand. An average increase of 40 percent in maintenance funding would be required to meet the DGMEPM maintenance demand.

An increase in funding alone would not eliminate the shortfall in maintenance. Increased DGMEPM HR are required to meet the maintenance demand. DGMEPM defines the executable demand as the maximum maintenance that DGMEPM staff is capable of processing. The executable demand averaged $819 million of the $937 million maintenance demand (87 percent) for the evaluation period. The increase in 2015 demand was in response to the RCN requirement to return six Kingston-class vessels to service. An average increase of 14 percent in maintenance funding would be required to meet the DGMEPM maintenance demand.

Table 10. DGMEPM Demand to Expenditures. This table compares the DGMEPM maintenance demand to the executable demand and the actual expenditures during the reporting period.

Key Finding 27: The DGMEPM strategic HR governance structure and the introduction of decision support tools improved DGMEPM efficiency.

ADM(Mat) is responsible for managing civilian human resources within their area of responsibility and providing input into and implementing strategic and operational HR plans.¹¹¹ ADM(Mat) has delegated the MEP strategic and operational HR responsibilities to DGMEPM.

DGMEPM identified SHRM as a core component of the MSI.¹¹² The aim of the SHRM project was to develop and implement a robust approach to addressing competency and knowledge management requirements and to establish a framework for assuring the acquisition and development of the DGMEPM capacity and competency necessary to meet current and future demands by 2017.¹¹³ DGMEPM created the Strategic Human Resource Board responsible for developing, monitoring and executing the Strategic Human Resource Management Plan in 2010. The DGMEPM Strategic Human Resource Board is supported by the human resource coordinators working group providing support, advice and recommendations. DGMEPM formalized the DGMEPM strategic human resource governance structure in 2014.¹¹⁴

The SHRM project is creating a competency-based management framework. The framework will be enabled by a collection of decision support tools collectively called the Human Resource Management Database.¹¹⁵ The Human Resource Management Database is comprised of the Human Resources/Capability Management, Standard Management, Human Resource Management and Level of Effort tools.

Efficient human resource management requires the ability to make informed decisions. The DGMEPM Human Resources/Capability Management tool provides both tactical and strategic information to support informed HR decisions. The HR database provides position-specific data such as designation criteria, training education, qualification, experience and professional association requirements. The Human Resources/Capability Management tool also provides strategic information including demographics, vacancy risk assessment (based on likelihood and impact), and staffing priority.¹¹⁶ DGMEPM utilizes the Human Resources/Capability Management tool as the key support tool for HR decisions. Each section head assesses their vacant positions noting the risk to the organization and the staffing priority in the Human Resources/Capability Management tool. The Human Resources/Capability Management tool-identified staffing priority positions are reviewed by the DGMEPM Strategic Human Resource Board for endorsement of the DGMEPM staffing priority.¹¹⁷ Efficiency is improved by focusing the limited HR staffing resources on positions with the greatest effect on DGMEPM operations.

DGMEPM developed the Level of Effort reporting decision support tool under the MSI. The purpose of Level of Effort reporting is to allow DGMEPM to better understand what type and amount of work is being performed by staff. A first pilot was conducted in summer 2014, with approximately 120 DGMEPM staff. A second pilot followed in the spring of 2015, with approximately 450 DGMEPM staff participants. The tools and processes used were subsequently refined and the new tool reflects what was learned during the first two pilots.¹¹⁸

The Level of Effort tool supports strategic situational awareness and decision making and addresses 183 specific activities. The activities in the Level of Effort tool are linked to the MEP business processes and business process categories in the MEP logic model.¹¹⁹ The Level of Effort tool provides strategic information such as that shown in Figure 1.

Key Finding 28: A common RCN/DGMEPM availability performance data structure for fleet availability data would improve efficiency.

As the indicators above demonstrate, both the RCN and DGMEPM utilize ship availability data for their respective performance measurement responsibilities. DGMEPM is required to report the PAA 4.2.1 performance to ADM(Mat) once per quarter. DGMEPM assesses the availability of each vessel within a key fleet against the RCN requirements expressed as a percent of the sailing requirements met.¹²⁰ As sailing requirements change, data is requested from the RCN. Gathering RCN data has proved a challenge to DGMEPM. DGMEPM requested a formal procedure to facilitate this process.

The Commander of the RCN is responsible for reporting PAA 3.1.1 Maritime Roles - Readiness Sustainment, and PAA 3.3.1 Maritime Environment - Force Element Production. The expected output of PAA 3.1.1 is that force elements assigned maritime roles remain continuously ready to apply Defence capabilities during operations against threats or to deliver Defence services.¹²¹ PAA 3.3.1 results are delivered through a tiered readiness process where force elements are assembled from the fundamental elements of Defence capability (i.e. military personnel, materiel and information systems, information and, in some cases, real property) and then readied through various training, certification and close-support maintenance/production programs.¹²²

The automation of a common set of availability data for the RCN and DGMEPM would improve efficiency. Vessel availability data is provided to DGMEPM by the RCN. DGMEPM has experienced challenges in acquiring the required quarterly ship availability data from the RCN.¹²³ The generation of common vessel availability data in DRMIS would improve the performance reporting capability of both the RCN and DGMEPM and provide the Commander RCN with the ability to rapidly access availability data.

Back to Table of Contents

Annex A—Management Action Plan

The following provides prefatory remarks to give context to the individual Management Action Plans. I would like to first express my appreciation for the thoroughness and diligence of the ADM(RS) evaluation team who worked in close collaboration with Maritime Equipment Program Management (MEPM) staff and identified opportunities to enhance the efficiency and effectiveness of our organization. MEPM has embarked on a multi-faceted and ambitious change agenda that touches all aspects of the Maritime Equipment Program; as such, the engagement of objective and experienced evaluators at this juncture provides us with the needed course corrections required to optimize our organization and processes.

The evaluation report’s overall assessment commences by observing on the continued relevance of the Maritime Equipment Program and then notes that the Division was in a state of transformation throughout the evaluation period. While the atmosphere of continual change has presented its own challenges and stresses within the workplace, the Division at large is cognizant that it will only remain relevant to the Royal Canadian Navy (RCN) and the Department if it is successful in implementing this change-specific agenda. The introduction of three new classes of ships constituting the largest fleet renewal since World War II; the need for the continued and safe operation of older and new vessels; an evolving human resources (HR) demographic with new competency requirements; and a tightly constrained financial resource base were all factors that militated in favour of the comprehensive re-engineering of our business.

As a result of this need for re-engineering, an exhaustive process of change management has altered the landscape of naval equipment support. Key features include Class Program Managers who act as single points of accountability to marshal the resources needed to meet operational requirements set by the RCN, while also ensuring safety and environmental compliance. Technical, procurement and business specialists, in turn, support these Class Program Managers while leveraging a myriad of business processes to regulate and manage the resources and specialist competencies needed within the enterprise. The Division is exploiting best practices of our close allies, including the engagement of classification societies, to establish a rigorous Naval Materiel Assurance framework that ensures safety and environmental compliance, and enhances operational effectiveness. On the coasts, the Fleet Maintenance Facilities are refocusing their efforts on their traditional 2nd line work while the Division is developing and implementing a rationalized approach in proportioning 3rd line work in an optimized manner. Finally, an overarching performance management framework is being put in place to measure performance and to provide the business intelligence needed for resource decisions and continuous improvement.

The challenges inherent in moving forward with this change agenda, while simultaneously supporting naval operations across the globe, were significant and the HR shortfall was noted as a further key finding in the report. While securing the necessary competencies to deliver technical and managerial excellence will always present challenges, the enhanced planning tools that have been developed within the Division are proving to be invaluable in identifying, quantifying and substantiating resource needs. Strategic guidance is progressively translated into class, system and equipment level support needs that include contracting funds and personnel, as well as technical and management competencies. These requirements are rolled up and prioritized to ensure resources are apportioned in accordance with strategic objectives and organizational needs. During execution, resources, progress of key milestones and the level of effort are objectively measured to ensure adherence to plans, and any deviation is both understood and managed effectively. As a result of these change initiatives, the Division has been able to assess and justify its resource requirements with enhanced assurance, and has been able to make sound decisions in assigning resources to key priorities.

Through the Maritime Equipment Program Management (MEPM) Strategic Initiative (MSI), a step in defining the future naval In-Service Support landscape has been taken. This initiative supports the deficit reduction goal of having tailored sustainment solutions to be developed through a business case analysis process. It will deliver the systemic approach to naval maintenance that is required to ensure that ships and systems remain within design intent. The ultimate outcome of the framework will be the financially viable sustainment of materiel that is fit for purpose, safe and environmentally compliant.

Execution of the change agenda, in concert with the professionalism of our personnel, will situate this Division for the successful support of the future Navy. While there are many challenges yet to be addressed, the foundational aspects of enhanced planning, greater visibility into resource usage and enhanced HR management practices are critical success factors.

Management Action

Concur with findings. To resource the change program activities, MEPM has leveraged a suite of mature tools and planning practices to assist in prioritizing HR requirements as well as developing a cascading and hierarchical set of plans that serve to substantiate and quantify funding demands. As a result of this planning process and the ensuing resource decisions, the change activities identified in the report are staffing new hires and/or putting contracted support in place.

Management Action Plan: As MEPM advances its change agenda over the next three years, resource requirements will evolve and achieve clarity as each activity is further defined, developed and institutionalized. Adherence to the rigorous planning process will ensure that the resource demands for the change agenda are identified and prioritized according to organizational need. As such, DGMEPM will continue to follow its planning process and will consolidate and incorporate resource requirements for change initiatives within the annual MEPM Business Plan.

Closure: This Management Action Plan (MAP) will be closed once the FY 2019/20 planning cycle is complete and the Maritime Equipment Program Plan (MEPP) resource allocation has been identified. It is acknowledged that this period extends beyond the target of two years for typical MAP close-out; however, the three-year duration is consistent with the magnitude of organizational change being effected.

OPI: ADM(Mat)/DGMEPM/DMMS(FM)

Target Date: June 2019

Management Action

Concur with findings. Leveraging the capabilities and experience that MEPM has gained through its development of performance data collection tools within DRMIS is wholly consistent with the objectives of a learning organization. As such, a number of forums were established to share MEPM experience with stakeholders including the RCN, and the Land and Air Equipment Program Management Divisions. The benefits gained through such activities include reducing duplication of effort as well as enhancing performance management and benchmarking activities.

Management Action Plan

To assess the potential gains accruing from broader adoption of the MEPM DRMIS toolkit within ADM(Mat), the following actions will be undertaken:

Management Action Plan 1: MEPM will carry out a review of their overall DRMIS performance data collection capability and identify MEPM performance parameters and associated business processes that could be applicable to the Materiel Group. These findings will be presented in a report for review and analysis prior to presentation at an appropriate Materiel Group Management Committee (MGMC).

Management Action Plan 2: DCOS(Mat)/DMGSP will continue reviewing DRMIS project performance data collection capabilities that have Materiel Group project performance and reporting applicability and produce a report that articulates a recommended implementation plan for review at an appropriate MGMC.

Closure: This MAP will be closed when the finalized reports have been presented to the appropriate MGMC (by June 2018).

OPI: ADM(Mat)/COS(Mat)/DMGSP // DGMEPM

Target Date: June 2018

Management Action

Concur with findings. The issues raised within the evaluation report are areas of concern for both ADM(IM) and the broader DRMIS user group. As a result, a two-pronged strategy has been put in place and is supported by a robust and hierarchical DRMIS governance construct at the Director, Director General and Assistant Deputy Minister levels. The two elements of the strategy are as follows:

• DRMIS incidents will be grouped and prioritized thereby eliminating duplication of effort and reducing the number of open incidents; and
• A federated development construct will be developed leveraging experienced analysts who are under contract to DRMIS user clients. These analysts will augment the ADM(IM) support group and aid in reducing the number of open incidents.

Both elements of the strategy are in place as pilot projects to support the November 2016 major DRMIS release. Based on user input, the strategy implementation will be finalized during the course of the next three major DRMIS releases in March, July and November of 2017, respectively.

Management Action Plan: Given their past experience with DRMIS, and their involvement in these initiatives, MEPM will prepare a report on the findings of the two pilot projects and the ensuing recommendations for strategy development. The report will articulate the proposed procedures to reduce DRMIS incidents within MEPM and the Materiel Group. MEPM will continue to monitor the progress of strategy development and advise the MGMC when the strategy is complete.

Closure: This MAP will be closed once the strategy is complete and all procedures have been identified and reported to the MGMC. This will follow the release of the November 2017 major DRMIS release.

OPI: ADM(Mat)/DGMEPM/DMMS(MIS)

Target Date: March 2018

Management Action

Concur with findings. The apportionment of third-line work between industry and the FMFs is governed through a collaborative effort between the RCN and ADM(Mat) and incorporates a governance structure that is both multi-tiered and robust. To reduce the amount of outstanding third-line maintenance at the FMFs, a number of actions were already taken. These actions include procedural changes to control and prioritize the third-line work that is directed to the FMFs.

Management Action Plan: MEPM, in conjunction with the RCN, will continue to develop existing processes and further investigate the means by which to reduce the amount of outstanding third-line maintenance both within MEPM and the FMFs. Progress reporting will be made to governance and the associated reduction measures will be reported in the FY 2018/19 planning guidance.

Closure: This MAP will be considered closed when direction on the RCN/MEPM third-line work planning process has been fully encapsulated in the FY 2018/19 MEPM Business Planning Guidance Document, the RCN Integrated Business Planning Directive and the Performance Measurement documents of both organizations.

OPI: ADM(Mat)/DGMEPM/DMMS(FM)

Target Date: December 2017

Management Action

Concur with findings. As part of the planning and management cycle, the RCN, in conjunction with ADM(Mat), strive to achieve a balance between third- and second-line work within the FMFs. While there are no specific limitations on Repair and Overhaul (R&O) work, there are global limits on the total third-line work that can be directed to the FMFs. To achieve the optimum balance between work streams, a robust governance process is in place and procedures have been amended to prioritize all third-line work.

Management Action Plan: To address the recommendation made in the evaluation report, a number of additional actions will be undertaken. These actions include the alignment of MEPM’s demand-driven plan with the FMFs’ capacity-based Annual Operating Plans, as well as agreed MEPM/RCN procedures on ways to make the current policy more flexible to cope with year-to-year fluctuations. These procedures will include measures, where possible, that allow the FMF to take on R&O as opportunity work during periods of reduced ship availability. Revised procedures will be incorporated in FY 2018/19 planning guidance.

Closure: This MAP will be considered closed once strategic direction has been confirmed through governance and guidance has been formalized in annual planning documentation within both MEPM and the RCN. The relevant planning guidance documents are the MEPM Planning Guidance and the RCN Integrated Business Planning Directive.

OPI: ADM(Mat)/DGMEPM/DMMS(FM)

Target Date: December 2017

Management Action

Concur with findings. MEPM/D Mar P (Director Maritime Procurement) will investigate the adoption of a single contract management tool for use within the Directorate. This will include a review of existing DRMIS information and reports as well as consideration of existing contract databases to determine if DRMIS can be fully adopted to manage MEPM’s broad range of contract types. Once this investigation of contract management tool options is complete, a report will be made to MEPM governance that will assess benefits and costs and will provide recommendations for implementation.

Closure: This MAP will be closed once the report has been presented to the DGMEPM Board of Directors and a decision taken as to implementation (by December 2017).

OPI: ADM(Mat)/DGMEPM/D Mar P 3

Target Date: December 2017

Management Action

Concur with findings. Given the complexity and far-reaching implications of implementing Naval Materiel Assurance (NMA) within MEPM and the RCN, the development of a robust NMA framework is expected to take a further five years. One of the key challenges in meeting this timeline will be to find dedicated resources with the required knowledge and competencies.

To prioritize and quantify HR and funding demands, MEPM has developed a suite of mature tools and planning practices that have been used to generate an assessment of the resources required to support the NMA initiative. As a result of these planning processes and concomitant resource decisions, actions are underway to augment NMA staff with new personnel and contracted assistance.

Management Action Plan: As MEPM and the RCN implement the NMA program over the coming years, resource requirements will evolve as activities are further defined, developed and institutionalized. The staffing and contractual support levels achieved to date are estimates and will be evaluated for sufficiency. It is assessed that greater clarity with regard to steady state resource demands will be achieved within three years. As such, MEPM will continue to follow its planning process and will consolidate and incorporate resource requirements for NMA within the annual MEPM Business Plan.

Closure: This MAP will be closed once the FY 2019/20 planning cycle is completed and the NMA component of the Maritime Equipment Program Plan resource demand has been identified. It is acknowledged that this period extends beyond the target of two years for typical MAP close-out; however, the three-year duration is consistent with the complexity inherent in the introduction of the NMA program.

OPI: ADM(Mat)/DGMEPM/DNPS 8 - NaMMS Program Manager

Target Date: June 2019

Management Action

Concur with findings. To complete the development of the Performance Management Framework several key steps were taken. A Performance Management Framework architecture has been developed that links to the Departmental Program Alignment Architecture, is fully integrated into the MEPM family of plans, and rests upon DRMIS as a data repository. Further, and to support development, a MEPM Performance Management Advisory Group, with representation from ADM(RS), as well as other Subject Matter Experts, has recently been established to support the project team.

Related Findings:

The development of the Performance Management Framework will address specific findings noted in the evaluation report as follows:

• Finding 5 – increasing the ADM(Mat) maintenance project sample size. The process and timeline by which this will be achieved will be formalized in the Performance Management Framework definition phase documentation with a target date of December 2016.
• Finding 6 – creating a performance baseline. A comprehensive Maritime Equipment Program Performance Management Report (MEP PfR) is under development with a target date of September 2016. The MEP PfR includes performance analysis on the Maritime Equipment Program Plan, the Naval Maintenance Management System, business processes and strategic objectives and will constitute the performance baseline for the Division.
• Finding 12 – tracking outputs and performance for disposals. The initial actions have been a comprehensive disposal policy review. Policies will be updated by June 2017 and relevant Key Performance Indicators will be identified, developed and reported to governance with a target date of March 2018.
• Finding 20 – including impact reduction measures in planning and process documentation. Through the adoption of enhanced prioritization processes, which were informed by impact reduction measures, Director Maritime Procurement staff met targeted goals for spares procurement for FY 2015/16. These revised procedures have been documented in the D Mar P procedures and instructions.
• Finding 22 – completing the plans for class, equipment group and through life management and update of the logic model. Class Program and Equipment Group Plans have achieved stability as of March 2016. Through Life Management Plans are evolving with stability forecast for March of 2018. The MEPM Logic Model and subordinate Business Process logic models will undergo further refinement and development with a target date of end-September 2017.

Management Action Plan: To progress the Performance Management Framework, an incremental spiral development process will be undertaken. A Definition phase is scheduled to conclude in the Fall of 2016 and Initial Operational Capability is targeted for end-December 2017. Project close-out is planned for June 2018. MEPM will prepare a Performance Management Framework Project Close-Out Report incorporating mitigation measures for each of the above findings and present its results to an appropriate MGMC.

Closure: This MAP will be closed through submission of a Performance Management Framework Project Close-Out Report to governance with a target date of June 2018.

OPI: ADM(Mat)/DGMEPM/DMMS(FM)

Target Date: June 2018

Management Action

Concur with findings. As part of its ongoing renewal of HR planning and associated efforts to enhance performance management, a number of tools have been developed within MEPM. These tools include those related to Level of Effort Recording (LoER), Integrated Program View (IPV), Human Resource Management (HRM) and Competency-Based Management (CBM).

Management Action Plan: To address the recommendation within the evaluation report, DGMEPM will carry out an investigation into the broader adoption of these HR tools within the Group to enhance performance management. This investigation will assess the feasibility of adopting these tools as well as associated costs and benefits and will provide recommendations. These findings will be presented at an appropriate MGMC.

Closure: This MAP will be closed when the finalized report has been presented to the appropriate review committee (by December 2017).

OPI: ADM(Mat)/COS Mat // DGMEPM/DMMS(FM)

Target Date: December 2017

Management Action

Concur with findings. The challenges with generating an automated common availability performance data structure for ADM(Mat) and the RCN include, inter alia, data security and the identification of suitable metrics to capture the complexities of a warship’s multi-domain capabilities. Steps that have been taken include those to automate performance reporting through the joint RCN/ADM(Mat) Command Analytics Support Centre. This work has complemented other discrete performance measurement activities that are reported through governance.

Management Action Plan: To achieve the recommendations provided in the report, MEPM will, in conjunction with stakeholders, carry out the following actions:

• Create a Naval Performance Management Working Group, with appropriate governance, by a target date of June 2017;
• Identify and rationalize performance reporting and associated data requirements with a target date of September 2017;
• Develop a Naval Materiel Management System (NaMMS) Performance Management Framework with a target date of March 2018;
• Investigate current DRMIS performance reporting capabilities and identify gaps by June 2018; and
• Complete a performance management pilot project or projects centered on assessing the feasibility of an availability performance construct with a target date of March 2019.

Closure: This MAP will be considered closed when the above items have been achieved and an appropriate brief provided at an MGMC (by April 2019). It is acknowledged that this period extends beyond the target of two years for typical MAP close-out; however, the duration is consistent with the complexity of the issue.

OPI: ADM(Mat)/DGMEPM/DNPS 8 - NaMMS Program Manager

Target Date: April 2019

Back to Table of Contents

Annex B—Evaluation Methodology and Limitations

1.0 Methodology

1.1 Overview of Data Collection Methods

The evaluation of the MEP included the use of multiple lines of evidence and complementary research methods to strengthen the rigour and reliability of the assessment. The methodology used a consistent approach when collecting and analyzing data to help ensure the reliability of the evaluation findings and recommendations. Quantitative and qualitative data collection methods were used and included reviews of literature and program documents, access to MEP data, and key informant interviews. Following data collection and analysis, preliminary evaluation findings were presented to the key stakeholders. Discussions from these presentations helped to further refine and clarify the findings and recommendations that are presented in this report.

1.2 Details on Data Collection Methods

1.2.1 Literature and Program Document Review

A review of program documents was conducted in the initial phase of the evaluation to establish an understanding of the background and context of the DND/CAF MEP. These documents included:

• federal/departmental accountability documents;
• strategic and operational program documents (i.e. orders, directives, briefing notes);
• MSI documentation;
• website contents;
• guidance documents, process and procedure manuals;
• previous internal and external assessment reports;
• relevant academic literature and publications; and
• program products/outputs.

The document review was integral in the assessment of relevance of the Program, as well as to support performance findings from other lines of evidence.

1.2.2 Access to MEP Data

Full access to MEP data was made possible by allowing the evaluation team direct access to the following systems:

• DRMIS
• RDIMS
• Level of Effort
• SharePoint

Having direct access to the repositories reduced DGMEPM HR requirements to support the evaluation. It allowed for an in-depth analysis with regard to financial data, performance data, HR data, and guidance documents and processes.

1.2.3 Key Informant Interviews

A formal advisory group was established, which included representatives from the RCN (Director General Naval Force Development (DGNFD) and FMF) and ADM(Mat) (DGMEPM). Furthermore, consultations were conducted at different phases in the evaluation with key stakeholders, both at the L1 level between ADM(RS) and ADM(Mat) and at the L2 level between DG Evaluation and DGMEPM. These interviews were used to discuss the relevance and performance of the MEP and to gather evidence of any issues affecting the program. Interviews also provided context and elaboration of trends observed in the program data. Information gathered by ADM(RS) from the interviews was cross-referenced against documentation to assess performance.

2.0 Limitations

The following table shows the limitations and mitigation strategies:

Table B-1. Evaluation Limitations and Mitigation Strategies. List of the limitations of the evaluation and the corresponding mitigation strategy.

Back to Table of Contents

Annex C—Logic Model

Figure C-1. Logic Model for the Maritime Equipment Program. This flow chart shows the relationship between the Maritime Equipment Program’s main activities, outputs and expected outcomes.

Back to Table of Contents

Annex D—Evaluation Matrix

Table D-1. Evaluation Matrix—Relevance. This table indicates the data collection methods used to assess the evaluation issues/questions for determining the Maritime Equipment Program’s relevance.

Table D-2. Evaluation Matrix—Performance (Effectiveness). This table indicates the data collection methods used to assess the evaluation issues/questions for determining the Maritime Equipment Program’s performance in terms of achievement of outcomes (effectiveness).

Table D-3. Evaluation Matrix—Performance (Efficiency and Economy). This table indicates the data collection methods used to assess the evaluation issues/questions for determining the Maritime Equipment Program’s performance in terms of efficiency and economy.

Back to Table of Contents

___________________________________________________________________________________________________________________________

Footnote 1 North Atlantic Treaty Organization.

Footnote 2 Canadian Forces website. http://www.forces.gc.ca/en/about-org-structure/assistant-deputy-minister-materiel.page Consulted February 20, 2015.

Footnote 3 NaMMS Volume 1. Dated 2011.

Footnote 4 DND Performance Report 2014-15.

Footnote 5 ADM(Mat) L1 Business Plan 2013-2014.

Footnote 6 National Defence Program Alignment Architecture – 2014.

Footnote 7 New Capability Acquisition (Vote 5) will be covered in a future evaluation in accordance with the ADM(RS) five-year evaluation plan.

Footnote 8 Ibid.

Footnote 9 ADM(RS) conducted an Evaluation of Naval Forces in December 2013.

Footnote 10 ADM(RS) conducted an Audit of Warehouse Management, and the 2011Fall Report of the Auditor General of Canada included Chapter 5: Maintaining and Repairing Military Equipment – National Defence.

Footnote 11 ADM(RS) conducted an Evaluation of Naval Forces in December 2013.

Footnote 12 Population is defined as the physical number of people whatever time schedule they are on, while FTE is defined as the number of hours worked by one employee on a full-time basis.

Footnote 13 These figures are population, not FTEs, and exclude personnel in positions dedicated to capability acquisition.

Footnote 14 Human Resource Management System.

Footnote 15 TBS, Directive on the Evaluation Function, April 1, 2009. Consulted July 4, 2014. This was rescinded as of on July 1, 2016 and has been replaced with the TBS Directive on Results.

Footnote 16 Department of National Defence and the Canadian Armed Forces Defence Renewal Charter, October 2013.

Footnote 17 Canada First Defence Strategy.

Footnote 18 MEPM Strategic Plan 2012-2017.

Footnote 19 LEP and AEP.

Footnote 20 Commander R.C. Gray’s email “RE: Amendment (MEP EVAL) RFI” – dated January 21, 2015, and DRMIS records as of February 2015.

Footnote 21 MEPM Strategic Plan 2012-2017.

Footnote 22 Department of National Defence and the Canadian Armed Forces Defence Renewal Charter, October 2013.

Footnote 23 TBS Policy on Management of Materiel.

Footnote 24 As stated in the Defence Production Act, PSPC has the responsibility to acquire, utilize, store, transport, sell, exchange or dispose of defence supplies; and to manufacture/produce, finish, assemble, process, develop, repair, maintain or service defence supplies.

Footnote 25 Procurement Administration Manual, Revision 52, June 2013.

Footnote 26 Assistant Deputy Minister (Finance and Corporate Services) (ADM(Fin CS)) has cancelled DAOD 3004-0, DAOD 3004-1, and DAOD 3004-2 and now refers to the conditions set out in the Procurement Administration Manual.

Footnote 27 Equipment Management Team Handbook. Master Document, Vol. 2, November 24, 2004.

Footnote 28 GC Speech from the Throne, October 2013.

Footnote 29 Canada First Defence Strategy.

Footnote 30 Descriptors for GC Outcome Areas. http://www.tbs-sct.gc.ca/ppg-cpr/descript-eng.aspx

Footnote 31 Director General Maritime Equipment Program Management Strategic Plan 2012-2017.

Footnote 32 DND Report on Plans and Priorities 2014-2015.

Footnote 33 Commander’s Guidance and Direction to the RCN – Executive Plan 2013-2017.

Footnote 34 ADM(Mat) Strategic Priorities FY 2015/16, from DWAN. Accessed October 21, 2015.

Footnote 35 DGMEPM DWAN website. Accessed October 21, 2015.

Footnote 36 DGMEPM defines level of effort as “time, in hours and minutes, spent against discrete work activities. Recording the level of effort against specific activities will allow MEPM to better understand what type and amount of work is being performed by staff.” MEPM Level of Effort Users Guide, version August 20, 2015.

Footnote 37 Weekly average of the percentage of DGMEPM hours expended from October 20, 2014 to March 29, 2015, extracted utilizing DGMEPM Level of Effort tool.

Footnote 38 The 5F represents Force Management, Force Development, Force Generation, Force Support and Force Employment.

Footnote 39 Maritime Equipment Program Management (MEPM) Strategic Plan 2014-2019.

Footnote 40 Analysis contained in MSI update to MEPM strategic planning, December 15, 2014 RDIMS.

Footnote 41 DND Report on Plans and Priorities 2014-2015. http://www.forces.gc.ca/assets/FORCES_Internet/docs/en/DND-RPP-2014-15.pdf

Footnote 42 MEPM PAA Performance Measurement Framework FY 2014/15. Q4 Report, April 2015.

Footnote 43 National Defence Performance Report 2014-15. http://www.forces.gc.ca/en/about-reports-pubs-departmental-performance/2014-2015/section-ii-strategic-outcome-2.page

Footnote 44 MEPM Strategic Plan 2014-2019.

Footnote 45 Commander’s Guidance and Direction to the RCN – Executive Plan 2013 to 2017.

Footnote 46 MEPP Guidance for FY 2016-18.

Footnote 47 Data from DND performance reports 2011-2015.

Footnote 48 DGMEPM fourth quarter report – April 2015.

Footnote 49 MEPM 2014 second quarterly report – October 2014.

Footnote 50 NaMMS, Volume 1, dated May 18, 2011.

Footnote 51 MEPM Strategic Plan 2012-2017.

Footnote 52 The Change Management Process. http://www.educational-business-articles.com/change-management-process.html

Footnote 53 MEPM Strategic Plan 2012-2017.

Footnote 54 MMEPM Strategic Plan 2012-2017.

Footnote 55 MEPM Strategic Plan 2014-2019.

Footnote 56 MDIA.

Footnote 57 Chief of Staff MEPM and Director Naval Logistics.

Footnote 58 LEPM.

Footnote 59 ADM(RS) Evaluation of Land Equipment Program report, June 2015.

Footnote 60 Data provided by ADM(IM) DRMIS Directorate.

Footnote 61 DDRMIS.

Footnote 62 Briefing note to Director General Materiel Systems and Supply Chain from MASIS Project Director, dated October 29, 2013.

Footnote 63 Date Accessed: October 7, 2015.

Footnote 64 Disposal is defined as the removal of an asset (including stores).

Footnote 65 MEP Business Process Management Control Register. .

Footnote 66 MDWG.

Footnote 67 Date Accessed: October 7, 2015.

Footnote 68 Date Accessed: October 7, 2015.

Footnote 69 DAOD 3003-0 Controlled Goods, DAOD 3003-1 Management of Controlled Goods, DAOD 3013-0 Surplus Materiel, and 3013-1 Disposal of Surplus Materiel. Date Accessed: October 15, 2015.

Footnote 70 Date Accessed: October 15, 2015.

Footnote 71 Naval Materiel Management System Manual (NaMMS), Volume 1 Section 5.

Footnote 72 FMFCB DWAN website.

Footnote 73 FMFCS DWAN website.

Footnote 74 ADM(RS) Evaluation of Naval Forces 2013.

Footnote 75 Fleet Maintenance Facilities Integrated Management System Manual.

Footnote 76 Salary Wage Envelope.

Footnote 77 Maritime Equipment Program Planning Guidance (MEPP) FY 2016-18.

Footnote 78 Maritime Equipment Program Planning Guidance (MEPP) FY 2015-17 and FY 2016-18.

Footnote 79 DRIMS extraction provided by DGMEPM.

Footnote 80 R&O.

Footnote 81 Defence Terminology Bank.

Footnote 82 Interviews with FMF staff.

Footnote 83 DAOD 3000-0, dated April 4, 2000.

Footnote 84 Naval Materiel Management System Manual (NaMMS), November 1, 2013.

Footnote 85 DGMEPM Strategic Plan 2014-2019.

Footnote 86 Naval Materiel Risk Management Canadian Forces Technical Order, October 12, 2012.

Footnote 87 Data based upon MSI Change Program Requirements Register, version 23B.

Footnote 88 MSI Change Program Requirements Register, version 20.

Footnote 89 Inventory management website. Last accessed November 2, 2015.

Footnote 90 ADM(Mat) letter: DRP for centrally managed inventory in the CF/DND, June 18, 2009.

Footnote 91 D Mar P brief to DNCS/DNPS, December 17, 2014.

Footnote 92 D Mar P brief to DNCS/DNPS, December 17, 2014.

Footnote 93 Data provided by DSCO DRP tool, May 20, 2015.

Footnote 94 D Mar P brief to DNCS/DNPS, December 17, 2014.

Footnote 95 FY 2014/15 Initial allocation letter, dated April 22, 2013.

Footnote 96 FY 2013/14 Initial allocation letter, dated March 30, 2013.

Footnote 97 TRANREQ

Footnote 98 TRANREQs are numbered by calendar year and not by fiscal year.

Footnote 99 VISSC.

Footnote 100 DND Performance Management Framework FY 2014/15.

Footnote 101 DGMEPM Strategic Plan 2012-2017.

Footnote 102 DGMEPM organizational change, July 2, 2013.

Footnote 103 MWVA.

Footnote 104 TBS Guide to developing performance measurement strategies, Chapter 5.

Footnote 105 Draft MEPM competency-based management framework.

Footnote 106 MSI Project Charter Revision 2.

Footnote 107 DGMEPM Memorandum MEPM Strategic Human Resource Governance, December 2014.

Footnote 108 Draft MEPM competency-based management framework.

Footnote 109 Treasury Board of Canada Secretariat’s Policy on Evaluation, April 1, 2009. Consulted November 25, 2014.

Footnote 110 Expenditures calculated based upon 4 percent inflation.

Footnote 111 Service level agreement between ADM(Mat) and ADM(Hr-Civ) concerning the integration of civilian human resource planning programs and operational services. Dated December 9, 2013.

Footnote 112 Draft MEPM competency-based management framework.

Footnote 113 MSI Project Charter Revision 2.

Footnote 114 DGMEPM Memorandum MEPM Strategic Human Resource Governance, December 2014.

Footnote 115 Draft MEPM competency-based management framework.

Footnote 116 DGMEPM Human Resources Database tool. Accessed October 22, 2015.

Footnote 117 HR strategic board meeting record of decisions, dated October 2015.

Footnote 118 MEPM Level of Effort Users Guide, version August 20, 2015.

Footnote 119 Level of Effort Business Process Elements, version 5 RDIMS.

Footnote 120 DGMEPM third quarter report - January 2015.

Footnote 121 DND Performance Measurement Framework FY 2014/15 - Annex A.

Footnote 122 DND Performance Measurement Framework FY 2014/15 - Annex A.

Footnote 123 MEPM 2014 third and fourth quarter reports to ADM(Mat).