Evaluation of the Military Training and Cooperation Program

February 2024

1258-3-063 (ADM(RS))

Reviewed by ADM(RS) in accordance with the Access to Information Act (AIA). Information withheld in accordance with the AIA under section 15(1).

Acronyms

Back to Table of Contents

Executive Summary

Evaluation Scope

The evaluation of the Military Training and Cooperation Program (MTCP) focused on relevancy in terms of the program’s alignment with Government of Canada (GC) priorities, program effectiveness concerning training deliverables and supporting diplomacy, and program efficiency from a governance and resourcing perspective. The period examined was between fiscal years (FY) 2018/19 and 2022/23. The evaluation is in accordance with the Department of National Defence (DND)/Canadian Armed Forces (CAF) Five-Year Departmental Evaluation Plan FY 2022/23 to 2026/27. 

Program Overview

The program was established in 1963 and supports DND’s global engagement goals by building partner capacity and advancing defence diplomacy. From FY 2018/19 to FY 2022/23, MTCP had an annual budget of approximately $14 million. It provided military training and education to over 60 non-NATO member countries in Africa, the Middle East, Europe, Asia and the Americas.

In 2023, the program’s funding was renewed at the same levels for traditional program activities. MTCP will also be used to support emerging capacity building needs and as a mechanism to provide military assistance. A recent example of this assistance is the $500 million announced in summer 2023 for military support to Ukraine.

Results

The program appears to be making positive contributions to Canada’s defence diplomacy and to the GC and DND/CAF’s global engagement objectives. More targeted strategic direction on the program’s scope and prioritization of member countries would help optimize program capacity. 

While the program continues to enhance the skills and knowledge of participants in alignment with its training areas, there are opportunities to improve data collection and management practices to better demonstrate the impact it is having on military capacity in member countries. Better measurement mechanisms are also required to show the impact on bilateral and multilateral defence relations.

Finally, the program struggled to expend funds during the pandemic and had high administrative costs. Spending appears to be back on track based on FY 2023/24 planning. Based on currently projected funding levels, the program seems to be adequately prepared to deliver activities under the new enhanced military assistance and capacity building pillars. Improved coordination with key stakeholders could result in program efficiencies and augment the program’s ability to advance areas of continued and evolving strategic interest to Canada. 

Overall Conclusion

Overall, the program remains relevant in the context of an evolving global landscape of international relations, which requires ongoing and critical diplomatic engagement and aligns with key governmental priorities such as Women, Peace and Security (WPS); democratic values; and professional military standards. Improvement opportunities have been identified regarding the program’s strategic direction, its performance measurement framework and its coordination efforts with key stakeholders—which would enhance program efficiency and position DND/CAF to demonstrate the impact of the program.

Back to Table of Contents

1.0 Introduction

1.1 Evaluation Context

Program Context

MTCP supports the department’s global engagement objectives by building partner capacity and advancing defence diplomacy. Although the program’s annual budget of approximately $14 million represented only about 4 percent of the Global Engagement Strategy program expenditures, MTCP sponsored training for about 5,500 participants—representing 62 member countries—during the evaluation period. Training was delivered across four activity areas: language training; professional development and staff training; peace support operations training; and expert and delegation visits. There are six different course types offered under these four areas. MTCP expenditures on training by course type are illustrated in the following graph. Courses and training were provided by multiple partners, including the Military Personnel Generation Training Group, the Canadian Defence Academy, the Canadian Army, Calian Group Limited, foreign academies and schools, and by other CAF and DND personnel. Courses varied in length from several days to 10 months.

Figure 1 Summary

In 2023, two additional activity areas were added to MTCP: Defence Capacity Building and Enhanced Military Assistance. These areas, however, were not funded during the evaluation period.

The program was previously evaluated in 2019, covering the FY 2013/14 to FY 2017/18 period. Four recommendations were raised, which focused on the terms and conditions of the program; alignment with the capacity building initiative under Canada’s defence policy: Strong, Secure, Engaged (SSE); staffing strategy; and a comprehensive performance measurement strategy, including data collection and management. There are no open Management Action Plans (MAP) from this evaluation.

Methodology

The evaluation of MTCP employed several methods, including key informant interviews, surveys, international comparative analysis, document review, case studies and data analysis (see Annex C for more details). Seven groups of stakeholders were consulted to inform the findings of this evaluation. The following questions guided the evaluation:

Evaluation Questions

Relevance

Q1. To what extent is MTCP relevant to the objectives and priorities of the GC?

Effectiveness

Q2. To what extent has MTCP contributed to improving military trainees’ knowledge and abilities and the extent to which they can undertake activities including peace support operations and other military operations?

Q3. To what extent has MTCP contributed to Canada’s bilateral and multilateral defence relations?

Efficiency

Q4. To what extent is MTCP efficiently coordinated, resourced and governed?

Back to Table of Contents

2.0 Findings

2.1 Relevance

Why It Matters

Defence diplomacy is a critical tool for Canada, and MTCP plays a vital role in advancing this mandate. Strategic guidance is important to provide a common, unifying aim to further facilitate cooperation and the achievement of national and departmental objectives.

What We Found

MTCP supports the GC and DND's global engagement goals of building partner capacity and advancing defence diplomacy. 

Stakeholders and partners indicated the relevance and importance of MTCP. A survey of Canadian Defence Attachés (CDA) indicated that 72 percent agreed that MTCP activities are aligned with the priorities of the GC. MTCP training and programming directly incorporate many important GC priorities including WPS, democratic values and professional military standards. MTCP's Integration of Women and Gender Perspectives in the Armed Forces course contributes to Canada's WPS Agenda, Canada's Feminist International Policy and United Nations Security Council Resolution 1325 (2000).

Defence diplomacy is a priority for DND/CAF. SSE states that, "a military cannot be engaged in the world unless it is present in the world, and this includes building and sustaining strong relationships with allies, partners, other militaries and multilateral institutions." MTCP is one of the principal mechanisms for defence capacity building and engagement with Canada's non-NATO allies as outlined in the Guidance on International Priorities for Defence Engagement.

MTCP is responsible for a significant portion of DND/CAF's diplomacy efforts based on document review. A majority of CDAs (67 percent) agreed that MTCP is the primary mechanism through which DND/CAF engages with member countries, and 83 percent agreed that MTCP has contributed to Canada's presence in the regions of member countries.

However, DND's overarching global engagement strategy is not up-to-date and does not provide adequate direction to MTCP. As a result, the program broadly targets over 60 countries in different parts of the world. The following table illustrates the wide range of countries that MTCP targets. MTCP could contribute in a more focused and strategic way if there were clear strategic and policy direction. 

 With the addition of the new capacity building and enhanced military assistance activity areas as well as the injection of increased funding, MTCP will be required to expand its programming further, including support for capacity building in member countries. This expansion would benefit from increased information sharing and coordination to clarify capacity building roles and responsibilities for MTCP and the broader CAF community.

2.2 Effectiveness

Why It Matters

An effective performance measurement framework supported by reliable data and tracking mechanisms is key to understanding whether training is achieving its intended outcomes for both trainees and the overarching military capacity of member countries.

What We Found

From FY 2018/19 to FY 2022/23, MTCP sponsored training in and outside Canada for about 5,500 participants, representing 62 member countries. Training was offered in the following areas: language training, professional development, staff officer courses, peace support operations training, trade training and exercise planning. MTCP leverages CAF capabilities for delivering training such as skills training, including combat medical, naval and air force skills. The majority (97 percent) of participants surveyed for this evaluation agreed or strongly agreed that that they feel equipped to apply the knowledge learned in the MTCP-sponsored courses to their work.

The evaluation was able to use basic tombstone data available from the nominal roll of participants; however, data that would assess the overall achievement of program objectives was not consistent or tracked by the program. Data that demonstrates the application of MTCP training by participants is very limited and even non-existent in most cases. Courses are delivered by partners external to MTCP (e.g., Calian, the Caribbean Military Academy and the Canadian Forces Language School). Course results are not systematically forwarded by those partners. While the program uses surveys of CDAs, instructors and sponsored students for its annual assessments, these surveys were not regularly conducted nor used to inform decision making.

The evaluation found positive evidence of MTCP's impact on military capacity in member countries. For example, Directorate Military Training and Cooperation (DMTC) annual reports indicated that MTCP training was a contributing factor to a decrease in Tunisian Armed Forces casualty rates during the first hour of injury. This result was noted during the first year of MTCP's combat first aid training (see Annex D for more details on the case studies).

The program also seems to be having some success when it comes to the participation rate of women in MTCP courses. The nominal roll shows that up to 22 percent of participants on all MTCP courses during the evaluation period were women. Interviews also identified efforts by the program to increase the participation of women on courses overall, despite limitations due to the inherently lower proportion of women serving in the armed forces of member countries.

Why It Matters

Complete and consistent performance data allows the measurement of the program's impact on Canada's bilateral and multilateral defence relations. This will become more important with new responsibilities related to capacity building and enhanced military assistance, as well as their associated additional funding.

What We Found

Of the CDAs surveyed for this evaluation, 72 percent agreed that MTCP has strengthened defence relations with member countries, and 83 percent agreed that the program has contributed to Canada's regional presence.

Figure 2 Summary

Key informant interviewees noted that MTCP has made meaningful contributions to Canada's defence relationships. Program officials frequently cited MTCP as "opening doors" for diplomatic engagement and as being leveraged as an entry point for further GC engagement. 

"MTCP is absolutely leveraged to further engagement in the Indo-Pacific. It is one of the cornerstones of most of our relationships in the Indo-Pacific, in Southeast Asia in particular."

- DND/CAF Stakeholder

Case studies for this evaluation provided further evidence of the program's ability to reinforce defence relations; for example, in Jamaica, several years of MTCP activities have led to the development and sustainment of Centers of Excellence. In Tunisia, military staff talks held in 2017 and 2018 identified gaps in capacity and areas of support for MTCP to address, thus strengthening Canada–Tunisia bilateral defence relations. 

Ukrainian relations have been greatly enhanced through MTCP training including Exercise MAPLE ARCH, which is an annual series of events that MTCP coordinates. Between FY 2018/19 and FY 2021/22, over 600 Ukrainians were sponsored for training and courses by MTCP, including at least 43 activities in eight countries. This training has made an indirect contribution to Canada and Ukraine's growing defence relationship, which has been further enhanced with new military assistance activities and funding.

The program's performance indicators related to defence relations are mostly output-related and do not adequately capture the effectiveness or quality over time. An example of an indirect indicator is whether Foreign Affairs judges that the overall bilateral relationship is improving.

2.3 Efficiency

Why It Matters

Efficient management of program administrative costs allows for more funding to be dedicated to the primary objectives while demonstrating good stewardship of public funds.

What We Found

Historically, administrative costs for MTCP have been high. During the evaluation period, MTCP had a high ratio of administrative costs relative to program expenditures with an average of 38 percent between FY 2018/19 and FY 2019/20, and higher during the pandemic. An example of a grants and contributions program overhead ratio is 9.3 percent for the Innovation Solutions Program, which only included personnel salaries and the Employee Benefits Program over total program funding, and it excluded HST. Under the funding and activities starting in FY 2023/24, it is anticipated that the program's overhead ratio will be reduced because of the injection of capacity building and military assistance funding. 

Why It Matters

Coordination of MTCP activities with OGDs and with similar programs offered by allies is important to avoid costly overlap and unnecessary training, and to maximize concerted efforts when the program and DND/CAF have limited resources.

Back to Table of Contents

3.0 Conclusion & Annexes

Overall Conclusion

The evaluation found that MTCP made a positive contribution to Canada's defence diplomacy. Over five years, the program trained approximately 5,500 participants from 62 countries. This was achieved with an annual budget of $13.89 million, representing about 4 percent of Global Engagement Program expenditures. It contributed to GC and DND/CAF objectives, including those related to SSE and the WPS agenda. More targeted strategic direction on the program's scope, particularly given the program's recent expansion, could provide clarity on the program's capacity building roles and responsibilities, as well as greater focus for all MTCP activities.

While program data and tracking remained limited or lacking to fully assess the extent of MTCP's impact and achievement of its objectives, the evaluation was able to assess some impact based on limited nominal roll data and surveys conducted for this evaluation. Improved data collection and management practices could enable the program to articulate to what extent it is successful in achieving its desired outcomes, thereby enabling the program to better direct its resources. 

The COVID-19 pandemic had a significant impact on the program's planned activities and its ability to fully disburse its funding. Since emerging from the pandemic, MTCP has consistently increased its number of activities, and preliminary planning indicates that the program is back on track to complete its planned activities and to fully expend its budget in FY 2023/24. Coordination with stakeholders and allies was varied and could be increased, especially with Canada's major international allies. This could grow in importance as the program takes on new capacity building and military assistance activities. 

This is a time of significant change for MTCP as it expands from its traditional concept and structure to one that includes capacity building and enhanced military assistance. This expansion will augment its ability to achieve its objectives with key partners in areas of both continuing and evolving strategic interest to Canada.

Annex A: Findings and Recommendations

Annex B: Management Action Plan

Annex C: Methodology

The findings and recommendations of this report were informed by multiple lines of evidence collected throughout the conduct phase of the project. These lines of evidence were triangulated with each other and verified with program officials to ensure their validity. The research methodology used in the scoping and conduct of the project was as follows:

Annex D: Case Studies

Annex E: MTCP Membership List (2022–2023 Annual Report)