Evaluation of Global Engagement/Military Diplomacy
- Acronyms and Abbreviations
- Executive Summary
- 1.0 Introduction
- 2.0 Findings and Recommendations
- Annex A—Management Action Plan
- Annex B—Evaluation Methodology and Limitations
- Annex C—Logic Model
- Annex D—Evaluation Matrix
- Assistant Deputy Minister
- Assistant Deputy Minister (Finance)
- Assistant Deputy Minister (Information Management)
- Assistant Deputy Minister (Materiel)
- Assistant Deputy Minister (Policy)
- Assistant Deputy Minister (Review Services)
- Canadian Army
- Canadian Armed Forces
- Canadian Defence Attaché
- Canadian Defence Attaché Office
- Canadian Defence Liaison staff (London)
- Canadian Defence Liaison staff (Washington)
- Chief of the Defence Staff
- Canadian Forces Intelligence Command
- Canadian Joint Operations Command
- Deputy Canadian Defence Attaché
- DFATD Act
- Department of Foreign Affairs, Trade and Development Act
- Director of Foreign Liaison
- Director General, International Security Policy
- Deputy Minister
- Department of National Defence
- Departmental Results Framework
- Deputy Vice Chief of the Defence Staff
- Defence Wide Area Network
- Full time Equivalent
- Fiscal Year
- Global Affairs Canada
- Government of Canada
- Global Engagement Strategy
- Head of Mission
- Minister of National Defence
- MOD (UK)
- Ministry of Defence (United Kingdom)
- Memorandum of Understanding
- North Atlantic Treaty Organization
- Office of Collateral Interest
- Other Government Departments
- Official Hospitality Outside Canada
- Office of Primary Interest
- Privy Council Office
- Performance Measurement and Evaluation Committee
- Permanent Resident Mission in New York
- Royal Canadian Air Force
- Royal Canadian Navy
- Strategic Joint Staff
- Shared Services Canada
- Strong, Secure, Engaged (2017 Defence Policy)
- Treasury Board
- Treasury Board Secretariat
- Travel, Hospitality, Conference and Event Expenditures
- United States
- Vice Chief of the Defence Staff
- During the last five years, the need for the Military Diplomacy Program, particularly the CDA activities has increased – 11 new CDA Offices have been added recently.
- There is a need to conduct a systematic and periodic review of CDA Office strategy and approach in view of the defence priorities.
- Timely advice and information provided by CDAs have been described as very valuable by all Program-recipients.
- There is a need to review and revise the interdepartmental Memorandum of Understanding (MOU) signed with GAC to reflect CDA Office representation needs outside of Canada.
|1. There is evidence of ongoing need for Military Diplomacy activities. Recent Defence Policy (SSE, 2017) places substantial focus on Military Diplomacy and CDA’s role and contribution to diplomacy and global engagement.||See Recommendation 1|
|2. During the evaluation period, there was an increase in the number of CDAs. Recently, the number of CDA Offices was increased by 11.||See Recommendation 1|
|3. The Military Diplomacy Program aligns with federal roles and responsibilities. Although DND/CAF works in cooperation with OGDs in the conduct of Military Diplomacy, there is no evidence of a duplication of roles.||See Recommendation 1|
|4. The Military Diplomacy Program aligns with Government and Departmental priorities. However, the evaluation could not find any evidence whether the current locations of CDA Offices align with the recently renewed Defence Policy (SSE, 2017) and GES.||
1. Ensure that there is a systematic and periodic review of CDA Office strategy and approach in view of the defence priorities stipulated in the Defence Policy (SSE) and GES.
|Performance – Effectiveness|
|DND/CAF, GAC and Government of Canada (GC) Decision Makers are informed by current and up-to-date information on foreign defence and military affairs|
|5. Most Program-recipients expressed a high level of satisfaction with the quality and timely advice and information provided by CDAs.||See Recommendation 2|
|6. Some challenges were observed in receiving information from countries, where CDAs were accredited, due to the distances involved to the accredited countries and limited resources. However, it was underlined that having no CDA presence would make it extremely hard to secure a view of a specific region.||See Recommendation 2|
|7. Currently, CDA reports do not have a standard format. Program-recipients indicated having a standard format would increase the value of CDA reports.||
2. Review and standardize the CDA report formats, so that they have a common look and feel, and provide Program-recipients with timely and relevant information.
|8. Information and advice provided by CDAs add value for better decision making.||
3. Leverage CDAs in areas such as policy advice at an earlier point rather than at the back end, which is the current practice.
|9. A more coordinated effort in defence engagement at large is required, preferably in a negotiated and well thought out visit plan format for each country, particularly for key priority regions.||See Recommendation 4|
|10. Challenges continue in the area of global engagement involving a collective view, particularly prior to decisions that concerned interdepartmental collaborations.||
4. (4.1) Review and develop a system to track the number of global engagement visits the Department staff takes annually, including the number of people, frequency of visits, name of the organizations involved, and results achieved.
(4.2) It is recommended that the VCDS direct the establishment of a team to lead the review.
|Cooperation is enhanced between the DND/CAF and foreign defence and military forces through engagement|
|11. CDAs’ presence has been instrumental to HOM, particularly in those embassies where the CAF has ongoing missions.||-|
|12. Over the years, budget constraints have impacted the implementation of the GES. It is believed that, the revised version of the GES, particularly its Annexes, will provide constructive guidance to CAF leadership, when conducting defence engagement.||-|
|13. Based on interviews with Program-recipient stakeholders, CDA contributions have also been valuable in establishing and maintaining Defence industrial relations, despite the fact that it is not seen as a high-priority role for CDAs.||-|
|Performance – Efficiency|
|Demonstration of Efficiency in the Military Diplomacy Program|
|14. DND/CAF has not adopted the Official Hospitality Outside Canada (OHOC) policy for CDAs that was established by GAC.||
5. Re-examine the possibilities of adopting a policy similar to GAC’s OHOC policy on hospitality that would allow greater flexibility for CDAs while maintaining rigorous oversight and reporting.
|15. The requirement for suitable housing has been a long-standing issue for CDAs, following a GC decision that eliminated entitlements for official residences.||
6. Review and revise the interdepartmental MOU signed with GAC to reflect CDA Office representation needs outside of Canada.
|16. Currently there is no mechanism in DND/CAF to capitalize on the knowledge and experience of CDAs upon their return. Often, many CDAs retire after their posting, or are employed in unrelated CAF jobs upon their return.||
7. Review and assess the opportunities for the Defence Attaché employment modelling practices used in Allied countries.
|17. None of the CDAs have access to the DND computer network abroad. This has resulted in impediments to communication and dissemination of information and reports.||
8. Review and explore the option of using the appropriate network for sharing of CDA reports with DND/CAF, Five-Eyes and other Allies as required.
Table 1 Details - Summary of Key Findings and Recommendations
Key Finding 9: A more coordinated effort in defence engagement at large is required, preferably in a negotiated and well thought out visit plan format for each country, particularly for key priority regions.
|FY 2013/14||FY 2014/15||FY 2015/16||FY 2016/17||Variation since FY 2013/14|
|Transfers to GAC||$2,112,480||$3,886,534||$3,918,768||$7,046,702||234%|
|Transfers to SSC||$67,100||$59,502||$68,564||$152,875||128%|
Table 2 Details - DND transfer of funds to GAC and SSC from FY 2013/14 to FY 2016/17
|ELEMENTS||FY 2012/13||FY 2013/14||FY 2014/15||FY 2015/16||FY 2016/17||Variation since FY 2012/13|
|DFL expenditures other than travel and hospitality under 3180AA (for all CDAs)||$2,237,396||$1,115,219||$1,877,905||$1,243,080||$2,015,687||-10%|
|DFL 2 expenditures under 3180AB||$613,511||$572,707||$712,912||$632,763||$646,560||5%|
Table 3 Details - DFL2 and CDA office expenditure from FY 2012/13 to FY 2016/17
|TRAVEL EXPENDITURES||FY 2012/13||FY 2013/14||FY 2014/15||FY 2015/16||FY 2016/17||Variation 5 years|
|Total travel expenditures||$2,356,451||$1,857,059||$1,580,724||$1,569,460||$1,512,209||-36%|
|Variation according to previous year||-||-21%||-15%||-1%||-4%||-|
Table 4 Details - Travel expenditures for both the CDAOs and CDLS Washington, London and PRMNY from FY 2012/13 to FY 2016/17
|HOSPITALITY EXPENDITURES||FY 2012/13||FY 2013/14||FY 2014/15||FY 2015/16||FY 2016/17||Variation 5 years|
|Total travel expenditures||$438,284||$480,316||$537,351||$523,059||$470,917||7%|
|Variation according to previous year||-||10%||12%||-3%||-10%||-|
Table 5 Details - Hospitality expenditures for CDAOs and CDLS London, Washington and PRMNY from FY 2012/13 to FY 2016/17
Key Finding 15: The requirement for suitable housing has been an issue for CDAs, following a GC decision that eliminated entitlements for official residences.
|YEAR (Annual Posting Season)||End of Tour as CDA||Retired right after CDA Tour||Cross-posting to another CDAO||Employed in the CAF, but not as a CDA||Still in the CAF at Annual Posting Season 2017|
Table 6 Details - CDAs, retired and/or cross-posted to another CDAO, ore employed in the CAF in a different position
|Distinguishing the CDAs’ activities and their engagement with GAC from the entirety of DND/CAF’s Global Engagement Program.
Based on the recommendation from the July 2017 PMEC committee, the purpose of this evaluation was to assess CDA activities and engagement with GAC and effectiveness of those activities.
|The evaluation team was careful in the scoping phase of the evaluation to determine the outputs and outcomes of the CDAs’ activities and their engagement with GAC. Although there are many outputs and outcomes of DND/CAF’s Global Engagement/Military Diplomacy Program, the evaluation only assessed what was determined to be the most important aspects as requested by PMEC.|
|Number and diversity of internal and external stakeholders. The evaluation team found that the Program has many stakeholders and Program-recipients within DND/CAF and outside organizations. Due to short timelines it was difficult to set interviews with all Program-recipients involved.||The evaluation team emphasized the use of a questionnaire to obtain facts, evidence and opinions from CDAs and DCDAs. The return rate was close to 95 percent, giving credibility to the data. In addition, the team validated the information with DFL through interviews|
|Interview bias. Interviews access the subjective impressions of stakeholders and, as such, can lead to narrow, very wide, or potentially biased views.||Insights derived from interviews required corroboration from at least one other source, either objective data or agreement with other interviewees. Also, the questionnaire allowed the evaluation team to survey a much larger sample population (as compared to interviews) to allow for better generalizations about program performance. All interviews were conducted by three interviewers in order to confirm and enhance understanding and to minimize bias.|
|The use of DPRs and DRRs to determine expenditures at various levels was a challenge as the figures posted in annual reports were updated at multiple occasions in subsequent reports.||To ensure proper tracking by readers, fair comparisons between FYs and to keep trend analysis of performance data in the report is applicable to the identified FY.|
|Lack of centralized financial data covering the Program.||Existing data was leveraged to the extent possible. The evaluation team had to rely on all other available sources of information (interviews, questionnaires) as evidence to support the evaluation findings.|
|Compressed timing. The evaluation mandate was received in August and had to be completed from August 2017 (planning) to January 2018 (draft report) while an evaluation usually take 12 to 18 months to be delivered.||The evaluation team interviewed as many stakeholders as possible. The interview list continued to develop throughout the evaluation process and interviews were conducted as new stakeholders were identified.
All stakeholders were informed of the effect created by the compressed timeline and contributed to the study in accepting changing their schedule to accommodate the evaluation.
In hindsight, with the number of stakeholders involved, a questionnaire should have been conducted instead of individual interviews.
Table B-1 Details - Evaluation Limitations and Mitigation Strategies
Figure C-1 Details - Logic Model for Global Engagement / Military Diplomacy
|Evaluation Issues/Questions||Indicators||Literature & Document Review||Key Informant Interviews||CDAs Questionnaire||Comparative Analysis with Allies||Program and Financial Data Review|
|1. Is there a continuing and future need for the Global Engagement /Military Diplomacy Program?||1.1 Evidence of ongoing need for Global Engagement/ military diplomacy activities (importance and utility of CDA program to its users)||Yes||Yes||Yes||No||Yes|
|2. Does the Global Engagement/Military Diplomacy Program align with the federal roles and responsibilities?||2.1 Alignment of the Program with government acts and legislation||Yes||No||No||No||No|
|2.2 Extent of duplication of Military Diplomacy Program activities that are the responsibility of OGDs, agencies, or the private sector (role of GAC in comparison to DND/CAF)||Yes||Yes||No||No||No|
|2.3 Alignment between the Program and DND/CAF roles and responsibilities||Yes||No||No||No||No|
|2.4 Evidence of complimentary or duplicate activities||Yes||Yes||Yes||No||No|
|3. Does the Global Engagement/Military Diplomacy Program align with Government and DND/CAF priorities?||3.1 Alignment between the Program activities and DND/CAF priorities||Yes||No||No||No||Yes|
Table D-1 Details - Evaluation Matrix – Relevance
Table D-1 Summary: This table indicates the sources of evidence used to assess the evaluation issues and questions for determining the relevance of the Program. The study used three evaluation questions to determine the Program’s relevance these questions appear listed in the left-hand column, each under its own subheading. Select a question and read across that row to learn, first, the indicators used to assess the corresponding relevance question. Read across the following five columns to learn the sources of evidence used to evaluate the question. From columns three to five, the sources of evidence are literature and document review, key informant interviews, CDAs questionnaire, comparative analysis with Allies, and program and financial data review.
|Evaluation Matrix—Performance: Achievement of Expected Outcomes (Effectiveness)|
|Evaluation Issues/Questions||Indicators||Literature and Document Review||Key Informant Interviews||CDAs Questionnaire||Comparative Analysis with Allies||Program and Financial Data Review|
|4.1 To what extent are DND/CAF, GAC, and GC decision makers informed by current and up-to-date information on foreign defence and military affairs?||4.1.1 Evidence of up-to-date and timely information is received by stakeholders||Yes||Yes||Yes||No||No|
|4.1.2 Evidence of response to changes in world politics and Canadian priorities||Yes||Yes||Yes||Yes||No|
|4.1.3 Existence of a coordinated effort for high-level visits/collective view of DND/CAF||Yes||Yes||Yes||No||No|
|4.2 To what extent are cooperation and networking enhanced between the DND/CAF and foreign defence and military forces through engagement?||4.2.1 Extent of cooperation between DND/CAF and foreign defence and military forces||Yes||Yes||Yes||No||No|
|4.2.2 Extent of CDAs’ impact on cooperation and international missions||Yes||Yes||Yes||No||No|
|4.3 To what extent has DND/CAF an effective network based on its global engagement through CDAs?||4.3.1 CDAs’ impact on cooperation and international missions||Yes||Yes||Yes||No||No|
|4.3.2 Degree of engagement with international defence industry||Yes||Yes||Yes||No||No|
Table D-2 Details - Evaluation Matrix – Performance: Achievement of Expected Outcomes (Effectiveness)
|Evaluation Matrix—Performance: Demonstration of Efficiency|
|Evaluation Issues/ Questions||Indicators||Literature and Document Review||Key Informant Interview||CDAs Questionnaire||Comparative Analysis with Allies||Program and Financial Data Review|
|5.1 Is the Military Diplomacy Program delivering in an efficient manner?||5.1.1 Trends in expenditures||Yes||Yes||No||No||Yes|
|5.1.2 Impediments to global engagement activities||Yes||Yes||Yes||No||Yes|
|5.1.3 Efficiency of the employment model used for CDAs – comparison to allied countries||Yes||Yes||Yes||No||Yes|
|5.1.4 Possibilities for improvement in service delivery||Yes||Yes||Yes||Yes||Yes|
Table D-3 Details - Evaluation Matrix – Performance: Demonstration of Efficiency
Report a problem or mistake on this page
- Date modified: