Evaluation of Global Engagement/Military Diplomacy

Acronyms and Abbreviations
Executive Summary
1.0 Introduction
2.0 Findings and Recommendations
Annex A—Management Action Plan
Annex B—Evaluation Methodology and Limitations
Annex C—Logic Model
Annex D—Evaluation Matrix

Alternate Formats

PDF Version
(433 KB, 54 pages)

Assistant Deputy Minister (Review Services)

ADM: Assistant Deputy Minister
ADM(Fin): Assistant Deputy Minister (Finance)
ADM(IM): Assistant Deputy Minister (Information Management)
ADM(Mat): Assistant Deputy Minister (Materiel)
ADM(Pol): Assistant Deputy Minister (Policy)
ADM(RS): Assistant Deputy Minister (Review Services)
CA: Canadian Army
CAF: Canadian Armed Forces
CDA: Canadian Defence Attaché
CDAO: Canadian Defence Attaché Office
CDLS(L): Canadian Defence Liaison staff (London)
CDLS(W): Canadian Defence Liaison staff (Washington)
CDS: Chief of the Defence Staff
CFINTCOM: Canadian Forces Intelligence Command
CJOC: Canadian Joint Operations Command
DCDA: Deputy Canadian Defence Attaché
DFATD Act: Department of Foreign Affairs, Trade and Development Act
DFL: Director of Foreign Liaison
DGISP: Director General, International Security Policy
DM: Deputy Minister
DND: Department of National Defence
DRF: Departmental Results Framework
DVCDS: Deputy Vice Chief of the Defence Staff
DWAN: Defence Wide Area Network
FTE: Full time Equivalent
FY: Fiscal Year

GAC: Global Affairs Canada
GC: Government of Canada
GES: Global Engagement Strategy
HOM: Head of Mission
MND: Minister of National Defence
MOD (UK): Ministry of Defence (United Kingdom)
MOU: Memorandum of Understanding
NATO: North Atlantic Treaty Organization
OCI: Office of Collateral Interest
OGD: Other Government Departments
OHOC: Official Hospitality Outside Canada
OPI: Office of Primary Interest
PCO: Privy Council Office
PMEC: Performance Measurement and Evaluation Committee
PRMNY: Permanent Resident Mission in New York
RCAF: Royal Canadian Air Force
RCN: Royal Canadian Navy
SJS: Strategic Joint Staff
SSC: Shared Services Canada
SSE: Strong, Secure, Engaged (2017 Defence Policy)
TB: Treasury Board
TBS: Treasury Board Secretariat
THCEE: Travel, Hospitality, Conference and Event Expenditures
US: United States
VCDS: Vice Chief of the Defence Staff

Overall Assessment

During the last five years, the need for the Military Diplomacy Program, particularly the CDA activities has increased – 11 new CDA Offices have been added recently.
There is a need to conduct a systematic and periodic review of CDA Office strategy and approach in view of the defence priorities.
Timely advice and information provided by CDAs have been described as very valuable by all Program-recipients.
There is a need to review and revise the interdepartmental Memorandum of Understanding (MOU) signed with GAC to reflect CDA Office representation needs outside of Canada.

Program Description

The Military Diplomacy component of the Global Engagement Program falls under the authority of the Vice Chief of the Defence Staff (VCDS). It represents the contribution of the CDAs reporting to the Director of Foreign Liaison (DFL); the Canadian Defence Liaison Staff (London) (CDLS(L)) and the Canadian Defence Liaison Staff (Washington) (CDLS(W)), reporting to VCDS; the Permanent Resident Mission in New York (PRMNY) reporting to Assistant Deputy Minister (Policy) (ADM(Pol)); and various Liaison/Exchange Officers. Although CDLS(L) and CDLS(W) have many similar functions to CDAs, these staff are not organizationally part of the CDA Program. The DFL’s mission is to ensure the coordination and delivery of the CDA Program. DFL also coordinates Departmental interactions with Foreign Service Attachés and plans and coordinates activities to support visits to DND by foreign officials. It should be noted that this evaluation did not include the Foreign Service Attachés and Liaison/Exchange Officers’ activities.

CDAs are responsible to the Chief of the Defence Staff (CDS) through DFL for the provision of military advice and appropriate support and assistance to the Canadian Ambassador/High Commissioner or designated Head of Mission (HOM). Specifically, CDAs provide advice and support to achieve greater interoperability and enhanced delivery of Defence capabilities for on-going, contingency and potential operations. CDAs collect, analyze and provide intelligence to support strategic and operational decision making. Recently, Deputy CDAs have been appointed to the CDA Offices (CDAO) to assist and/or take on CDA duties.

Relevance

Canada’s recent Defence Policy: Strong, Secure, Engaged (SSE, 2017) places substantial focus on Global Engagement, underlining that Military Diplomacy is critical to its success. According to SSE, Defence representation at Canada’s diplomatic missions abroad enhances local engagement and provides support to Canadian foreign policy and trade priorities.

Almost all of the interviewed Program-recipients believed that without the CDA program, there would be a serious impact on missions abroad. Clients of CDAs would lose insight, information and reach into critical regions.

During the evaluation study period, there was an increase in the number of both CDAs and CDAOs. During the last three years, 11 new CDAOs were opened, increasing the total number of offices to 41 with 48 additional personnel.^{Footnote 1}

The Program aligns with Government and Departmental priorities. However, the evaluation could not find any supporting evidence to show that the current locations of CDAOs align with the recently renewed Defence Policy and Global Engagement Strategy (GES). Going forward, the strategic presence of CDAs should reflect the requirements stipulated in the SSE and therefore, align with the regions/countries identified in the GES by a systematic and strategic review of CDA office locations.

The Military Diplomacy program aligns with federal roles and responsibilities. Although DND/CAF works in cooperation with Other Government Departments (OGDs) in the conduct of Military Diplomacy activities, there is no evidence of any duplication of roles.

Performance

Effectiveness

Most Program-recipients view CDAs as the “Department’s eyes on the ground.” CDAs not only provided regular reports on country-specific issues, but also delivered strategic level advice to senior leadership. CDAs have also been the primary interlocutors between the operational groups and strategic level leadership. In addition, nearly all Program-recipients interviewed underlined that military-to-military relationships have been more favoured, particularly in some regions of the world, enabling access to resources and a better flow of information.

Program-recipients agreed that CDA reports, particularly if they contained analysis and comments, have been very valuable. However, recipients added that having a standard template format, including industrial activities and statistical trends and analysis of the specific regions involved, would increase the value of these reports.

Most CDAs stated that their advice was considered, and influenced decision-making. However, they also commented that they were often consulted at the last minute or not consulted at all prior to decision making involving policy or process changes, leading to occasional lost opportunities with the host countries. In addition, Program-recipients who were interviewed mentioned that CDAs were underused and that their capacity and experience could be leveraged more often and at the onset of deliberations in areas such as policy advice. In this regard, it was noted that consultations with CDAs should start near the beginning stages of any international negotiation or policy development process rather than the current practice of at the back end of products and decisions.

The evaluation team found challenges with the coordination of high-level visits. The Department still lacked a coordinated visit plan in many cases, which led to some confusion and, at times, inconsistency in the information and advice provided through Defence engagement. This problem was even more evident when a country/region suddenly became a priority and increased travel to the region overwhelmed the mission. Going forward, there is an opportunity for better tracking/recording of the number of visits.

Similarly, the evaluation team found that there were challenges with the consistency of view in the area of global engagement, particularly prior to decisions that concerned interdepartmental collaborations. These challenges varied from changing Government of Canada (GC) priorities, to a lack of a clear and overarching global engagement policy, to competing interests and communication problems. Going forward, the Ministry of Defence (United Kingdom) (MOD (UK)) model for international engagement was suggested as a role model to emulate. According to this model, a Deputy Vice Chief of the Defence Staff (DVCDS) position was created with the primary role of managing MOD engagement around the world. The UK’s North Atlantic Treaty Organization (NATO) staff was also placed under this deputy to ensure consistency. Furthermore, the DVCDS is required to communicate with CDAs and provide support in delivering a collective view.

CDAs also played a critical role as representatives of the Canadian Defence industry in host nations. It was underlined that having a CDA on the ground facilitated the establishment and maintenance of industrial relations with the host and accredited nations.

Efficiency

The new Defence Policy has placed a substantial level of emphasis on the CDA’s role in global engagement. However, based on feedback from CDAs, challenges continue in staff capacity and the appropriate flexibility required in the area of hospitality budget authority. Based on a review of documents of current and previous Treasury Board (TB) policies, it was found that in 2013, DND/CAF briefly considered, but did not adopt a hospitality policy similar to the Official Hospitality Outside Canada (OHOC) policy that was established by GAC. Based on this policy, GAC and the Heads of Post are exempt from Appendix B - Standards, of the TB Directive on Travel, Hospitality, Conference and Event Expenditures. In this respect, both the CDAs and Program-recipients interviewed voiced their concerns underlining that more flexibility and funding are required to administer hospitality activities. Program-recipients provided multiple examples where “lack of authority over hospitality” has strained diplomatic relationships. The region of Asia-Pacific was provided as an example of a culture of reciprocity, where DND/CAF had difficulties due to hospitality authority restrictions. Going forward, re-examining the establishment of a hospitality policy for CDAs, similar to the GAC OHOC policy is recommended.

GAC is mandated to manage the procurement of goods and real property in support of Diplomatic and Consular Missions. Based on evidence from interviews with key stakeholders, an ongoing issue has been the housing entitlement for CDAs. Previously, there was a GAC decision that eliminated entitlements for official residences. Later on, this directive became a GC policy. Although CDAs were not affected by this policy, as per the MOU signed between the two departments, it was noted that this disparity of policy created problems for GAC officials at missions unaware of the DND exception. Going forward, reviewing and revising the interdepartmental MOU signed with GAC to reflect CDA Office representation needs outside of Canada is recommended.

The evaluation team reviewed the employment process of CDAs and noted that currently there is no mechanism in place to capitalize on the knowledge and experience gained by CDAs upon their subsequent posting. Many CDAs either retire after their foreign tour, or are employed in unrelated jobs upon their return. It was noted that some allied countries such as the UK and US have recently introduced different selection and retention models to capitalize on the knowledge and experience of Defence Attachés. In general, both models streamline and select well-rounded individuals at the Major rank for a career as a Defence Attaché. There is opportunity to improve the current employment model of CDAs by further exploring approaches used by Allied countries.

None of the CDAs have access to the DND computer network and therefore impediments exist with communication and the dissemination of information and reports. It was noted that, due to the lack of connectivity on a secure universal network, most reports were only disseminated to a limited number of stakeholders in DND/CAF despite being in high demand, and were not easily shared with NATO or other Allies. Reviewing and exploring the possibility of using a secure network for sharing CDA reports with the Five-Eyes and other allies as required would improve Program results.

Key Findings and Recommendations

Table 1. Summary of Key Findings and Recommendations. This table provides a consolidation of report findings and recommendations.
Key Findings	Recommendations
Relevance
1. There is evidence of ongoing need for Military Diplomacy activities. Recent Defence Policy (SSE, 2017) places substantial focus on Military Diplomacy and CDA’s role and contribution to diplomacy and global engagement.	See Recommendation 1
2. During the evaluation period, there was an increase in the number of CDAs. Recently, the number of CDA Offices was increased by 11.	See Recommendation 1
3. The Military Diplomacy Program aligns with federal roles and responsibilities. Although DND/CAF works in cooperation with OGDs in the conduct of Military Diplomacy, there is no evidence of a duplication of roles.	See Recommendation 1
4. The Military Diplomacy Program aligns with Government and Departmental priorities. However, the evaluation could not find any evidence whether the current locations of CDA Offices align with the recently renewed Defence Policy (SSE, 2017) and GES.	1. Ensure that there is a systematic and periodic review of CDA Office strategy and approach in view of the defence priorities stipulated in the Defence Policy (SSE) and GES. OPI: DFL/VCDS OCI: ADM(Pol)
Performance – Effectiveness
DND/CAF, GAC and Government of Canada (GC) Decision Makers are informed by current and up-to-date information on foreign defence and military affairs
5. Most Program-recipients expressed a high level of satisfaction with the quality and timely advice and information provided by CDAs.	See Recommendation 2
6. Some challenges were observed in receiving information from countries, where CDAs were accredited, due to the distances involved to the accredited countries and limited resources. However, it was underlined that having no CDA presence would make it extremely hard to secure a view of a specific region.	See Recommendation 2
7. Currently, CDA reports do not have a standard format. Program-recipients indicated having a standard format would increase the value of CDA reports.	2. Review and standardize the CDA report formats, so that they have a common look and feel, and provide Program-recipients with timely and relevant information. OPI: DFL
8. Information and advice provided by CDAs add value for better decision making.	3. Leverage CDAs in areas such as policy advice at an earlier point rather than at the back end, which is the current practice. OPI: ADM(Pol) OCI: VCDS
9. A more coordinated effort in defence engagement at large is required, preferably in a negotiated and well thought out visit plan format for each country, particularly for key priority regions.	See Recommendation 4
10. Challenges continue in the area of global engagement involving a collective view, particularly prior to decisions that concerned interdepartmental collaborations.	4. (4.1) Review and develop a system to track the number of global engagement visits the Department staff takes annually, including the number of people, frequency of visits, name of the organizations involved, and results achieved. (4.2) It is recommended that the VCDS direct the establishment of a team to lead the review. OPI: VCDS OCI: SJS/DFL
Cooperation is enhanced between the DND/CAF and foreign defence and military forces through engagement
11. CDAs’ presence has been instrumental to HOM, particularly in those embassies where the CAF has ongoing missions.	-
12. Over the years, budget constraints have impacted the implementation of the GES. It is believed that, the revised version of the GES, particularly its Annexes, will provide constructive guidance to CAF leadership, when conducting defence engagement.	-
13. Based on interviews with Program-recipient stakeholders, CDA contributions have also been valuable in establishing and maintaining Defence industrial relations, despite the fact that it is not seen as a high-priority role for CDAs.	-
Performance – Efficiency
Demonstration of Efficiency in the Military Diplomacy Program
14. DND/CAF has not adopted the Official Hospitality Outside Canada (OHOC) policy for CDAs that was established by GAC.	5. Re-examine the possibilities of adopting a policy similar to GAC’s OHOC policy on hospitality that would allow greater flexibility for CDAs while maintaining rigorous oversight and reporting. OPI: DFL/VCDS OCI: ADM(Fin)
15. The requirement for suitable housing has been a long-standing issue for CDAs, following a GC decision that eliminated entitlements for official residences.	6. Review and revise the interdepartmental MOU signed with GAC to reflect CDA Office representation needs outside of Canada. OPI: DFL/VCDS OCI: ADM(Fin)/JAG
16. Currently there is no mechanism in DND/CAF to capitalize on the knowledge and experience of CDAs upon their return. Often, many CDAs retire after their posting, or are employed in unrelated CAF jobs upon their return.	7. Review and assess the opportunities for the Defence Attaché employment modelling practices used in Allied countries. OPI: VCDS/CMP
17. None of the CDAs have access to the DND computer network abroad. This has resulted in impediments to communication and dissemination of information and reports.	8. Review and explore the option of using the appropriate network for sharing of CDA reports with DND/CAF, Five-Eyes and other Allies as required. OPI: DFL OCI: Assistant Deputy Minister (Information Management) (ADM(IM))

Table 1 Details - Summary of Key Findings and Recommendations

Back to Table of Contents

1.0 Introduction

1.1 Context for the Evaluation

This report presents the findings and recommendations of the evaluation of the Military Diplomacy activities within the Global Engagement Program. The focus of the evaluation was on the CDA activities and collaboration with GAC as well as the effectiveness of these activities. The evaluation study also examined the coordination of high-level visits within the Department. The evaluation timeframe included the FY 2012/13 to FY 2016/17.

The evaluation study was conducted by ADM(RS) from September 2017 to December 2017, as a component of the DND/CAF Evaluation Plan, approved at the July 2017 PMEC meeting.

An evaluation of the Defence Policy and Diplomacy was conducted and published in 2013 by ADM(RS). It was preceded by other evaluations that covered some of the sub-elements of this evaluation. These are: Evaluation of the NATO Contribution Program (2017); Evaluation of the Defence Engagement Program Grant Program (2016); Evaluation of Security and Defence Forum, Class Grant Program (2010); Summative Evaluation of the Contribution Agreement with Military Training Assistance Program (2009); and Evaluation of Canadian Forces Outside Canada (OUTCAN) Exchange and Liaison Programs (2006).

Although there were no audits of the program conducted by ADM(RS), an Audit of “International Platform Service Delivery to Canada's Network Abroad (2014)” was conducted by GAC.^{Footnote 2}

1.2 Program Profile

1.2.1 Program Description

Background

Military Diplomacy, a component of the Global Engagement Program, falls under the authority of the VCDS. It represents the contribution of the CDAs reporting to DFL; CDLS(L) and CDLS(W) reporting to VCDS; PRMNY reporting to ADM(Pol); and various Liaison/Exchange Officers. Although the CDLS(L) and CDLS(W) have many similar functions to CDAs, these staff are not organizationally part of the CDA Program.

The DFL’s mission is to ensure the delivery of the CDA Program in support of the defence of Canada and Canadian interests abroad. It should be noted that this evaluation focused only on the activities of the CDAs and current agreements and collaborations with GAC.

DFL coordinates, develops and implements policies and procedures governing the activities of the CDAs abroad. DFL also coordinates the Departmental interactions with Foreign Service Attachés, and plans and coordinates activities to support the visits of foreign officials to DND.

The Canadian Ambassador/High Commissioner or designated HOM^{Footnote 3} appointed for the Diplomatic Mission in a country, exercises oversight and control of the CDA and CDAO staff in relation to the execution of Canada’s Diplomatic Mission. All members of the CDAO are members of Canada’s Diplomatic Mission and act under the guidance of the Canadian HOM in all matters except for those of a specifically DND/CAF nature.

During the last three years, demand for CDAs has increased. At the same time, the CDAO number has increased by 11 new CDAOs. In addition, select CDAOs were reinforced with Deputy CDAs as directed by the Minister of National Defence (MND) and CDS. It was noted that the number of deployed military increased by 45 percent during the same period.^{Footnote 4} The expansions of the CDAOs have been specifically in Eastern Europe, the Middle East and Africa. For example, Iraq and Latvia are two of the new CDA positions, created in 2017. Currently there are 41 CDAOs in 138 accredited countries, including CDLS(L)/CDLS(W) and PRMNY.

Information gathered by CDAs assists many organizations within the Department, including ADM(Pol), Strategic Joint Staff (SJS), Canadian Joint Operations Command (CJOC), Environmental Commands (Royal Canadian Navy (RCN), Canadian Army (CA), Royal Canadian Air Force (RCAF), Assistant Deputy Minister (Materiel) (ADM(Mat)), and Canadian Forces Intelligence Command (CFINTCOM). Information collected also contributes to the external affairs policy, coordinated by GAC and the Privy Council Office (PCO).

1.2.2 Program Objectives

As the designated point of contact between the Ministry of Defence and Armed Forces of the accredited country(ies), the CDA is responsible to the CDS through DFL for the provision of military advice and appropriate support and assistance to the Canadian Ambassador/High Commissioner or designated HOM. CDAs collect, analyze and provide defence intelligence to support strategic and operational decision making. As stipulated in the CDS Directive Force Posture and Readiness document (2013), the CDAs are capitalized to provide feedback and assessment on engagement activities conducted in their respective regions.^{Footnote 5}

The program activities are performed under the guidance of the GES. The strategy was first introduced in 2010 as a broad-reaching initiative. It is currently being revised by ADM(Pol) to deliver a more region specific approach and reflect updated government priorities.

1.2.3 Stakeholders

The primary stakeholders involved in the delivery of this program are DFL from the VCDS group and the CDAOs abroad.

The Program-recipients, who rely upon it for generating knowledge and information and establishing international relationships are: ADM(Pol), in particular, the International Security Policy Division (DGISP); Strategic Joint Staff (SJS); CFINTCOM. Other Program-recipients are the three Environments (RCN, CA, RCAF), ADM(Mat) as well as the MND, the Deputy Minister (DM) and the CDS Staff and other DND/CAF senior officers and officials.

External Program-recipients include primarily GAC; the Prime Minister’s Office; PCO; Immigration, Refugees and Citizenship Canada; and Canadian defence industry contractors.

Back to Table of Contents

1.3 Evaluation Scope

1.3.1 Coverage and Responsibilities

The evaluation focused on the relevance and performance of the Military Diplomacy activities within the Global Engagement Program, centering on the roles played by the CDAs deployed around the world. The evaluation supported programs detailed in the August 2017 interim DND/CAF Departmental Results Framework (DRF), Core Responsibility (CR) 1: Operations, Engagement and Outreach – Program 1.5 Global Engagement. (Corresponding Program Alignment Architecture line: 1.3.4). The evaluation was performed in accordance with the TB Policy on Results (July 2016).

The evaluation scope excludes the following elements of the Global Engagement Program, which were already assessed in the previous Evaluation of the Defence Policy and Military Diplomacy (2013): Foreign Defence Attachés posted to Canada; Liaison Officers posted abroad; Exchange/Seconded Officers; Military Training Assistance Program and policy development activities.

1.3.2 Resources

Between FY 2013/14 and FY 2016/17 payments to GAC made by DND for all accommodations and various services have increased by over 234 percent or $5.4 million.^{Footnote 6} In 2016/17, payments to GAC were approximately $7 million. Similarly, payments to Shared Services Canada (SSC) by DND for IT services and support have increased by 128 percent or $86,000. In 2016/17 payments made for IT services were around $153,000. This increase is associated with the recent addition of new CDAOs and full time equivalents (FTEs).

Following MND and CDS direction, 11 new CDAOs have been opened recently, increasing their number to 41. It is estimated that the number of personnel at post will have doubled by 2018/19.^{Footnote 7} The total number of military staff in CDAOs when the study was conducted (FY 2017/18) has reached 88. There were 32 Colonels, 13 Lieutenant Colonels, seven Majors and 36 Non-Commissioned Officers employed in 38 CDAOs.^{Footnote 8} In addition, there is a mix of approximately 30 civilian and military staff employed in the CDLS(W), CDLS(L), and PRMNY, bringing the total of FTEs in all Offices to 118.

1.3.3 Issues and Questions

In accordance with the TB Directive on Results (2016), the evaluation addresses issues related to relevance and performance. The methodology used to gather evidence in support of the evaluation questions can be found in Annex B. A logic model providing a theory of change for this program is included in Annex C. An evaluation matrix, listing each of the evaluation questions with associated indicators and data sources is provided in Annex D.

Back to Table of Contents

2.0 Findings and Recommendations

The following sections examine the extent to which the Military Diplomacy Program addresses a demonstrable need, is appropriate to the role of the federal government, is aligned with DND/CAF priorities, and demonstrates efficiency.

For key informant interviews and the questionnaire, the evaluation study used the following scale throughout the report to indicate the relative weight of the responses for each of the respondent groups:

almost all/most: findings reflect the opinions of 85 percent or more of respondents;
many: findings reflect the views and opinions of at least 60 percent of respondents;
some/several: findings reflect the views and opinions of at least 25 percent of respondents; and
a few: findings reflect the views and opinions of at least two respondents but fewer than 25 percent.

2.1 Relevance—Continued Need

Is there a continuing and future need for the Military Diplomacy Program?

This section examines whether there is a continued need for the Military Diplomacy Program. The findings in this section are based on program documents reviewed, results of key informant interviews with senior program staff, Program-recipients (client/user organizations) within the Department, and interviews conducted with OGDs.

The following indicator was used to determine the continued need for the Program:

Evidence of ongoing need for Military Diplomacy Activities (importance and utility of the CDA program to its users):
- Trends in number of CDAs and CDAOs;
- Perception of adequacy of reach/presence/engagement of CDAs; and
- Risk of not having a CDA network.

Canada’s new Defence Policy (SSE, 2017) places substantial focus on Global Engagement and Military Diplomacy, underlining that Military Diplomacy is critical to successful global engagement. The Policy further emphasizes the CDAs’ role and their contribution to military diplomacy by providing advice and information that lead to an understanding of global perspectives and regional security dynamics. According to SSE, Defence representation at Canada’s diplomatic missions abroad enhances local engagement and provides support to Canadian foreign policy and trade priorities. Perhaps most importantly, Defence Attachés build and sustain relationships that facilitate operational cooperation and communication, including in times of crisis.^{Footnote 9}

During the evaluation period, there was an increase in the number of both CDAs and CDAOs. Following the MND and CDS direction, 11 new CDAOs have been opened, increasing the number to 41, and a total of 48 additional personnel were assigned to them during the last three years. This increase in number is in line with the 45 percent increase in deployed members of the CAF during the last 3 years and is also in support of GC direction and Canada’s new Foreign Policy Statement (June 2017).^{Footnote 10} It is estimated that the number of CDA personnel at post will have doubled in 5 years.^{Footnote 11}

Up to 30 Program-recipients were interviewed. Almost all (85 percent) Program-recipients interviewed believed that without the CDA program, there would be a serious void in the missions abroad. Clients of CDAs would lose insight, information and reach into critical regions.

Back to Table of Contents

2.2 Relevance—Alignment with Federal Roles and Responsibilities

2.2.1 Does the Military Diplomacy Program Align with Federal Roles and Responsibilities?

This section examines the extent to which the Military Diplomacy Program aligns with Departmental and federal roles and responsibilities. The findings in this section are based on documents reviewed and key informant interviews, including with senior Program staff.

The following indicators were used to determine the extent of alignment:

Alignment of the Military Diplomacy Program with government acts and legislation;
Alignment of the Military Diplomacy Program with government roles and responsibilities; and
Extent of duplication of Military Diplomacy Program activities that are the responsibility of OGDs, agencies, or the private sector (role of GAC in comparison to DND/CAF).

A review of applicable federal legislation, department mandates, objectives and reports were conducted in order to determine whether the Program areas aligned with the roles and responsibilities of the GC and DND in particular. While the evaluation found that the review of federal legislation did not explicitly reference defence policy or diplomacy, these activities are seen to be an appropriate role for the GC to deliver on its Whole-of-Government Framework and support of the stated strategic outcome of a “safe and secure world through international engagement.” As per the outcome description:

The involvement of DND in this role is also appropriate. As per the Reports on Plans and Priorities from 2012/13 to 2016/17, the activity areas of Military Diplomacy align with the three key roles of DND: the defence of Canada; partnership in the defence of North America; and the contribution to international peace and security, by projecting leadership abroad.^{Footnote 13}

2.2.2 Evidence of complimentary or duplication activities

Although DND works with various other departments in the conduct of defence diplomacy, (including GAC), the evaluation found no evidence of duplication of roles. The main Program partner is GAC, followed by SSC. In this respect, while DND/CAF is clearly responsible for all defence-related issues and the involvement of military personnel, other departments are responsible for security and international affairs. Mandated by the Department of Foreign Affairs, Trade and Development Act (DFATD Act),^{Footnote 14} GAC manages Canada's diplomatic missions and services abroad and serves approximately 26 partners. GAC is responsible for providing a wide range of services to its clients in accordance with the TB Common Service Policy (2005),^{Footnote 15} including the provision of the following services: financial; property and material management; the administration of Foreign Service directives; the conduct of security services; among others.

According to the documents reviewed, the increasing complexity and horizontal nature of defence policy and military diplomacy requires cooperation and coordination among government departments, external leaders and international partners. For example, the ADM(Pol) organization consults regularly with GAC and OGDs to ensure activities are aligned and complementary to each other.

DFL program staff interviewed mentioned that DFL has established good working relations with GAC's International Client Services over the last couple of years. This issue will be explored further in the 2.4 Effectiveness and 2.5 Efficiency sections of this report.

Back to Table of Contents

2.3 Relevance—Alignment with Government Priorities

2.3.1 Does the Military Diplomacy Program Align with Government and DND/CAF Priorities?

The Program aligns with the “safe and secure Canada” outcome of the GC. In the Speech from the Throne (2015), the Federal Government pledged to strengthen its relationship with allies, contribute to greater peace throughout the world, and renew Canada’s commitment to United Nations peacekeeping operations, and will continue to work with its allies in the fight against terrorism.^{Footnote 16} Subsequently, the 2016 Federal Budget identified international re-engagement and defending Canada as one of its top priorities by articulating its commitment to providing international leadership in support of peace, security and economic growth.^{Footnote 17}

More recently, the new Defence Policy was released (June 2017), outlining GC’s Defence priorities: strong at home, secure in North America, and engaged in the world. During his speech at the release of the new Policy, the MND pledged to strengthen Canada’s engagement in the world. The Minister underlined that “Canada has much to offer to the cause of global peace, stability and prosperity.” The Minister also referenced what was mentioned by the Minister of Foreign Affairs on Canada’s foreign policy priorities, stating that “this is a time when Canada must be engaged in the world.”^{Footnote 18}

2.3.2 Alignment between the Program activities and DND/CAF priorities

The Military Diplomacy activities within the Global Engagement Program aligns with the DND/CAF priority, “Ensuring Sustainable Operational Excellence both at Home and Abroad for the Defence of Canada and the Protection of Canadians.” Through CDA activities, the Program supports DND/CAF’s strategic interests as identified in the new Defence Policy. The Minister’s Mandate Letter^{Footnote 19} and the GES CDA activities enable DND/CAF to uphold itself as a reliable partner with Allies and other like-minded countries to enhance international peace and security.

DFL delivers the CDA Program in support of the new Defence Policy (SSE, 2017), and GES, as well as the CDS direction. In addition, the program’s contribution was referenced, although indirectly, in previous DND Departmental Reports on Plans and Priorities under the Global Engagement Program by supporting Canada’s contribution to international peace and security, and also by providing support to Federal departments and agencies, foreign governments, international organizations and alliances.^{Footnote 20}

Most (85 percent) Program-recipients interviewed believed that CDA locations should be re-assessed on a regular basis, and the need should be re-evaluated. Also it was noted that, going forward, CDAs and their presence should reflect the needs of the SSE, and align with the regions/countries identified in the GES. Despite the recent release of new policies, some senior staff interviewed voiced the need for further clarity in Canada’s GES, particularly in regions such as Africa, where networking has recently been refreshed, i.e. is Canada going to focus on a few countries or distribute efforts throughout Africa? Should Canada still continue engagements with regions that are no longer a priority? Going forward, linkages with policies/priorities for all CDA locations should be reviewed and re-assessed for future planning.

ADM(RS) Recommendation

Back to Table of Contents

2.4 Performance—Achievement of Expected Outcomes (Effectiveness)

This section evaluates the achievement of the Military Diplomacy activities within the Global Engagement Program’s expected outcomes, with a focus on the following immediate outcomes: (1) DND/CAF, GAC and GC Decision Makers are informed by current and up-to-date information on foreign defence and military affairs; (2) Cooperation/Networking is enhanced between the DND/CAF and foreign defence and military forces through engagement; and (3) DND/CAF has an effective network based on its global engagement through the CDAs – CDAs impact on cooperation and international missions.

The evaluation study applied key performance indicators against each outcome. Findings are based on the triangulation of data collated from a review of program administrative data and documents, including Departmental and inter-departmental documents (acts, legislation, program administrative data); interviews with key program staff, external interviews with representatives from GAC and PCO; and country comparisons (benchmarking with UK and US, particularly on the Defence Attaché employment model). A questionnaire was administered to the CDAs and Deputy CDAs as well as the CDLS(W), CDLS(L), and PRMNY to seek their input on challenges and impediments to military diplomacy and engagement activities.

2.4.1 Immediate Outcome – 1: DND/CAF, GAC and GC Decision Makers are informed by current and up-to-date information on foreign defence and military affairs

The following indicators were used to assess the extent of satisfaction:

Evidence of up-to-date and timely information is received by Program-recipients: Response time to requests; level of satisfaction with information received – quality, utility, clarity and timeliness of information/advice;
Evidence of response to changes in world politics and Canadian priorities: Degree of adaptability and flexibility of CDAs; trends in requests; and
Existence of a coordinated effort for high-level visits/collective view within DND/CAF.

2.4.1.1 Evidence of up-to-date and timely information received by Program-recipients

CDAs are viewed as the Department’s “eyes on the ground” by the Program-recipient organizations, namely (ADM(Pol), CFINTCOM, SJS, Environments, GAC and PCO). Most (85 percent) Program-recipients interviewed expressed a high level of satisfaction with the quality and timely information received from CDAs, specifically mentioning that their interaction with CDAs has been very positive. CDAs not only provided regular reports on country-specific issues, but also delivered political level advice to senior leadership. CDAs have also been the primary interlocutors liaising between the operational groups and the strategic level groups – such has been the case in preparing for military missions in the Ukraine. In addition, most of the Program-recipients interviewed underlined that military-to-military relationship enabled quick access to resources and better information.

Based on the interview data, CDAs were viewed as valuable advisers to HOMs. During an interview with a former HOM, it was underlined that the CDAs role in providing advice in security areas was particularly “crucial and instrumental.” Accordingly, the CDA was present in almost all embassy meetings to support the HOM.

Based on the responses to the CDA questionnaire, 97 percent of the CDAs and 100 percent of DCDAs, CDLS(W), CDLS(L) and PRMNY mentioned that ‘provision of military, strategic advice to stakeholders’ was a very high-priority task for them. Furthermore, all the above mentioned groups strongly agreed that the advice and information provided have been valuable to users. In addition, all groups viewed “provision of appropriate support and assistance to HOM” as an equally very high-priority task (97 percent agreed).

Program-recipients agreed that CDA reports, particularly if they contained analysis and comments, have been very valuable. Equally useful were the country information sheets that included up-to-date defence spending, size of armed forces of the host country and defence related issues, etc. Additionally, most (85 percent) Program-recipients interviewed mentioned that the reports produced by CDAs were valuable for their organizations. These reports have been produced on a monthly and quarterly basis, and informed stakeholders on a variety of issues, such as major ongoing geopolitical events in and around the region; the current and historical defence and security environment; and the various initiatives that are taking place. Some Program-recipients underlined that providing feedback to CDAs on their reports contributed to the increased quality of these reports. The following were mentioned as further improvements to the existing reporting system: (1) inclusion of industrial activities; (2) report presentation in a standard/set format, similar to practices in some allied countries; and (3) the overall content of the report to include statistical trends and analysis of the regions involved.

Similar to the Program-recipients, 80 percent of the CDA questionnaire respondents (100 percent of the Liaison Staff in Washington, London and PRMNY, and 73 percent of the DCDAs) reported that their reports contributed to a well-informed DND/CAF decision-making process. CDAs who did not receive any feedback on their reports questioned the value/contribution of them and mentioned that some feedback would help them to improve the content.

Impediments to communication and reporting

Most (over 85 percent) Program-recipients mentioned that they contacted each CDA directly as there were no formal lines or methods of communicating with them. Some stakeholders mentioned that they were in touch with their respective CDAs on a daily basis either requesting advice/guidance and/or providing feedback.

Almost all (85 percent) Program-recipient stakeholders indicated experiencing only occasional challenges, discussed below, when they reached out to CDAs. However, this has not been the case for priority requirements. Most challenges were experienced when the resident CDAs were away on duty to one of the accredited countries (other countries under their responsibility).

In general, challenges in this area were experienced due to the following:

Accreditation.^{Footnote 21} Program-recipients mentioned that reaching and/or receiving information from accredited countries generally took longer, as some CDAs were accredited for up to ten countries. In this regard, Program-recipients interviewed indicated a preference for dealing with resident CDAs, as opposed to accredited ones. It was mentioned that most CDAs generally traveled to the accredited countries occasionally due to reasons related to proximity, limited resources and funding. Impediments to travelling limited CDA access to resources. Similarly, responding to the questionnaire, 64 percent of the CDAs and 73 percent of the DCDAs agreed that being accredited to multiple countries was a challenge. However, the Program-recipients interviewed underlined that having no CDA presence would make it extremely hard to have a current and accurate view of a specific region.

CDAs having access to relevant information. CDA questionnaire respondents commented that they observed challenges in reporting when they had no access to relevant information. They reported that while some countries were very transparent and did not hold back any information, others chose to be unresponsive to requests for information. There were also instances where the information provided was of limited relevance. Therefore, CDAs were sometimes left with very little on which to report.

Connectivity and lack of DWAN^{Footnote 22} access. This challenge was mentioned as a longstanding impediment to the exchange of information between CDAs and their recipient stakeholders. With no DWAN access, CDAOs lacked access to key DND references and situational awareness information critical to the CDA’s roles, duties and function.^{Footnote 23} In addition, 70 percent of the CDAs and 73 percent of the DCDAs questionnaire respondents felt that dissemination of information was problematic, and commented that solutions should be explored. They also commented that the MOU signed between DND/CAF and GAC required an update for support purposes as it did not meet the CDA Office communication requirements. This issue will be further discussed in 2.5 Efficiency section.

ADM(RS) Recommendation

Most (85 percent) Program-recipients interviewed mentioned that they rely on CDAs, particularly when there is an important political change in a country. For example, if a secretary of defence in a country resigns, CDAs interpret and relay the impact, find out who the replacement will be, etc. They felt that CDAs are not only a source of information but also provided analytic assessments to better inform, which added additional value.

Responding to the questionnaire, 73 percent of the CDAs, 70 percent of the DCDAs, and 78 percent of the CDLS(W), CDLS(L), and PRMNY staff agreed that ‘responding to changing world politics and Canadian priorities’ was a challenge. However, they also commented that either they were consulted at the last minute or not consulted at all prior to major decision making involving policy or process changes, which, at times, resulted in lost opportunities with the host countries. Similarly, some (25 percent) Program-recipients interviewed mentioned that CDAs were underused and that their capacity and experience could be leveraged more often and timelier in areas such as policy advice. In this regard, it was advised that consultations with CDAs should start at the beginning stages of any international negotiation or policy development rather than the current practice of at the back end of products.

Trends in requests received by CDAs

Both the program stakeholders interviewed and CDAs who responded to the questionnaire mentioned that there has been an increase in the number of requests received by CDAs. Requests were placed not only by DND/CAF Program-recipient stakeholders, but also originated from the host nations (resident or accredited) and OGDs, particularly concerning priority countries and/or where there were ongoing CAF missions. In addition, VIP visit requests from foreign nations to Canada have tripled during the last three years (the current number is about 80 high-level visits per year). This results in an increased workload for CDAOs as they are charged with coordinating and/or scheduling these visits.

ADM(RS) Recommendation

Key Finding 9: A more coordinated effort in defence engagement at large is required, preferably in a negotiated and well thought out visit plan format for each country, particularly for key priority regions.

The following indicators were used for assessment:

Extent of effort put for the coordination of high-level visits; and
Degree of consistency in opinions when providing information/advice to international partners.

Extent of effort put in for the coordination of high-level visits

Senior leadership interviewed expressed a desire for a more coordinated effort in defence engagement at large, preferably in a negotiated and well thought out visit plan format for each country, particularly for key priority areas. This issue was also raised and followed by a recommendation in the 2013 Evaluation of the Defence Policy and Diplomacy. The recommendation was in the direction of centralizing the coordination of high-level visits.^{Footnote 24}

Based on the document review and interviews conducted, there are challenges with managing high-level visits. Strategic Joint Staff (SJS) primarily manages this activity, but other organizations such as the CDAs and DFL3^{Footnote 25} also contribute to the plan.^{Footnote 26} The initial intent of tasking SJS with high-level visits was to centralize the effort and develop planning guidance directives that could be sent to all L1s to allow coordination of visit activities. The coordination activity consists of prioritizing requests based on the current Departmental policy guidance directives. Based on interviews with SJS key informants, currently there is not a single “joint global engagement plan.” Instead, each L1 prepares visit plans and submits them to SJS. SJS subsequently develops a “joint plan.” However, the SJS staff explained that in its current status in the Chain of Command, they cannot directly task an L1 with the visit plan, but may guide them in scheduling decisions. Nevertheless, SJS staff interviewed explained that establishing good relationship with L1s has enabled them to influence visit scheduling decisions to a certain extent.

SJS staff interviewed also underlined that CDAs have been the conduit for most visit requests both to and from Canada. Furthermore, they indicated that CDAs need to be aware of all incoming and outgoing visits that concern their region. During the evaluation period, SJS typically received 1,000 visit requests from DFL annually. However, it was noted that sometimes a foreign country’s visit request came directly through their embassy in Canada, in which case the CDA in charge might be unaware of such a request.

Responding to the questionnaire, CDAs and DCDAs were divided in their responses to this issue. CDAs at 47 percent and DCDAs at 45 percent found it challenging that engagement visits were not centrally coordinated. Furthermore, 67 percent of the CDLS(W), CDLS(L), and PRMNY staff agreed that not having a central coordination body negatively impacted the coordination of high-level visits.

In all, most Program staff interviewed agreed that the Department still lacked a coordinated visit plan, which leads to confusion and, at times, inconsistency in information and advice provided through defence engagement. This problem was even more evident when a country/region suddenly became a priority and increased travel to the region overwhelmed the mission.

Going forward, there is an opportunity for better tracking/recording of the number of visits the Department staff had to coordinate in the previous year. Furthermore, additional information including the number of people, names of the organizations, frequency of visits, as well as results achieved, would improve the quality of the information recorded. This would also prevent future incidents of frequent visits by the Department to the same country within a short timeframe, followed by no visits for a long duration.

Degree of consistency in opinions when providing info/advice to international partners

Senior key program informants mentioned experiencing challenges in the area of global engagement that involve a collective view, particularly prior to decisions that concerned interdepartmental collaborations. These challenges varied from changing GC priorities, to a lack of a clear overarching global engagement policy, to competing interests and communication problems. For instance, while engagement with one country looked appealing and/or was a priority for one department for trade-related reasons, for another it raised security-related concerns. Each department’s approach, therefore, differed at times.

It was underlined that consistency in presence within the Department and engagement in a centralized manner was important, as was effective global engagement. Responding to the questionnaire, CDAs agreed that inconsistency in opinions between various organizations (GAC, ADM(Pol), HOM) was a challenge. Fifty-eight percent (and 64 percent of the DCDAs) agreed that inconsistencies in this area created challenges for them. Some CDAs commented and underlined that a clear vision on government priorities and objectives for each region would help, particularly when engaging with volatile regions. Similarly, 56 percent of CDLS(W), CDLS(L), and PRMNY staff agreed that inconsistencies in opinions caused challenges. However, they commented that a more structured military diplomacy effort across the Department is needed with the inclusion of CDAs posted globally.

Furthermore, 70 percent of the CDAs, 72 percent of the DCDAs, and 78 percent of the CDLS(W), CDLS(L), and PRMNY staff agreed that rapid changing world politics coupled with Canadian priorities have been a challenge to integrate in their day-to-day work. All CDAs indicated that decision making would be faster if there were a common practical global strategy that prioritized issues in an efficient manner. Going forward, a GES continually updated and well aligned with SSE, GAC and GC priorities would ensure effective engagement activities for DND/CAF.

During discussions with senior leadership, the MOD (UK) model for international engagement was given as an example. In this regard, the evaluation team met with the British High Commission Defence Attaché and Liaison staff in Ottawa. Accordingly, the MOD (UK) model was created with a whole-of-government approach, based on UK’s new Strategic and Defence Review.^{Footnote 27} A DVCDS position was created with an entire role of managing Ministry of Defence engagement around the world. UK’s NATO staff were placed under this deputy to ensure consistency. Further, the deputy is required to communicate with CDAs and provide support in delivering a collective view. In comparison, key stakeholders interviewed mentioned that currently there is no similar mechanism in DND/CAF. Therefore and at times, even important defence engagements were not communicated departmentally. Furthermore, the majority of the CDAs,^{Footnote 28} 57 percent, and DCDAs, 63 percent, who responded to the questionnaire mentioned inconsistencies in opinions between various organizations causing confusion (e.g. GAC, ADM(Pol), HOM).

During key informant interviews, it was mentioned that a centralized coordination model, similar to that of the UK, could be formed to lead future global engagement activities while liaising with GAC. The interviewees mentioned that this activity is better suited under VCDS, since it is considered a neutral location with authority over most civilian and military organizations. Accordingly, the small VCDS team would be envisioned to lead the review of current and future ‘whole of the global engagement’ needs of the Department. In addition, this team would coordinate the high-level visit activity.

ADM(RS) Recommendation

CDAs’ impact on cooperation and international missions

According to the new Defence Strategy (SSE, 2017), Defence Attachés build and sustain relationships that facilitate operational cooperation and communication, including in times of crisis.^{Footnote 29}

Key stakeholders interviewed underlined that the establishment of cooperation with some countries around the world depended on having a military-to-military and face-to-face relations. Accordingly, the presence of military personnel who are trustworthy and knowledgeable on ongoing missions is of the utmost importance for relationship building. Furthermore, it was noted that CDA presence has been instrumental to HOMs, particularly in those embassies where Canada has ongoing missions. Similarly, responding to the questionnaire CDAs (including the CDLS(W), CDLS(L), and PRMNY staff) and DCDAs considered assisting the HOM and missions and providing strategic advice, and collection of information and reporting as their highest priority tasks. GAC staff interviewed expressed the need to have a resident CDA in embassies, particularly in unstable regions where significant security concerns exist. During interviews with the three Environments, there were mixed responses in regards to CDAs’ handling of requests from the RCN, CA and RCAF. Requirements vary for each Environment and, therefore, some exposure is needed to acquire a good understanding of issues. The three Environments confirmed that in most circumstances CDAs manage the requirements satisfactorily.

2.4.3 Immediate Outcome – 3: Evidence that DND/CAF has an effective network based on its global engagement through the CDAs

CDAs’ impact on cooperation and international missions

Based on responses to the CDA questionnaire, 97 percent of the CDAs and 100 percent of DCDAs and 100 percent of the CDLS(W), CDLS(L), and PRMNY staff mentioned that ‘establishment and maintenance of partnerships and relationships with host and accredited countries’ are a high-priority tasking. Most CDAs and DCDAs mentioned that representational events in the diplomatic community permit rapport building and information exchange among all Defence Attaché colleagues and other members of the diplomatic community, including the host nation. Furthermore, they noted that these activities boost Canada’s credibility with regards to military diplomacy in addition to supporting the HOM’s mission.

The recently introduced Defence Policy placed an appropriate level of emphasis on the CDAs’ role in global engagement. However, based on responses from CDAs, challenges continue in maintaining staff capacity. Without these resources, CDAs have been limited to in some cases only addressing high-priority requests and pressing issues. Lack of flexibility in processes, particularly concerning hospitality and travel were mentioned as impediments to relationship building. The hospitality funding issue will be further discussed in Section 2.5 Efficiency.

Appropriate housing quarters was another impediment that was brought up by CDAs who responded to the questionnaire. CDAs underlined that while the expectation to represent/host remained a high-priority tasking for CDAs, appropriate quarters for this activity were not recognized by some GAC administrators at mission. The CDAs indicated that the two organizations needed to be aligned on this expectation, so that suitable housing is provided with the expectation to represent/host, or this expectation be removed. This issue will be further discussed in the 2.5 Efficiency section.

Introduced in 2010, the GES was a broad-reaching initiative that was more comprehensive than the previous approach to military diplomacy.^{Footnote 30} It was revised in 2015 to guide the Department’s international defence diplomacy activities and promote prioritization and coherence in resource allocation.^{Footnote 31} Based on the 2015 version of the GES, over time the dividends of defence diplomacy would directly enable the execution of the defence mandate and support broader GC national security and foreign and trade policy priorities. However, budget constraints have impacted the implementation of the strategy.

The GES has been under review by an ADM(Pol) team since 2015. According to interviews with ADM(Pol) staff, the aim of the review is to align the GES with the new Defence Policy, SSE and expand its interpretation of global engagement priorities. For example, it was mentioned that the Asia-pacific region is at the front and centre of the regional priorities. Accordingly, DND/CAF is focusing on increasing its level of engagement in the region, while at the same time collaborating and consulting with GAC. It is believed that the revised version of the GES, particularly its Annexes, will provide constructive guidance to CAF leadership when conducting defence engagement. In addition, during interviews most Program-recipients mentioned that, going forward, DND and GAC should review engagement priorities once a year, identifying a “prioritization of countries.”

Responding to the questionnaire, 80 percent of the CDAs, 90 percent of the DCDAs, and 100 percent of the CDLS(W), CDLS(L), and PRMNY staff agreed that ‘contribution to the implementation of the GES’ was a high-priority task for them.

Degree of engagement with international defence industry

CDAs play a critical role as a representative of the Canadian defence industry in host nations. ADM(Mat) staff interviewed underlined that having a CDA on the ground facilitated establishing and maintaining industrial relations with the host and accredited nations. They also believed that ADM(Mat) has been well represented in major industrial trade shows, where CDAs assist greatly in preparing for these meetings. CDAs also liaised with host nations when they required information on the Canadian defence industry. In some embassies CDAs worked closely with trade commissioners and representatives from the Canadian Commercial Corporation.^{Footnote 32}

Prior to postings, CDAs receive training from ADM(Mat) and industry representatives regarding the defence industry. The training outlines the Canadian defence industry in general and historical relations/activities to date. It was also noted that some CDAs occasionally participate in specific contractor meetings thereby enhancing their knowledge of the Canadian defence industry. Since most CDAs are not informed about every company/supplier in Canada, the aforementioned training and meetings with companies have been valuable during CDAs’ engagement with international defence industry.

Based on interviews with ADM(Mat) staff, historically Canada’s defence industrial relations have been more prevalent in Europe. However, in recent years, this trend has changed due to CDAs contributions to the establishment of an effective business network. CDAs have been instrumental in the promotion of Canadian defence industry in other regions of the world by setting up bilateral meetings and providing advice and guidance in such countries as New Zealand and Chile.

Based on interviews and results of the CDA questionnaire, the evaluation team noted that the ‘promotion of defence industrial relations’ was not among one of the high-priority tasks for CDAs. Only 56 percent of the CDAs, 46 percent of the DCDAs, and 44 percent of the CDLS(W), CDLS(L), and PRMNY staff who responded to the questionnaire considered promotion of defence industry as their primary tasks.^{Footnote 33} Seventy-eight percent of the CDLS(W), CDLS(L), and PRMNY staff agreed that DND/CAF is adequately represented at international defence industry events. Conversely, only 38 percent of the CDAs and 45 percent of the DCDAs who responded to the questionnaire considered that DND/CAF is adequately represented at international defence industry events.

Back to Table of Contents

2.5 Performance—Efficiency

The following section examines the extent to which the CDA Program provides value for money by using the most efficient means to achieve the expected outcomes.

2.5.1 – Demonstration of Efficiency

To assess the efficiency of this Program, the evaluation team focused on the expenditure trends and resources used; it also examined the expenditure management between GAC, SSC, and DND/CAF; and challenges and/or any discrepancies observed.

In order to do this assessment, numerous sources of data were accessed, including review of DND financial program data and systems, payments made to GAC and SSC for accommodations and services rendered, input from the CDA questionnaire, program stakeholders’ opinions, data collected during interviews and program documents, such as the MOUs and GC Directives, and a literature review.

The following indicators were used to assess the efficiency of the Program:

Trends in expenditures;
Impediments to global engagement activities;
Efficiency of the employment model used for CDAs – comparison to allied countries; and
Possibilities for improvements in service delivery.

2.5.1.1 Trends in expenditures

The evaluation team attempted to calculate the operating costs associated with both DFL and CDA activities within the Global Engagement Program. There were several difficulties in calculating the total program costs. Due to the fact that the Departmental Results Framework has not yet been approved, programs are not aligned with financial expenditures. Therefore the evaluation team used ACL^{Footnote 34} data as a source to combine Program related expenditures and validated them with the concerned units.

Based on Table 2, between FY 2013/14 and FY 2016/17,^{Footnote 35} payments to GAC made by DND for all accommodations and various services have increased by over 234 percent or $4.9 million. Similarly, payments to SSC by DND for IT services and support have increased by 128 percent or $86,000. This increase is associated with the recent addition of new CDAOs and FTEs.

Table 2. DND Transfer of funds to GAC and SSC from FY 2013/14 to FY 2016/17. Source: Director Budget / ADM(Fin). This table shows payments to GAC made by DND for accommodations and services from FY 2013/14 to FY 2016/17.
	FY 2013/14	FY 2014/15	FY 2015/16	FY 2016/17	Variation since FY 2013/14
Transfers to GAC	$2,112,480	$3,886,534	$3,918,768	$7,046,702	234%
Transfers to SSC	$67,100	$59,502	$68,564	$152,875	128%

Table 2 Details - DND transfer of funds to GAC and SSC from FY 2013/14 to FY 2016/17

Table 3. DFL2 and CDA Office Expenditures from FY 2012/13 to FY 2016/17. Source: ACL. This table shows operating costs for DFL/CDAOs from FY 2012/13 to FY 2016/17.
ELEMENTS	FY 2012/13	FY 2013/14	FY 2014/15	FY 2015/16	FY 2016/17	Variation since FY 2012/13
DFL expenditures other than travel and hospitality under 3180AA (for all CDAs)	$2,237,396	$1,115,219	$1,877,905	$1,243,080	$2,015,687	-10%
DFL 2 expenditures under 3180AB	$613,511	$572,707	$712,912	$632,763	$646,560	5%
TOTAL	$2,850,907	$1,687,926	$2,590,817	$1,875,843	$2,662,247	-7%

Table 3 Details - DFL2 and CDA office expenditure from FY 2012/13 to FY 2016/17

Table 3 Summary: This table represents operating costs for DFL/CDAOs, the percentage variation and total over the fiscal years of 2012/13 to 2016/17. There are four rows and seven columns, including titles. Row one represents titles in each column, column one states “elements”; columns two to six have titles for fiscal years 2012/13, 2013/14, 2014/15, and 2015/16; and column seven has the title for variation since FY 2012/13. Row two represents DFL expenditures other than travel and hospitality under 3180AA (for all CDAs), with amounts over the fiscal period of 2012/13 to 2016/17 in columns two to six, and the variation percentage in the final column. Row three represents DFL expenditures under 3180AB, with amounts over the with amounts over the fiscal period of 2012/13 to 2016/17 in columns two to six, and the variation percentage in the final column. Row four represents the total with the amounts over the fiscal period of 2012/13 to 2016/17 in columns two to six, and the variation percentage in the final column.

Table 4. Travel expenditures for both the CDAOs and CDLS Washington, London and PRMNY from 2012/13 to 2016/17. Source. ACL. This table shows travel expenditures for all CDAO from FY 2012/13 to FY 2016/17.
TRAVEL EXPENDITURES	FY 2012/13	FY 2013/14	FY 2014/15	FY 2015/16	FY 2016/17	Variation 5 years
CDLS London	$209,565	$158,520	$209,165	$174,444	$159,425	-24%
CDLS Washington	$653,254	$420,061	$399,954	$357,375	$306,946	-53%
PRMNY	$110,207	$92,072	$36,605	$52,647	$60,802	-45%
CDAOs	$1,383,425	$1,186,406	$935,000	$984,994	$985,036	-29%
Total travel expenditures	$2,356,451	$1,857,059	$1,580,724	$1,569,460	$1,512,209	-36%
Variation according to previous year	-	-21%	-15%	-1%	-4%	-

Table 4 Details - Travel expenditures for both the CDAOs and CDLS Washington, London and PRMNY from FY 2012/13 to FY 2016/17

Table 4 Summary: This table represents overall travel expenditures for all CDAOs as well as CDLS London, CDLS Washington, and PRMNY. There are seven rows and seven columns, including titles. Row one represents titles for each column. Column one states “travel expenditures”; columns two to six have titles for fiscal years 2012/13, 2013/14, 2014/15, and 2015/16; and column seven has the title for variation the last five years. Row two represents CDLS London travel expenditures, with amounts over the fiscal period in columns two to six, and the variation percentage over the last five years in the final column. Row three represents CDLS Washington travel expenditures, with amounts over the fiscal period in columns two to six, and the variation percentage over the last five years in the final column. Row four represents PRMNY travel expenditures, with amounts over the fiscal period in columns two to six, and the variation percentage over the last five years in the final column. Row five represents CDAOs travel expenditures, with amounts over the fiscal period in columns two to six, and the variation percentage over the last five years in the final column. Row six represents total travel expenditures, with amounts over the fiscal period in columns two to six, and the variation percentage over the last five years in the final column. Row seven represents variation percentages according to the previous fiscal year. Column two does not contain any data because the examination did not include FY 2011/12 for travel expenditures. Columns three to six contain the percentage change from fiscal year 2013/14 to 2016/17. There is no data in column seven.

Table 5. Hospitality expenditures for CDAO and CDLS London, Washington and PRMNY New York from 2012/13 to 2016/17. Source: ACL. This table shows hospitality funding for all CDAOs from FY 2012/13 to FY 2016/17.
HOSPITALITY EXPENDITURES	FY 2012/13	FY 2013/14	FY 2014/15	FY 2015/16	FY 2016/17	Variation 5 years
CDLS London	$59,955	$58,596	$64,841	$43,729	$36,141	-40%
CDLS Washington	$85,695	$109,368	$114,207	$117,423	$107,169	25%
PRMNY	$11,172	$6,979	$4,431	$2,387	$3,885	-65%
CDAOs	$281,462	$305,373	$353,872	$359,520	$323,722	15%
Total travel expenditures	$438,284	$480,316	$537,351	$523,059	$470,917	7%
Variation according to previous year	-	10%	12%	-3%	-10%	-

Table 5 Details - Hospitality expenditures for CDAOs and CDLS London, Washington and PRMNY from FY 2012/13 to FY 2016/17

Table 5 Summary: This table represents hospitality expenditures for CDAOs as well as CDLS London, CDLS Washington and PRMNY. There are seven rows and seven columns, including titles. Row one represents titles for each column. Column one states “hospitality expenditures”; columns two to six have titles for fiscal years 2012/13, 2013/14, 2014/15, and 2015/16; and column seven has the title for variation the last five years. Row two represents CDLS London travel expenditures, with amounts over the fiscal period in columns two to six, and the variation percentage over the last five years in the final column. Row three represents CDLS Washington travel expenditures, with amounts over the fiscal period in columns two to six, and the variation percentage over the last five years in the final column. Row four represents PRMNY travel expenditures, with amounts over the fiscal period in columns two to six, and the variation percentage over the last five years in the final column. Row five represents CDAOs travel expenditures, with amounts over the fiscal period in columns two to six, and the variation percentage over the last five years in the final column. Row six represents total travel expenditures, with amounts over the fiscal period in columns two to six, and the variation percentage over the last five years in the final column. Row seven represents variation percentages according to the previous fiscal year. Column two does not contain any data because the examination did not include FY 2011/12 for travel expenditures. Columns three to six contain the percentage change from fiscal year 2013/14 to 2016/17. There is no data in column seven.

As discussed in the previous sections of this report, funding limitations, particularly in hospitality and travel funding, has been identified by CDAs as one of the impediments to effectively achieving program results.

The evaluation team reviewed the recently amended TB Directive on Travel, Hospitality, Conference and Event Expenditures (THCEE) (April 2017) and compared it with the practice used at GAC and DND/CAF. It was noted that GAC is exempted from Appendix B - Standards, of the policy.^{Footnote 36} Instead, GAC follows the hospitality guidelines governed by the OHOC Policy. It was noted that in 2013, DND briefly considered adopting a policy similar to the OHOC policy, but did not go ahead with it. Therefore, while conducting military diplomacy/industrial engagements, CDAs experienced challenges compared to their counterparts in GAC.^{Footnote 37} It was mentioned that the renewed TB Directive has increased the hospitality approval thresholds, which are expected to allow faster approvals and a reduced number of requests submitted to senior management. According to an ADM(Fin) publication,^{Footnote 38} approvals are expected to be faster.^{Footnote 39} However, this amendment is not comparable to the benefits of the OHOC policy that provides easier access to hospitality funding.

Based on feedback from the questionnaire, CDAs expressed their concern regarding their requirement to have more flexibility in the way that the administration of hospitality and minor gifts is addressed in response to hosted events. CDAs and Program-recipient stakeholders provided multiple examples where “lack of authority over hospitality” has strained military diplomatic relationships. The region of Asia-Pacific was provided as an example of a culture of reciprocity, and where DND/CAF had difficulties due to restrictions. The VCDS Business Plan (BP) for FY 2018/19 states that “the current ceiling imposed upon DFL for both the TD and hospitality funding is prohibitive and constrains DFL.”^{Footnote 40}

Stakeholders consulted suggested re-examining the possibilities of establishing a hospitality policy for CDAs, similar to GAC’s OHOC policy. Having a special funding arrangement or an exemption from the TB Directive on THCEE would align CDA global engagement activities with SSE, and, therefore, would place substantial emphasis on CDAs role in this respect. In this regard, while speaking at a conference in 2016, Vice admiral Glenn Davidson (ret’d) mentioned that Canada must invest in the ability to execute both diplomatic activities and military force projection. He went on to state that, “this is necessary for Canada to have credibility at the international table.”^{Footnote 41}

ADM(RS) Recommendation

Key Finding 15: The requirement for suitable housing has been an issue for CDAs, following a GC decision that eliminated entitlements for official residences.

Mandated by the Department of Foreign Affairs, Trade and Development Act (DFATD Act), GAC manages Canada's diplomatic and consular missions and services abroad to enable the GC to effectively serve Canadians and advance our country's interests. The maintenance of a functional mission network of infrastructure and services plays a crucial role in achieving the GC's international priorities.

In support of the international programs of all federal departments and agencies of the GC, GAC has an Interdepartmental MOU on Operations and Support at Missions. The MOU dated September 2014 is aimed at providing the framework for an efficient and cost-effective delivery of operations and common services. For resources related to representation abroad, partner departments provide GAC with agreed levels of funding prior to the delivery of common services, using the appropriate financial instruments. On an exceptional basis, GAC headquarters may agree to cash-manage the requirements of a particular partner department, subject to a written commitment of intent to proceed and until the completion of the financial transfer processes.

Both the generic and partner-specific MOUs,^{Footnote 42} outlining the expenditure management between the two departments are scheduled for renewal in FY 2018/19. The MOUs are governed by several committees such as the DM sub-committee on representation abroad, the interdepartmental ADM council on common services abroad, and the interdepartmental working group on common services abroad. According to the MOUs, and in accordance with the TBS Common Service Policy, GAC is mandated to manage the procurement of goods and real property in support of diplomatic and consular missions. These services are delivered to a variety of clients.

Based on evidence from interviews with key stakeholders, one of the outstanding issues has been about the housing entitlement for CDAs. Previously, there was a GAC decision that eliminated entitlements for official residences. Later on, this direction became a GC policy. CDAs were not affected by this policy due to the MOU^{Footnote 43} signed between GAC and DND, citing housing requirements for CDAs for their representation functions. As a result, the requirement for suitable housing with special requirements continued for CDAs. It was noted that this created confusion for GAC officials at missions, since most were unaware of the exception for CDAs’ housing requirements as stipulated in the MOU.

GAC officials interviewed also mentioned that although there was a process for reimbursement for extra costs accrued from CDA housing requirements, this did not guarantee that GAC recovered the full cost that was paid, unless funding was included in the Departmental budget process. The MOU referencing the CDA housing requirement was communicated to all embassies. However, the issue is currently ongoing. Due to these complications, GAC will be re-examining and developing client specific profiles, particularly for DND. There will also be changes to the current cost recovery framework.

Other issues related to the MOU signed between GAC and DND

During interviews with key program stakeholders it was mentioned that GAC’s unexpected mid-year expenditure requests created problems with DND’s business planning cycle. For example, in July 2016, GAC sent DND a Mission Security Funding Requirement outlining “the continued extraordinary mission security pressures being faced by GAC this fiscal year.”^{Footnote 44} GAC requested financial assistance from DND. This issue was further confirmed in a 2014 GAC Audit – Audit of International Platform Service Delivery to Canada’s Network Abroad in which it made several recommendations for the improvement of the Cost Recovery Framework, particularly to the design and the application of the costing methodology as well as monitoring and reporting. According to the Audit, these improvements would enable GAC to confidently provide assurance to internal and external stakeholders.^{Footnote 45} The Audit also stated that GAC’s method of calculation for charges was a challenge, particularly for clients such as DND attempting to manage the financial risk. As mentioned in the previous section, the GAC cost recovery methodology will be implemented in 2019 and is expected to eliminate the current problems.

ADM(RS) Recommendation

The evaluation team reviewed the employment process of CDAs within DND/CAF and compared it to allied countries. In the DND/CAF system, CDAs are senior officers, selected based on their extensive professional experience. Accordingly, most CDAs are near the end of their careers and often retire upon their return from a posting. This means that the Department does not get to benefit from their acquired experience and knowledge.

Based on interviews with key stakeholders, it was noted that in DND/CAF there currently are no mechanisms in place to map-out the post-employment of CDAs returning from posting. Former CDAs either retire or are employed in jobs that are unrelated to their acquired knowledge and skills. CDAs receive extensive training prior to a posting and gain vital experience in the defence policy areas of the regions they were involved. Therefore, it was noted that the system mostly does not allow return on investment for those experiences, skills and knowledge acquired by CDAs. According to Table 6, between 2013 and 2017, of the 50 CDAs who completed a tour as a CDA, 33 (66 percent) were retired, 10 (20 percent) were employed and posted into a different position, and only 7 (14 percent) were cross-posted to another CDA Office.

Table 6. CDAs, retired and/or cross-posted to another CDAO, or employed in the CAF in a different position. Source: DFL 2. This table shows post-employment of CDAs returning from post from 2013 to 2017.
YEAR (Annual Posting Season)	End of Tour as CDA	Retired right after CDA Tour	Cross-posting to another CDAO	Employed in the CAF, but not as a CDA	Still in the CAF at Annual Posting Season 2017
2013	10	6	0	4	3
2014	11	8	1	2	0
2015	7	6	1	0	1
2016	12	7	4	1	1
2017	10	6	1	3	3
TOTAL	50	33	7	10	8
-	-	66%	14%	20%	-

Table 6 Details - CDAs, retired and/or cross-posted to another CDAO, ore employed in the CAF in a different position

Table 6 Summary: This table represents the number of CDAs who have completed their tour and whether they have, retired, were cross-posted to another CDAO, employed in the CAF in a different position as well as those still in the CAF as of the annual posting season 2017. There are eight rows and six columns, including titles. Row one has titles for the year (annual posting season) in the first column, the end of tour as CDA in the second column, retired right after CDA tour in the third column, cross-posting to another CDAO in the fourth column, employed in the CAF, but not as a CDA in the fifth column, and still in the CAF at annual posting season 2017 in the final column. Row two has data for each of these titles for the year of 2013. Row three has data for each of these titles for the year of 2014. Row four has data for each of these titles for the year of 2015. Row five has data for each of these titles for the year of 2016. Row six has data for each of these titles for the year of 2017. Row seven has the total amount for the compiled years for each of the titles. Row eight only has data in columns three, four, and five, which have the percentages based on the totals from row seven. The rest of row eight has no data.

Based on responses to the CDA questionnaire, most CDAs mentioned that selection of CDAs should be done carefully, and the employment models that are used in the United States (US) Army or the UK should be considered. They also suggested that consideration should be given to (1) possibility of career progression, and (2) employment of CDAs within the policy and intelligence realms in between assignments. For instance, a Major and/or Lieutenant-Colonel Deputy Canadian Defence Attaché (DCDA) should be able to get employed in a policy or intelligence position upon return from a posting and then return to another post as a CDA and/or DCDA in the future. Some responders also indicated concerns that eligible candidates for CDA posts are also in decline due to the impact on spousal employment.

The evaluation team examined Defence Attaché employment models in allied countries, including in the US and UK, which have introduced different selection and retention models. For example, the US Army model aims to streamline and select well-rounded individuals relatively early in their careers at the rank of Major for a career as a Defence Attaché.^{Footnote 46} This model is similar to the new MOD (UK) employment model for defence engagement, which was released in March 2017. It is built on talent management and introduces a concept called “Functional Skills Knowledge and Experience”.^{Footnote 47} Unless a candidate has the required skills upon arrival, he/she acquires them during employment in certain designated appointments. The candidates are assessed on performance by line managers through an appraisal reporting process. This information is available to career managers to identify candidates potentially to be re-employed.

While most key DND/CAF stakeholders interviewed favoured the employment system used in allied countries, and found it particularly advantageous for young officers to be employed for short periods of time (e.g. two years), they also cautioned that DND/CAF may not fully support the UK or US model due to its small size and to the silo approach among trades, succession planning and career management imperatives. However, DND/CAF could adopt a similar approach by assessing the possibility of a specific career path related to Global Engagement / Military Diplomacy.

ADM(RS) Recommendation

The need for a better Information Technology (IT) connectivity and system access was raised repeatedly by CDAs and Program-recipient stakeholders. Connectivity and DWAN access was mentioned in the effectiveness section of the report as an impediment to the exchange of information. With no DWAN access, even simple tasks such as acquiring the contact information of a DND/CAF employee proved to be difficult. Due to the lack of connectivity on a secure universal network, most reports were only disseminated to a limited number of stakeholders in DND/CAF including select OGDs and were not easily shared with allies or with NATO, despite being in high demand. The new Defence Policy, SSE 2017 emphasized the importance of the Five-Eyes network by stating, “Building on our shared values and long history of operational cooperation, the Five-Eyes network of partners, is central to protecting Canada’s interests and contributes greatly to operational success.”^{Footnote 48} GAC officials reaffirmed that CDAs’ reports are useful when distribution is extended to them, and highlighted a need for better reporting and dissemination of information. Program-recipient stakeholders deemed the restrictions unwarranted and requested more releasable information to maintain the information sharing relationship between the Five-Eyes. In this regard, establishing a secure network (repository) for the sharing of CDA reports was suggested as a solution for moving forward.

A majority of questionnaire respondents, 70 percent of the CDAs and 73 percent of the DCDAs respectively felt that dissemination of information was challenging. Feedback from the CDAs’ and DCDAs’ questionnaire revealed that they are not provided with proper IT equipment to conduct their work. The survey results also highlighted insufficient IM/IT support from GAC, and SSC. The VCDS BP for FY 2018/19 reaffirms this stating that due to IM/IT changes driven by SSC and failings in the GAC systems, CDAOs are repeatedly denied timely communications support; cannot access key DND references; and are not able to access DND situational awareness information critical to their roles, duties and function.”^{Footnote 49}

ADM(RS) Recommendation

Back to Table of Contents

Annex A—Management Action Plan

I would like to thank ADM(RS) staff for all of the work that they have put into this evaluation of Military Diplomacy. The effort that was undertaken by the evaluation team has produced a constructive and beneficial evaluation with sound recommendations that will lead to positive changes and improvements to our delivery of International Engagement activity in support of SSE.

The evaluation has highlighted several key issues related to CDA activities, DND’s engagement with GAC and coordination of Departmental high-level visits that will translate into measureable actions through which we will strengthen our effectiveness and coherence in our Military Diplomacy as it relates to the Global Engagement Program. I agree to take on the following recommendations and will support the target dates for implementation of the MAP actions.

ADM(RS) Recommendation

Management Action

DFL with ADM(Pol), and in cooperation with primary stakeholders in the CDA program, have been undertaking a review of the placement, cross-accreditation and tasks assigned to the CDAOs in relation to the priorities and outcomes associated with SSE as well as the expectations and requirements defined in the draft GES.

DFL as the responsible Level 2 organization for the execution of tasks and requirements identified in support of SSE and the GES, will undertake a deliberate review of CDAO with all stakeholders (MNDO, CDSO, SJS, ADM(Pol), CFINTCOM, CJOC, CANSOFCOM, ADM(Mat) and ADM(PA)) to ensure positioning of CDAO and the strategy guiding their engagements support the outcomes required to meet DND and CAF objectives.

Pending publication of the new GES, DFL will chair a Level 1 Stakeholder Review to confirm that all requirements against CDAO are identified, and reissue the VCDS Instructions to Post to all CDAO to ensure they are consistent with SSE, DRF and the GES.

DFL will continue to chair the DND/CAF CDA Program Special Review Board (SRB) each September as a means to register new demands and tasks associated with the CDA Programme.

A confirmation that an initial review is complete and revised Instructions to Post have been issued will be provided by October 2018, with an annual update provided after each SRB commencing October 2019.

OPI: DFL/VCDS
OCI: ADM(Pol)
Target Date: Annual review in October of each year with first to report October 2019

ADM(RS) Recommendation

Management Action

DFL is working in cooperation with all stakeholders to develop a common “template” for reporting submissions which will improve data capture, while supporting each stakeholder’s requirements. DFL has provided initial examples which could be used on both DWAN and Classified systems allowing for common formats and data tagging in both domains.

Common reporting formats and tools to be delivered in support of this Recommendation and Recommendation 8 will allow for more effective inserting of critical CDA input, as well as a more effective mechanism to hold relevant information in a common database location. This will cue NDHQ staff to the resources available, and lend to earlier connections with CDA in the planning cycle.

ADM(IM) and CFINTCOM are assisting DFL in this process but the program’s importance remains low on the priority list for action. Once DFL gains technical support, it is assessed that a “beta” version of a common reporting tool could be fielded within 12 months to provide a baseline to develop a permanent solution.

OPI: DFL
Target Date: Common reporting tool November 2018

ADM(RS) Recommendation

Management Action

VCDS has an initiative within Business Plan 17/18 to resource a capability within DFL which will function as a coordination, tracking and information management hub in support of: global engagement activity, international engagement reporting, activity relevant to SSE objectives as well as coordination of tasks and request to the Attaché network.

This International Engagement cell will function in support of all stakeholders, Defence Diplomacy resources outside of Canada (CDA, CDLS(W), CDLS(L), NMR, etc.) and inform Departmental reporting related to SSE and DRF.

The capability currently does not reside within any organization in the CAF, and the function is currently executed as a secondary task within DFL. DFL analysis of delivery mechanisms confirmed that neither SJS, ADM(Pol) nor CJOC were capable of assuming this role and function.

An option to deliver this capability has been submitted within the Business Planning cycle and is anticipated to be resourced in FY 2017/18 in order to address this recommendation as well as support actions identified in recommendations: 1, 2, 3, 7 and 8.

OPI: VCDS
OCI: SJS/DFL
Target Date: Establish International Engagement Cell September 2018

ADM(RS) Recommendation

Management Action

VCDS/DFL has engaged GAC to determine when the MOU will be opened for review. DFL, as DND’s representative to GAC’s ADM Steering Committee for Representation Abroad, is working with GAC International Platform Branch to develop a clearer costing and accountability model for DND payments to GAC.

VCDS will ensure that key components related to the MOU as highlighted in the report (accommodation considerations, clear recognition of hosting responsibilities, etc.) will be address directly in any revised agreement.

ADM(Fin)/DCSFA shall be involved as an OCI given their role to ensure financial and tax provisions in MOUs are in compliance with applicable federal laws and government policies. The MOU review and development process is coordinated by JAG in accordance with DAOD 7014, MOU. Also, JAG, CMP/DCBA as well as the VCDS Comptroller, as OCI related to their roles will be part of the planned MOU review process. GAC has indicated that they will commence reviewing the MOU, once a date is confirmed in January 2017. A review team involving all noted OPI and OCI will be established to manage the review.

OPI: DFL/VCDS
OCI: ADM(Fin)/JAG
Target Date: Revised MOU September 2019

ADM(RS) Recommendation

Management Action

DFL is working in cooperation with ADM(IM) and CFINTCOM to address the secure systems requirement, and as a near-term objective, gaining access to DWAN for all CDAOs, as well as integrating CDAO email address information into the DWAN Global Address List (GAL).

ADM(IM) and CFINTCOM are assisting DFL in the process of developing a better secure systems solution.

Once DFL gains the necessary technical support, an interim solution to deliver DWAN access and inclusion of CDAO personnel on the DWAN GAL will be achieved within 12 months. This will allow all DND/CAF visibility of CDAO email and contact information, as well as all CDAO visibility of the DND GAL.

OPI: DFL
OCI: ADM(IM)
Target Date: April 2019

Back to Table of Contents

Annex B—Evaluation Methodology and Limitations

1.0 Methodology

The evaluation team used multiple lines of evidence and complementary qualitative and quantitative research methods to help ensure the reliability of information and data to support evaluation findings. In order to ensure the validity of data captured through different methodologies, a data triangulation approach was used. The methodology established a consistent approach in the collection and analysis of data to support findings, conclusions and recommendations. Based on the evidence from available sources, the evaluation reviewed the achievement of expected outcomes and the efficiency of the Military Diplomacy activities within the Global Engagement Program, to develop a balanced picture of the relevance and performance of the Program. Information and data were correlated to each evaluation question and corresponding indicators.

1.1 Overview of Data Collection Methods

Comparison of both qualitative and quantitative assessments was used to validate the overall analysis and to develop the evaluation’s findings and recommendations.

The following data collection methods were used to gather qualitative and quantitative data for the Evaluation:

Literature and document review;
Key informant interviews;
CDAs questionnaire;
Comparative research analysis with allies; and
Program and financial data reviews.

1.2 Details on Data Collection Methods

1.2.1 Literature and Document Review

A preliminary document review was conducted as part of the planning phase of the evaluation to garner a foundational understanding of the Military Diplomacy activities within the Global Engagement Program. A comprehensive document review was undertaken as part of the conduct phase of the evaluation, focusing on the relevance and performance of the Program.

The following documents were reviewed during the planning and conduct phases of the evaluation:

Government websites;
Government documents: 2015 Speech from the Throne; 2016 Federal Budget; 2017 Defence Policy: Strong, Secure, Engaged; 2006 Canada First Defence Strategy; 2016 TB Policy on Results; 2016 TB Directive on Results; TB Common Service Policy; 2017 TB Directive and Guide to Travel, Hospitality and Conference and Event Expenditures; MND Mandate letter; Interdepartmental MOU on Operations and Support at Missions (DFATD-GAC); GAC Audit of International Platform Service Delivery to Canada’s Network Abroad; 2017 GAC foreign policy statements;
DND documents: Reports on Plans and Priorities; Departmental Results Report (previously Departmental Performance Reports); GES; 2013 Evaluation of the Defence Policy and Military Diplomacy; VCDS Letter to incoming CDAs; CAF Force Posture and Readiness; 2015 MND and CDS direction; Treasury Board submissions; Memorandum to Cabinet; 2016 McKenzie Institute Conference Proceedings;
Program documents: CDA reports (monthly and quarterly); VCDS BP; DFL BP; various DFL presentations on Program components;
Legal documents: National Defence Act; DFATD Act.

1.2.2 Key Informant Interviews

Interviews were conducted in person and over the phone. Interviewees were provided with an interview guide in advance. During interviews, clarifying questions were asked. Notes were taken by the evaluators during interviews. These were later transcribed and compared for clarity and accuracy using an evidence matrix tool, designed to capture reoccurring themes in a common record.

Over the course of the evaluation study, thirty interviews were conducted with key program stakeholders: the Deputy VCDS, numerous staff within VCDS and DFL. Program-recipient stakeholders include the following: ADM(Pol), CFINTCOM, SJS, CJOC, ADM(Mat), and Environmental Commands (RCN, CA, RCAF). Interviews were also conducted with other OGDs officials (GAC and TBS), the UK Defence Attaché and a former HOM. Some respondents were interviewed more than once for clarity.

1.2.3 CDAs Questionnaire

CDAs are one of the main contributors to the Global Engagement Program and as such a bilingual (English and French) questionnaire was administered to gain insights into CDAs’ activities, their engagement with GAC and the effectiveness of those activities as well as challenges and impediments perceived. The questionnaire was distributed to all 42 CDAOs worldwide and included input from the following respondents: CDAs, DCDAs, CDLS(W), CDLS(L), and PRMNY. Response rates were very high at 98 percent. Questionnaire results were divided into three spreadsheets. One for CDAs, one for DCDAs, and a third one grouping CDLS(L), CDLS(W), and PRMNY together.

1.2.4 Comparative research with allies

Information from international allies was solicited to compare various aspects of the Program such as global engagement strategies and selection and retention employment models. Information was gathered through foreign government websites, interviews and email contact. The following documents were reviewed during the planning and conduct phases of the evaluation:

UK’s International Defence Engagement Strategy;
MOD (UK)’s New Employment Model;
UK’s Defence Engagement Career Field – Functional Skills Knowledge and Experience;
US Selection and Classification, Assignment of Army Personnel to the Defense Attaché System.

1.2.5 Program and Financial Data

Financial data on the Program activities was reviewed to determine the degree of efficiency of program activities. The data covering FY 2012/13 to FY 2016/17 was extracted from multiple official (DRMIS, ACL^{Footnote 50} ) and unofficial (ADM(Fin), BP, and interdepartmental transfer payments) systems and reports.

2.0 Limitations

Table B-1 describes the limitations and mitigation strategies employed in the evaluation of the Global Engagement/Military Diplomacy:

Table B-1. Evaluation Limitations and Mitigation Strategies. List of the limitations of the evaluation and the corresponding mitigation strategy.
Limitation	Mitigation Strategy
Distinguishing the CDAs’ activities and their engagement with GAC from the entirety of DND/CAF’s Global Engagement Program. Based on the recommendation from the July 2017 PMEC committee, the purpose of this evaluation was to assess CDA activities and engagement with GAC and effectiveness of those activities.	The evaluation team was careful in the scoping phase of the evaluation to determine the outputs and outcomes of the CDAs’ activities and their engagement with GAC. Although there are many outputs and outcomes of DND/CAF’s Global Engagement/Military Diplomacy Program, the evaluation only assessed what was determined to be the most important aspects as requested by PMEC.
Number and diversity of internal and external stakeholders. The evaluation team found that the Program has many stakeholders and Program-recipients within DND/CAF and outside organizations. Due to short timelines it was difficult to set interviews with all Program-recipients involved.	The evaluation team emphasized the use of a questionnaire to obtain facts, evidence and opinions from CDAs and DCDAs. The return rate was close to 95 percent, giving credibility to the data. In addition, the team validated the information with DFL through interviews
Interview bias. Interviews access the subjective impressions of stakeholders and, as such, can lead to narrow, very wide, or potentially biased views.	Insights derived from interviews required corroboration from at least one other source, either objective data or agreement with other interviewees. Also, the questionnaire allowed the evaluation team to survey a much larger sample population (as compared to interviews) to allow for better generalizations about program performance. All interviews were conducted by three interviewers in order to confirm and enhance understanding and to minimize bias.
The use of DPRs and DRRs to determine expenditures at various levels was a challenge as the figures posted in annual reports were updated at multiple occasions in subsequent reports.	To ensure proper tracking by readers, fair comparisons between FYs and to keep trend analysis of performance data in the report is applicable to the identified FY.
Lack of centralized financial data covering the Program.	Existing data was leveraged to the extent possible. The evaluation team had to rely on all other available sources of information (interviews, questionnaires) as evidence to support the evaluation findings.
Compressed timing. The evaluation mandate was received in August and had to be completed from August 2017 (planning) to January 2018 (draft report) while an evaluation usually take 12 to 18 months to be delivered.	The evaluation team interviewed as many stakeholders as possible. The interview list continued to develop throughout the evaluation process and interviews were conducted as new stakeholders were identified. All stakeholders were informed of the effect created by the compressed timeline and contributed to the study in accepting changing their schedule to accommodate the evaluation. In hindsight, with the number of stakeholders involved, a questionnaire should have been conducted instead of individual interviews.

Table B-1 Details - Evaluation Limitations and Mitigation Strategies

Figure C-1 Details - Logic Model for Global Engagement / Military Diplomacy

Figure C-1 Summary:

Inputs

Defence Policy
GAC Priorities
Privy Council Office objectives and priorities
Human
Financial
Infrastructure
Technical Resources
Whole-of-Government approach
Global Engagement Strategy

Activities

CDAs
- Collect information, provide initial analysis
  - Deliver reporting as directed
- Maintain awareness of issues relevant to the broader GC mandate
  - Reporting on trends and developments
  - Advise HOM on Defence Matters
- As POC manage/head Canadian defence diplomacy issues with accredited countries
- Promote defence industry
  - Provide assistance with bilateral defence materiel cooperation
- Build, sustain, & nurture relationships with Host Countries and with the network of Defence partners in accredited countries
ADM (Pol) and SJS
- Revie and maintain the Global Engagement Strategy
- Coordinate Strategic Engagements and High Level Visits

Outputs

Reports and documents
- CDAs
Military advice to stakeholders and appropriate support and assistance to HOM
- CDAs
Strategic advice to relevant stakeholders
- CDAs
Bi/multilateral and industrial agreements and information exchange on defence industry
- CDAs
Established channels for networking, engagement, and agreements
- CDAs
Global Engagement Strategy
- ADM (Pol) and SJS
High level visits and Global Engagement Plan
- ADM (Pol) and SJS

Immediate Outcomes

DND/CAF, GAC and GC decision makers are well informed by current and up-to-date
- All activities contribute to this outcome
Cooperation and networking are enhanced between the DND/CAF and foreign military forces through the GES
- All activities contribute to this outcome
DND/CAF has an effective network based on its global engagement through the CDAs
- All activities contribute to this outcome

Intermediate Outcome

Engagement with partner nations contributes to security and defence and provides support to Canadian foreign policy and trade priorities

Ultimate Outcome

Strategic planning, guidance and support to the successful conduct of defence operations, global engagement and outreach

Strategic Outcome

Defence operations support with the principles of “Strong at home, Secure in North America, and Engaged in the world”

Table D-1. Evaluation Matrix—Relevance. This table indicates the data collection methods used to assess the evaluation issues/questions for determining the Program’s relevance.
Evaluation Matrix—Relevance
Evaluation Issues/Questions	Indicators	Literature & Document Review	Key Informant Interviews	CDAs Questionnaire	Comparative Analysis with Allies	Program and Financial Data Review
1. Is there a continuing and future need for the Global Engagement /Military Diplomacy Program?	1.1 Evidence of ongoing need for Global Engagement/ military diplomacy activities (importance and utility of CDA program to its users)	Yes	Yes	Yes	No	Yes
2. Does the Global Engagement/Military Diplomacy Program align with the federal roles and responsibilities?	2.1 Alignment of the Program with government acts and legislation	Yes	No	No	No	No
	2.2 Extent of duplication of Military Diplomacy Program activities that are the responsibility of OGDs, agencies, or the private sector (role of GAC in comparison to DND/CAF)	Yes	Yes	No	No	No
	2.3 Alignment between the Program and DND/CAF roles and responsibilities	Yes	No	No	No	No
	2.4 Evidence of complimentary or duplicate activities	Yes	Yes	Yes	No	No
3. Does the Global Engagement/Military Diplomacy Program align with Government and DND/CAF priorities?	3.1 Alignment between the Program activities and DND/CAF priorities	Yes	No	No	No	Yes

Table D-1 Details - Evaluation Matrix – Relevance

Table D-1 Summary: This table indicates the sources of evidence used to assess the evaluation issues and questions for determining the relevance of the Program. The study used three evaluation questions to determine the Program’s relevance these questions appear listed in the left-hand column, each under its own subheading. Select a question and read across that row to learn, first, the indicators used to assess the corresponding relevance question. Read across the following five columns to learn the sources of evidence used to evaluate the question. From columns three to five, the sources of evidence are literature and document review, key informant interviews, CDAs questionnaire, comparative analysis with Allies, and program and financial data review.

Table D-2. Evaluation Matrix—Performance (Effectiveness). This table indicates the data collection methods used to assess the evaluation issues/questions for determining the Military Diplomacy Program’s performance in terms of achievement of outcomes (effectiveness).
Evaluation Matrix—Performance: Achievement of Expected Outcomes (Effectiveness)
Evaluation Issues/Questions	Indicators	Literature and Document Review	Key Informant Interviews	CDAs Questionnaire	Comparative Analysis with Allies	Program and Financial Data Review
4.1 To what extent are DND/CAF, GAC, and GC decision makers informed by current and up-to-date information on foreign defence and military affairs?	4.1.1 Evidence of up-to-date and timely information is received by stakeholders	Yes	Yes	Yes	No	No
	4.1.2 Evidence of response to changes in world politics and Canadian priorities	Yes	Yes	Yes	Yes	No
	4.1.3 Existence of a coordinated effort for high-level visits/collective view of DND/CAF	Yes	Yes	Yes	No	No
4.2 To what extent are cooperation and networking enhanced between the DND/CAF and foreign defence and military forces through engagement?	4.2.1 Extent of cooperation between DND/CAF and foreign defence and military forces	Yes	Yes	Yes	No	No
	4.2.2 Extent of CDAs’ impact on cooperation and international missions	Yes	Yes	Yes	No	No
4.3 To what extent has DND/CAF an effective network based on its global engagement through CDAs?	4.3.1 CDAs’ impact on cooperation and international missions	Yes	Yes	Yes	No	No
	4.3.2 Degree of engagement with international defence industry	Yes	Yes	Yes	No	No

Table D-2 Details - Evaluation Matrix – Performance: Achievement of Expected Outcomes (Effectiveness)

Table D-3. Evaluation Matrix—Performance (Efficiency). This table indicates the data collection methods used to assess the evaluation issues/questions for determining the Global Engagement/Military Diplomacy Program’s performance in terms of efficiency.
Evaluation Matrix—Performance: Demonstration of Efficiency
Evaluation Issues/ Questions	Indicators	Literature and Document Review	Key Informant Interview	CDAs Questionnaire	Comparative Analysis with Allies	Program and Financial Data Review
5.1 Is the Military Diplomacy Program delivering in an efficient manner?	5.1.1 Trends in expenditures	Yes	Yes	No	No	Yes
	5.1.2 Impediments to global engagement activities	Yes	Yes	Yes	No	Yes
	5.1.3 Efficiency of the employment model used for CDAs – comparison to allied countries	Yes	Yes	Yes	No	Yes
	5.1.4 Possibilities for improvement in service delivery	Yes	Yes	Yes	Yes	Yes

Table D-3 Details - Evaluation Matrix – Performance: Demonstration of Efficiency

Back to Table of Contents

Footnotes

Footnote 1

DFL Business Plan, FY 2018/19.

Return to footnote1 Referrer

Footnote 2

Audit of International Platform Service Delivery to Canada’s Network Abroad (2014). The Audit assessed the management control framework for the provision of common services to Canada’s network abroad, including the application and appropriateness of the costing methodology associated with supporting its clients and missions abroad. The audit recommended to improve the design and application of the common service delivery costing methodology. Retrieved on November 2, 2017. http://www.international.gc.ca/gac-amc/publications/audits-verification/2014/aipsdcna.aspx?lang=eng&_ga=2.72858835.1374786066.1507641526-266034156.1447950225#summ.

Return to footnote2 Referrer

Footnote 3

The HOM is the GC official representative to the country(ies), or the international organization to which they are appointed, as outlined in Section 15 of the Department of Foreign Affairs, Trade and Development Act. This specifies that the HOM is accountable to the Minister of Foreign Affairs for the management and direction of their mission and its activities and the supervision of the official activities of the various departments and agencies of the GC in the country or portion of the country or at the international organization to which they are appointed. VCDS Direction to Post, CDAO, 2017.

Return to footnote3 Referrer

Footnote 4

DFL Business Plan FY 2018/19.

Return to footnote4 Referrer

Footnote 5

CDS Directive for CAF Force (2013).

Return to footnote5 Referrer

Footnote 6

ADM(Fin) database records.

Return to footnote6 Referrer

Footnote 7

DFL Business Plan, FY 2018/19.

Return to footnote7 Referrer

Footnote 8

DFL database.

Return to footnote8 Referrer

Footnote 9

Defence Policy, Strong, Secure, Engaged, 2017.

Return to footnote9 Referrer

Footnote 10

Business Plans VCDS/DFL, (FY 2014/15-2018/19).

Return to footnote10 Referrer

Footnote 11

Ibid.

Return to footnote11 Referrer

Footnote 12

Department of Foreign Affairs Trade and Development Act (2013). Retrieved November 14, 2017. http://laws-lois.justice.gc.ca/eng/acts/F-27.5/FullText.html.

Return to footnote12 Referrer

Footnote 13

DND, Reports on Plans and Priorities 2016/17.

Return to footnote13 Referrer

Footnote 14

Department of Foreign Affairs Trade and Development Act (2013). Retrieved November 14, 2017. http://laws-lois.justice.gc.ca/eng/acts/F-27.5/FullText.html.

Return to footnote14 Referrer

Footnote 15

Treasury Board Common Service Policy, 2005. Retrieved November 14, 2017. http://tbs-sct.gc.ca/pol/doc-eng.aspx?id=12025.

Return to footnote15 Referrer

Footnote 16

Speech from the Throne, December 2015. https://www.canada.ca/en/privy-council/campaigns/speech-throne.html. Retrieved November 10, 2017.

Return to footnote16 Referrer

Footnote 17

Budget 2016. https://www.canada.ca/en/global-affairs/news/2017/06/address_by_ministerfreelandoncanadasforeignpolicypriorities.html. Retrieved November 9, 2017.

Return to footnote17 Referrer

Footnote 18

Address by Minister Freeland on Canada’s foreign policy priorities. https://www.canada.ca/en/global-affairs/news/2017/06/address_by_ministerfreelandoncanadasforeignpolicypriorities.html. Retrieved November 9, 2017.

Return to footnote18 Referrer

Footnote 19

Minister of National Defence Mandate Letter (November 12, 2015) https://pm.gc.ca/eng/minister-national-defence-mandate-letter. Retrieved November 10, 2017.

Return to footnote19 Referrer

Footnote 20

Department of National Defence, Report on Plans and Priorities FY 2012/13 – FY 2015/16 and Departmental Plan FY 2017/18.

Return to footnote20 Referrer

Footnote 21

CDAs who are residents in one country may be responsible for one other country and/or cross responsible (accredited) to several others in the region.

Return to footnote21 Referrer

Footnote 22

Defence Wide Area Network.

Return to footnote22 Referrer

Footnote 23

DFL Business Plan, FY 2018/19.

Return to footnote23 Referrer

Footnote 24

Evaluation of Defence Policy and Diplomacy (2013), Chief Review Services http://www.forces.gc.ca/assets/FORCES_Internet/docs/en/about-reports-pubs-audit-eval/p0976-eng.pdf.

Return to footnote24 Referrer

Footnote 25

DFL3 is in charge of accreditations of Foreign Service Attachés, planning and coordination of visits and activities, and visit clearance requests (VCR) for "Visits to Canada."

Return to footnote25 Referrer

Footnote 26

SJS Directorate of Strategic Engagement provides advice and recommendations to CDS on resource management for global engagement by both conducting planning and coordination activities.

Return to footnote26 Referrer

Footnote 27

UK’s International Engagement Strategy. 2016.

Return to footnote27 Referrer

Footnote 28

Includes responses of the CDLS(W), CDLS(L) and PRMNY.

Return to footnote28 Referrer

Footnote 29

SSE (2017).

Return to footnote29 Referrer

Footnote 30

Evaluation of Defence Diplomacy and Policy, Chief Review Services 2013, Retrieved December 5, 2017. http://www.forces.gc.ca/assets/FORCES_Internet/docs/en/about-reports-pubs-audit-eval/p0976-eng.pdf.

Return to footnote30 Referrer

Footnote 31

GES, 2015.

Return to footnote31 Referrer

Footnote 32

Canadian Commercial Corporation.helps Canadian exporters sell products and services abroad — and helps foreign governments purchase Canadian solutions to satisfy their national procurement priorities.Retrieved on February 2, 2018. https://www.bing.com/search?q=canadian+commercial+corporation&src=IE-TopResult&FORM=IETR02&conversationid=.

Return to footnote32 Referrer

Footnote 33

Based on the VCDS document titled ”Instruction to Post” document, CDAs are to assist applicable NDHQ clients and bona fide Canadian defence industry representatives in the promotion of defence-related bilateral trade and materiel cooperation where trade regulations and authorities permit.

Return to footnote33 Referrer

Footnote 34

ACL is an acronym for Audit Control Language; this is a system used by DND to integrate various thematic databases.

Return to footnote34 Referrer

Footnote 35

Payments made to SSC were not available for 2012/13. Therefore, this FY was omitted for GAC payments as well.

Return to footnote35 Referrer

Footnote 36

TB Directive on Travel, Hospitality, Conference and Event Expenditure. http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=27228.

Return to footnote36 Referrer

Footnote 37

Interview/Email correspondence, ADM(Fin)/ADM(RS), December 15, 2017.

Return to footnote37 Referrer

Footnote 38

Directorate of Financial Policy (DFP) Memo – Revisions to TB Directive on THCEE.

Return to footnote38 Referrer

Footnote 39

Hospitality and events, comptrollers’ forum. Nov 2017. Retrieved January 18, 2018. http://cfo-dpf.mil.ca/en/travel-hospitality/hospitality.page.

Return to footnote39 Referrer

Footnote 40

DFL/VCDS BP FY 2018/19.

Return to footnote40 Referrer

Footnote 41

McKenzie Institute Conference Proceedings, 2016.

Return to footnote41 Referrer

Footnote 42

Interdepartmental Memorandum of Understanding on Operations and Support at Missions, prepared by Client Services Division (AFR), Department of Foreign Affairs Trade Development Canada, September 2014.

Return to footnote42 Referrer

Footnote 43

Ibid.

Return to footnote43 Referrer

Footnote 44

GAC FY 2016/17 Mission Security Funding Requirement Document.

Return to footnote44 Referrer

Footnote 45

Audit of International Platform Service Delivery to Canada’s Network Abroad, 2014. http://www.international.gc.ca/gac-amc/publications/audits-verification/2014/aipsdcna.aspx?lang=eng&_ga=2.72858835.1374786066.1507641526-266034156.1447950225.

Return to footnote45 Referrer

Footnote 46

Personnel Selection and Classification, Assignment of Army Personnel to the Defense Attaché System, 2006. Army Regulation 611–60. Retrieved November 20, 2017. https://www.apd.army.mil/epubs/DR_pubs/DR_a/pdf/web/r611_60.pdf.

Return to footnote46 Referrer

Footnote 47

UK Ministry of Defence, Defence Engagement Career Field Note. March 2017. Functional Skills Knowledge and Experience is a new tool that allows to define and track priority specialist skills for its military officers, complementing the competencies already available. Such awards are made at one of three levels: aware, practitioner, or expert.

Return to footnote47 Referrer

Footnote 48

SSE 2017.

Return to footnote48 Referrer

Footnote 49

VCDS BP FY 2018/19.

Return to footnote49 Referrer

Footnote 50

Audit Control Language – System used by DND to interrogate various thematic databases.

Return to footnote50 Referrer

Page details

Date modified:: 2019-03-08

Language selection

Search

Evaluation of Global Engagement/Military Diplomacy

Table of Contents

Alternate Formats

Assistant Deputy Minister (Review Services)

Overall Assessment

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

ADM(RS) Recommendation

Page details