Evaluation of Military History, Heritage, Honours and Ceremonial Activities2

Acronyms and Abbreviations

Executive Summary

This report presents the findings and recommendations of the Evaluation of Military History, Heritage, Honours and Ceremonial Activities within the Department of National Defence and the Canadian Armed Forces (DND/CAF). Assistant Deputy Minister (Review Services) (ADM(RS)) conducted the evaluation between December 2014 and December 2015 as a component of the DND/CAF Five-Year Evaluation Plan and in compliance with the Treasury Board Secretariat (TBS) Policy on Evaluation (2009). The purpose of the evaluation was to examine program relevance and performance for the fiscal period 2009/10 to 2013/14 and to inform future management decisions related to program/service delivery and resource allocation.

Description

The Military History and Heritage program, as defined in this report, is focused on honouring and recognizing military personnel, as well as preserving and promoting Canada’s military history and heritage. The management of the program is a shared responsibility within DND. Principal stakeholders include Director History and Heritage (DHH) and Director Honours and Recognition (DH&R) within the Chief of Military Personnel (CMP), as well as the individual units within the various branches of the DND/CAF. The Military History and Heritage program also supports work done by other federal departments and agencies, such as Veterans Affairs Canada (VAC) and the Department of Canadian Heritage (DCH). Its activities also support academic research, specifically the military historical research and archival function of DHH.

Over the five-year period examined by the evaluation, the DND expenditures on Military History and Heritage declined by almost 10 percent, from $88 million annually in 2009/10 to $81 million in 2013/14 despite significant inflationary pressures.

Key Findings and Recommendations

Relevance

Over the past five years, the program activities were a priority for the Government of Canada (GC), used largely in support of promoting Canadian identity and increasingly used to support historical ceremonial events. Based on existing planned commitments, they will likely continue to be a priority. Further, the services provided by the program are seen to be extremely important within the CAF in support of preserving Canadian military values, recognizing achievements and sacrifices, and by fostering individual morale, esprit de corps and unit cohesion.

Performance

The program has demonstrated a strong capacity to support and conduct major ceremonial events and commemorations, particularly in the face of increased demand from the GC. The GC placed high priority on commemorations related to the war of 1812, major conflicts of the 20th Century (First World War, Second World War, Korean War), and marking the end of Canada’s commitment in Southwest Asia.

The program was also seen to be effective in support of CAF units with regard to communicating and promoting the CAF identity system, a key tool for fostering group identity, developing unit cohesion and supporting operational effectiveness. Furthermore, the delivery of the CAF honours and recognition process was viewed to have improved with respect to timeliness of recognition, and it was noted that CAF members are highly satisfied with the integrity of the process.

There are, however, challenges within the program, particularly with respect to preserving and promulgating CAF history and heritage. There are concerns with the system and tools in place, as well as their ability to meet modern expectations for information, knowledge preservation and dissemination. In addition, the quality of support to military museums is sporadic, with financial and technological shortcomings impacting the delivery of the effort.

Lastly, there is no overarching strategy to guide the program activities. This results in inconsistencies in the way resources are allocated and impedes achieving maximum value for money. Further, there is concern that the Department may pay for activities that support other departments but that are beyond the DND mandate. The Department could work more closely with external stakeholders, including, for example, the Canadian War Museum (CWM), to coordinate efforts and approaches. This is particularly necessary given the reduction in funding and the continuing high level of demand for support to commemorations.

Recommendations

The evaluation has made the following recommendations:

1. CMP should develop a long-term strategic plan for all military history, heritage, honours and ceremonial activities within DND that includes a review of funding allocations and collectively identifies key program area priorities.

2. The DND/CAF should review the issues related to the digitization of the collection and address the challenges related to information and knowledge dissemination, including the concerns raised with respect to the Canadian Military History Gateway (CMHG) in order to address shortcomings and meet the terms of its memorandum of understanding (MOU) with other departments.

3. In order to exploit the full potential of the Canadian military museums, DHH and Assistant Deputy Minister (Information Management) (ADM(IM)) should review the existing approach to funding and technology support mechanisms. This includes fostering a closer relationship between CAF museums, the CWM and other museum organizations that provide expertise to military museums.

Note: Please refer to Annex A—Management Action Plan for the management responses to the ADM(RS) recommendations.

1.0 Introduction

1.1 Context for the Evaluation

This report presents the findings and recommendations of the Evaluation of the Military History, Heritage, Honours and Ceremonial Activities within the DND/CAF. ADM(RS) conducted the evaluation between December 2014 and December 2015 as a component of the DND/CAF Five-Year Evaluation Plan and in compliance with the TBS Policy on Evaluation (2009). As per the TBS policy, the evaluation examined the relevance and the performance of the program over a five-fiscal-year period (2009/10-2013/14).

1.2 Program Profile

1.2.1 Program Description

Military history, heritage, honours and ceremonial activities are focused on honouring and recognizing military personnel and preserving and promoting Canada’s military history and heritage. The four main pillars to describe the program and its associated activities are as follows:

• Pillar 1 – History and Heritage: preserving, interpreting and communicating Canada’s military history, traditions, roles, contributions, professionalism, expertise and values to Canadians and members of the CAF.
• Pillar 2 – Honours and Recognition: administering the military honours and recognition system for units and individuals.
• Pillar 3 – Ceremonial Activities: preserving and promoting Canadian military customs and traditions.
• Pillar 4 – Outreach to Canadians: showcasing the men and women of the CAF, as well as their equipment, at public events across the country and abroad. (Note that this pillar is not within the scope of the evaluation as explained in section 1.3 – Evaluation Scope of this report. These activities will be the subject of a future evaluation.)

These activities will be referred to as the Military History and Heritage program henceforth, with the understanding that the program engages other activities as described earlier.

1.2.2 Program Objectives

The Military History and Heritage program encompasses the following three broad objectives:

• Instill within the Canadian public a sense of pride by sharing Canada’s military history and traditions and by showcasing Canadian military expertise and values;
• Contribute to the maintenance of a cohesive and professional culture within the DND/CAF; and
• Ensure recognition of serving and deceased military personnel and veterans, as well as their families, external individuals and organizations.

1.2.3 Stakeholders

As defined by the DND Program Alignment Architecture (PAA), the Military History and Heritage program targets groups both internal and external to the DND/CAF. These activities deliver support, directly and indirectly, to CAF commands, military personnel and internal branch (Level 1) organizations (e.g., Assistant Deputy Minister (Public Affairs)). The program also supports work done by other federal departments and agencies, particularly VAC and DCH, to fulfil their respective mandates and in response to GC priorities. Finally, these activities, specifically the military historical research and archival activities, provide support to academic institutions and researchers.

1.3 Evaluation Scope

1.3.1 Coverage and Responsibilities

The Military History and Heritage program is depicted in the PAA that was in use during the five-fiscal-year period under review¹ (2009/10-2013/14) as follows:

• 4.2.2.0 History, Protocol and Heritage Ceremonial Activities; and
• 4.1.1.6 Honours and Recognition.

The evaluation focused on the work of two directorates reporting to the Assistant CMP: DHH and DH&R.

The evaluation scope excluded demonstration units. The largest of the demonstration units in the CAF is 431 Air Demonstration Squadron (The Snowbirds), which is under the command of the Chief of the Air Service. They were excluded because the demonstration activity is considered more of an outreach activity and less a part of military history, heritage, honours and ceremonial activities. More information can be found in Annex E—Outreach Activities, which will be the subject of a future evaluation.

Responsibility for significant portions of the Military History and Heritage program also resides with many commanders as follows:

• Demonstration units such as the CF-18 demonstration aircraft, the Skyhawk parachute team and Her Majesty’s Canadian Ship Oriole report to the Chief of the Air Service, the Commander of the Canadian Army (CA) and the Commander of the Royal Canadian Navy (RCN) respectively;
• CAF Regular and Reserve Force bands are under the authority of the Commanders of the RCN, the CA, and the Royal Canadian Air Force (RCAF), as well as the Vice Chief of the Defence Staff (for the Central Band of the CAF);
• Military history, heritage-themed strategic communications advice, guidance, services and products in support of GC and Defence priorities are under the authority of the Assistant Deputy Minister (Public Affairs); and
• The reception, assessment, synchronization, capture of financial costs and tasking resources to execute CAF participation in approved events, domestically and internationally are under the combined authority of the Strategic Joint Staff (SJS) and the Canadian Joint Operations Command (CJOC).

1.3.2 Resources

As per Table 1, for the five-year period FY 2009/10 to FY 2013/14, the total estimated DND/CAF spending on military history, heritage, honours and ceremonial activities was $463.3 million, including for the Snowbirds and other air demonstration units. Although these were out of scope, it was not possible to identify, isolate and separate out these costs from within the PAA.

Table 1. Total Expenditures on the Military History and Heritage Program for FYs 2009/10 to 2013/14. This table shows the annual expenditures on military history and heritage for FYs 2009/10 to 2013/14.

1.3.3 Issues and Questions

In accordance with the TBS Directive on the Evaluation Function (2009), the evaluation addressed the five core issues related to relevance and performance. An evaluation matrix listing each of the evaluation questions, with associated indicators and data sources, is provided in Annex D. The methodology used to gather evidence in support of the evaluation questions can be found in Annex B.

2.0 Findings and Recommendations

2.1 Relevance—Continued Need

This section examines whether the Military History and Heritage program continues to address a demonstrable need. The findings in this section are based on documents reviewed and key informant interviews.

To assess continuing need, the evaluation used the following indicators:

• evidence that program outputs are being used by the DND/CAF and others;
• trends in demand for program outputs over the five-year evaluation period (2009/10-2013/14);
• perception among stakeholders of a need for the DND/CAF to deliver the program; and
• evidence that the program performs unique roles.

Key Finding 1: There has been a strong and ongoing need for the activities undertaken by this program.

The activities and services provided by the program were used by the DND/CAF throughout the evaluation period. These activities generated awareness among the Canadian public of Canadian military values, achievements and sacrifices. The activities also enhanced the military profession and CAF operations by fostering individual morale, esprit de corps and unit cohesion. The key drivers of demand were as follows:

• GC plan for major commemorations related to major conflicts of the 20th Century (e.g., First World War, Second World War, Korean War);
• marking the end of Canada’s commitment in Southwest Asia;
• commemorations of the War of 1812; and
• reintroducing distinctive environmental uniforms and royal designations for CAF branches.

The program also provided support to the following:

• responses to continuous inquiries from within the DND/CAF and from external clients concerning protocol, tradition and historical research;
• military museums, which attract an estimated 500,000visitors each year;²
• administering the Casualty Identification Program(CIP) (38 identified since 2007/08);³
• policy guidance and advice for the CAF honours system (11,424 medals, awards and decorations in 2014);⁴
• policy guidance and advice concerning the CAF identity system, which incorporates dress policy and the heritage of CAF units;
• engagements by CAF bands (1,300 engagements by Regular Force bands in 2014);⁵ and
• an ongoing requirement to safeguard Annual Historical Reports and Unit Operational Diaries, support scholarship and analysis and produce official histories.

The evaluation noted that the demand for program activities exceeded the capacity to deliver, and as such requests were prioritized based upon technical, logistical, political and budgetary considerations.

Going forward, there will be an increased demand for military history, heritage, honours and ceremonial activities. The GC’s plan for commemorations,⁶ which identifies commitments three to five years into the future, listed the need for CAF ceremonial formations, CAF bands and public affairs groups to support upcoming commemorative events for the 75th anniversaries of the Dieppe Raid and D-Day, the 100th anniversaries of the battle of Vimy Ridge, Beaumont-Hamel, Passchendaele and the Armistice, and Canada 150. There is also a potential increase in demand for the CIP as remains continue to be discovered in Northern France and Belgium where thousands of Canadian soldiers remain unaccounted for following the First and Second World Wars. It will fall upon the CIP to identify and upon the ceremonial section (DHH3) to honour these soldiers. Further, DH&R expects a significant increase in workload with the issuing of a medal to commemorate Canada 150.

2.2 Relevance—Alignment with Federal Roles and Responsibilities

This section examines the extent to which the military history, heritage, honours and ceremonial activities align with departmental and federal roles and responsibilities. The findings in this section are based on documents reviewed and interviews, and the assessment was centred around the following indicators:

• the extent to which the program aligns with current federal roles and responsibilities;
• the extent to which the program aligns with current DND/CAF roles and responsibilities; and
• the extent to which other departments, agencies and/or organizations provide a similar program.

Key Finding 2 There has been a strong and ongoing need for the activities undertaken by this program.

The program is aligned with federal mandates and with the roles and responsibilities of the GC through the combined support provided by DND, VAC and DCH.

According to VAC representatives, the activities of the Military History and Heritage program directly align with the VAC responsibility for commemorating the achievements and sacrifices of those who served Canada in times of war and peace. In response to a written request for support from VAC, the Chief of the Defence Staff (CDS) issued a May 27, 2011 Tasking Order: CF Ceremonial Support to Veterans Affairs Canada – Commemorative Events 2011.⁷ This order directed the CAF to provide support to VAC to ensure the successful execution of these events. As a result, the program plays a vital role in assisting VAC to conduct commemorative military events that celebrate Canadian military history, such as military funerals and anniversaries. This Tasking Order was superseded by the January 31, 2014 CDS / Deputy Minister (DM) Directive for Operation DISTINCTION.

With respect to DCH, the Military History and Heritage program provides critical assistance in numerous activities in support of the mandate for “Canadian identity and values, cultural development, and heritage.”⁸ An example would be the program’s role in the Bicentennial Commemoration of the War of 1812 that was led and coordinated by DCH.

The program also supports the Canadian Honours System and the Governor General’s Awards which are administered by the Chancellery of Honours, a body that includes the Governor General, and that is chaired by the Chief Justice of the Supreme Court.

Within the DND/CAF, the program supports departmental roles and responsibilities by interpreting and communicating Canada’s military history, traditions and practices to members of the CAF, the general public and to the international community. This enhanced identification with the CAF, provided support for its operations amongst Canadians and reinforced standards for professionalism and conduct for CAF members by acknowledging and celebrating those who have been notable and outstanding in their demonstration of military values.

The extent of duplication with other government departments, agencies and non-governmental organizations is limited. The expertise and experience provided by DND supported work across federal portfolio boundaries and were essential to the achievement of common goals and an integrated government response to particular initiatives, notably the 1812 bicentennial and the commemorations related to the First and Second World Wars.

2.3 Relevance—Alignment with Government Priorities

This section examines whether the objectives of the Military History and Heritage program are consistent with current GC and DND priorities. The assessment was based on the following indicators:

• the extent to which the program objectives align with federal government priorities;

and

• the extent to which the program objectives align with DND/CAF priorities.

Key Finding 3: The program aligns principally with the federal government priority of strengthening attachment to Canada, Canadian identity and Canadian military history.

The abundance of federal announcements and high-profile ceremonial activities that the program conducted during the five-year evaluation period has demonstrated that its work has been a priority area for the federal government and the DND/CAF.

The GC identified military history, heritage, honours and ceremonial priorities at the highest level throughout the evaluation period. It identified the priorities of celebrating the 100th Anniversary of the RCN and declaring April 9 as Vimy Ridge Day in the Speech from the Throne to open the Third Session of the 40th Parliament of Canada (2010).⁹ Noting that “anniversaries are an important part of how a society marks its collective progress and defines its goals for the future,” the GC made the celebration of the Bicentennial of the War of 1812 a priority in the Speech from the Throne to open the First Session of the 41st Parliament of Canada (2011).¹⁰ Moreover, the Speech from the Throne to open the Second Session of the 41st Parliament of Canada (2013)¹¹ announced a number of initiatives related to military history, heritage, honours and commemorative activities, namely the following:

• commemorating the centennial of the First World War and the 75th anniversary of the Second World War, as well as the enormous sacrifices made by Canadians and our allies in both;
• rededicating the National War Memorial to the memory of all men and women who fought for our country;
• marking the end of Canada’s mission in Afghanistan by honouring the service of our men and women in uniform, including those who made the ultimate sacrifice combating the spread of terrorism; and
• honouring the proud history of our CAF by restoring military traditions.

Federal budgets demonstrated commitments to these priorities too, including the following:

• Budget 2010: $2 million for a Community War Memorial Program; and
• Budget 2014: Commemorating Canada’s Mission in Afghanistan and Recognizing the historic titles and ranks of the CAF in the National Defence Act.¹²

The DND/CAF also identified the Military History and Heritage program as a departmental priority. The DND/CAF 2015-16 Report on Plans and Priorities states that one element of the priority of “Ensuring Sustainable Operational Excellence both at Home and Abroad for the Defence of Canada and the Protection of Canadians” is to celebrate “[i]n the coming years, a number of significant historical anniversaries… . Events such as the 100th anniversary of World War I, including a significant number of regimental centenaries, the 75th anniversary of World War II, as well as the 150th anniversary of the Confederation in 2017, will all have significant resource implications.”

2.4 Performance—Achievement of Expected Outcomes (Effectiveness)

This section presents the evaluation of the achievement of expected outcomes of the Military History and Heritage program, with a focus on how these activities support these outcomes.

First, the governance of program activities is assessed as an all-encompassing program activity. Following that, an assessment is made on the following outcomes, organized by the program’s main activity pillars:

Pillar 1: History and Heritage

• Canadian military history and heritage are preserved and promulgated; and
• The Military History and Heritage program enhances the military profession and CAF operations.

Pillar 2: Honours and Recognition

• Achievements, services and sacrifices of CAF organizations and members are honoured.

Pillar 3: Ceremonial Activities

• Canadian military customs and traditions are preserved and promulgated.

Pillar 4: Outreach to Canadians

• The proficiency, professionalism and sacrifices of the Canadian military are promoted, celebrated and recognized at home and abroad. As previously mentioned in section 1.3 – Evaluation Scope, this pillar is not within the scope of this evaluation but will be examined in a future evaluation.

2.4.1 Governance of Military History and Heritage Program Activities

To determine that the Military History and Heritage program is well-governed, the evaluation considered the following indicators:

• The program implements planning and performance measurement frameworks to prioritize work and allocate resources; and
• The program has the appropriate organizational structures across the DND/CAF.

Key Finding 4: The current management structure involves several stand-alone organizations that play an isolated role in supporting the DND/CAF in the promotion of Canadian interests and values.

The evaluation examined the policies, procedures, and practices that were intended to, or have the effect of, governing the management of resources dedicated to the Military History and Heritage program. While the evaluation found examples of strong individual governance structures in all areas, it also identified a “stovepipe” approach to the management of the program. The DND/CAF lacks a means to provide a clear picture of all the resources being dedicated to specific activities throughout the program. Thus, the DND/CAF cannot assess accordingly the opportunity costs and other implications of the decisions taken across activities. As an example, while the PAA provides overall expenditures, there is no unified planning undertaken to ensure that all of these expenditures come together in the most effective manner to achieve the program’s overall objectives.

The evaluation also noted the challenges inherent in trying to design useful or meaningful performance measures for outcomes such as Canadian identity or attachment to Canada. The evaluation noted that CMP has recently established some performance measurement for certain parts of the Military History and Heritage program, but to date has not used that information to manage the program. Another concern is that performance by other actors that complement, support or impact the work done by the two directorates is not considered. While efforts are underway to coordinate the work of CJOC/SJS and the Level 1 commands to support GC ceremonial and commemorative events, along with those of CMP, these are not yet fully in place. As a result, there is no strategic approach to determine what means are most effective and where scarce resources should be allocated.

2.4.2 Pillar 1: History and Heritage

2.4.2.1 Preservation and promulgation of Canadian military history and heritage

This section evaluates the extent to which military history and heritage were preserved and promulgated, and the assessment was based on the following indicator:

• evidence of sound processes for acquisition, conservation, preservation and protection of records, artefacts and other items of significance.

Key Finding 5: There are numerous challenges and concerns with the program and the tools in place in respect of its ability to meet current expectations for information, and knowledge preservation and dissemination.

A key expectation of the Military History and Heritage program is that DHH helps support the RCN, the CA and the RCAF to maintain their federal heritage collections, including that of military artefacts and art.¹³ Management of these collections requires the adherence to a clearly established policy and direction, a governance structure to plan and oversee delivery of the policy, sound delivery approaches and proper skills and resources. Indeed, the evaluation found that there were policies and standards in place that applied to most program activities. The authority for Canadian military history and heritage is defined under a Defence Administrative Order and Directive (DAOD) that identifies its mandate as “to preserve and communicate Canada’s military history and foster pride in Canadian military heritage.”¹⁴

Further, it was found that DHH acts as the Department’s central governing body for the management of historical collections, and that all stakeholders recognize it as such. In this role, there are clear strategies in place to recognize what constitutes a historical item and what approaches to follow to ensure that it is acquired, preserved and protected. Efforts are also made to keep that guidance current and to notify the DND/CAF of changes to those documents. These plans and processes are developed and implemented by professionally trained historians, librarians and archivists. In addition, the RCN, CA and RCAF employ subject matter experts on military history as it relates to those services. The evaluation found a consensus across the Department, and in selected federal departments and agencies, that DHH possesses and exercises expertise and competency in areas related to military history and heritage.

While the governing structure appears to be in place, there are concerns whether the information technology (IT) systems and tools necessary to meet the desired expectations are adequate. The evaluation examined progress on digitization, as well as information and knowledge dissemination. Deficiencies were noted here. In fact, the challenges associated with the shift towards creation and storage of information and knowledge in electronic form were identified more than 15 years ago: “vital information was no longer being put in a format that could easily be encoded or stored for future retrieval.” The program has been criticized, noting that “we are probably living today in what will be the worst documented era of Canadian military history ever.”¹⁵

The evaluation found that many of the problems identified in 2000, and discussed below, still remain to be addressed to a certain extent. For instance, DHH faced significant challenges in its efforts to “receive, catalogue, store and conserve unit historical reports and operations diaries until they are deposited at Library and Archives Canada in accordance with the Defence Subject Classification and Disposition System.” There is no strategy or plan in place for the transferral of documents from paper to digital format. Further, some information exists in digital formats that are no longer readable with current technology, and retaining the equipment and the expertise to work with this information is a further strain on efforts to perform and facilitate historical research.

The inability to exploit the potential of the CMHG is another restraint. The CMHG is an online service that provides access to websites and digitized resources about Canada’s military history. The goal of the Gateway is to “provide the public with free access to the collective military history resources of Canadian museums, libraries, archives and other heritage organizations through a single, dynamic and intuitive gateway.” It is a joint National Defence On-Line and DCH (Canadian Culture Online) program. Despite a MOU and initial promotion in DND Reports on Plans and Priorities, the Gateway is not being supported by other departments and agencies, and its links and connections are often broken. DHH’s capacity to generate new content and effect maintenance are limited under current circumstances.

Of particular note is the effort to preserve and exploit the information related to Canada’s mission in Afghanistan. The operational records for the mission are in digital form but security requirements mean that it cannot be viewed via the Department’s Consolidated Secret Network Infrastructure or Defence Wide Area Network. A segregated processing environment had to be created, but it is not yet operational.

2.4.2.2 Support to Academic and Historical Research

This section evaluates the extent to which academic and historical research was effectively supported. This assessment was based on the following indicator:

• evidence the program delivers satisfactory support to academic and historical research.

Key Finding 6: Academic and historical research is supported.

Academic and other historical research is facilitated by DHH staff as well as the directors responsible for history and heritage in the RCN, the CA, and the RCAF. Research is supported by facilitating access to archives and other documents and by providing guidance and subject matter experts.

The evaluation also found ample evidence of the collection and preservation of historical records and other material related to the CAF in order to ensure that information is available for current and future historical research. Such research is conducted within the DND/CAF, as well as by individuals external to the department, including academics and the general public. The program also works in a collaborative and collegial fashion with Library and Archives Canada and the CWM to provide a whole-of-government service. The evaluation noted that this whole-of-government effort is more informal than formal, and that it reflects personal relationships as opposed to a codified relationship.

2.4.2.3 Quality of support to the military museums of Canada

This section evaluates the quality of support provided to the CAF military museums. This assessment was based on the following indicator:

• evidence the program provides satisfactory leadership, guidance and support to the Military Museums.

Key Finding 7: The nature and level of support to military museums vary, both financially and technically, depending on the status of military museums, their relationship with the DND/CAF and the resources available to the DND/CAF.

The legal and administrative framework¹⁶ governing CAF military museums is based on the following:

• National Defence Act, section 38
• Canadian Forces Museums – Operations and Administration (A-AD-266-000/AG-001)
• Canadian Forces Museums (CFAO 27-5)¹⁷
• Base and Station Funds – Regular Force (CFAO 27-6)
• Disposal of Non-Public Property (CFAO 27-9)
• Artifacts and Memorabilia – Non-Public Property (CFAO 27-10)
• Organization – Establishment Policy and Procedures (A-AE-219-002/AG-001)
• Policy and Procedures for Non Public Property (A-FN-105-001/AG-001)
• CF Construction Engineering Manual (MC-08-005-120/AG-000).

It is the policy of the DND/CAF to encourage contact among all museums, whether operated by the CAF or by civilian agencies, to promote the preservation of military artifacts and to cooperate for the good of all. Furthermore, the CAF museum system is the collective term for the grouping of CAF museums under the supervision of the technical staff, for museological matters, of the DHH at National Defence Headquarters. The role of CAF museums is to preserve and interpret Canadian military heritage in order to increase the sense of identity and esprit de corps within the CAF and to support the goals of DND. Each official CAF museum (and unaccredited unit/base historical collection) is an independent, self-financed, non-public fund institution operating under the authority and direction of its supporting commander. Additionally, museums are activities operated in accordance with policy and orders governing such activities promulgated by the CDS under section 38 of the National Defence Act.

CAF museums can be viewed as the frontline for educating the public and members about military history and heritage, receiving as many as 500,000 visitors per year collectively. DHH is responsible for CAF museum governance as stipulated in the Canadian Forces Museums – Operations and Administration publication. DHH also chairs the CAF Museum Committee, which is responsible for overall CAF museum policy, accreditation and the conduct of periodic inspections to ensure accreditation standards are maintained. This committee also approves funding allocations to the network of military museums via the Museum Development Fund (MDF). From FY 2009/10 to FY 2013/14, the MDF oversaw the distribution of $31 million to accredited CAF museums. This represents critical funding upon which most museums rely to sustain their activities.

The evaluation noted issues with the MDF fund allocation process. For example, some museums, particularly smaller ones, stated the process was not transparent, somewhat complicated and time consuming. Justification for a refusal of funding was generally not consistently communicated, thereby limiting a museum’s ability to improve its funding eligibility. Consequently, successful museums receive ongoing funding, while less successful ones are impeded by the process. Prior to 2013, museums were not required to account to DHH for the status of the funds they received. Some museums viewed this as a positive thing as it allowed them to earn interest on unspent MDF allocations during the fiscal year. This resulted in MDF money remaining unspent in some museums while concurrently other museums were seeking additional funding.

DHH also provides training support to CAF museums via an annual museology course through the Organization of Military Museums of Canada. Interviews with museum staff indicated that the course is useful and informative, as well as an excellent venue for networking and coordinating best practices with other CAF museums. While there were some issues raised about the format and content of the course, the principal concern was the funding source. MDF funding cannot be used to pay for course attendance, so museums relied upon other means (i.e., base funding or museum revenues) to pay the course fees and associated travel costs. It was suggested by multiple interviewees that it would be beneficial if DHH funding could be used to cover these expenses as well, particularly for staff of the smaller, more modest museums as they are more likely to benefit from the course.

DHH also provides technological support to CAF museums through a common artefact management software. The aim is to ensure all museums are using the same programs and processes thereby enabling DHH to better manage and support them. In 1999, DHH deployed the initial release of the Canadian Forces Artefact Management System (CFAMS). DHH developed CFAMS with the intention to facilitate the cataloguing and managing of artefacts; however, CFAMS has not met its implementation intent. Interviewees unanimously stated that CFAMS has never achieved its full operational capability, and that it was not capable of properly managing their collections. Interviewees also noted a lack of technical support from DHH when encountering issues with CFAMS. Some museums have now moved onto other commercially available products. DHH has plans to replace CFAMS with a new system in the near future: The Museum System. However, interviewees indicated that they were not consulted on the proposed change, have not received sufficient communication about the proposed implementation date, and are not confident that the system chosen by DHH will be adequate for their needs. In fact, more than one interviewee expressed concern that The Museum System is more appropriate for an art gallery than a military museum.

The evaluation also found that, in accordance with orders issued by CMP, DHH ensures CAF museums are in compliance with DND/CAF nuclear safety orders and directives and the Nuclear and Ionizing Radiation Safety Program, improving awareness and education for staff and volunteers. This effort is mainly in relation to museum artefacts.

Key Finding 8: A closer relationship between the CWM and the CAF museums would strengthen military history and heritage activities.

The evaluation found that, despite a good relationship between DND and the CWM, there is little collaboration between the CAF museums and the CWM, the largest military museum in Canada. A recent internal study conducted by the CWM concluded that the vast majority of military museums and historical collections have had little or no contact with the CWM since the early nineties. There is a misconception among some military museums that by collaborating with the CWM, they will lose ownership of their artefacts. Where the DND/CAF can exercise influence and leadership to enhance the relationship between the CAF Museums and the CWM, it would support the achievement of departmental and GC priorities related to military history and heritage. This would be consistent with the larger GC objectives of promoting and preserving Canadian cultural heritage, as well as the DND/CAF’s commitment to promoting a common agenda among military museums.¹⁸

2.4.2.4 Advice and expertise to support CAF units and formations

This section evaluates the extent to which advice and expertise are provided to support CAF units and formations. This assessment was based on the following indicator:

• evidence the program provided satisfactory technical advice and expertise to support CAF units and formations.

CMP is responsible for managing all aspects of the CAF identity system, including insignia, rank structure and uniforms. Acting on behalf of CMP, DHH is responsible for the following:

• communicating and promoting the CAF identity system to CAF members and the Canadian public;
• maintaining the templates, archives, sealed patterns and manuals necessary for preserving the various elements of the CAF identity system;
• making changes to policies, plans and procedures associated with the maintenance of the CAF identity system; and
• providing staff and technical advice to the chain of command.

In addition to command intent, managed through the command chiefs and command structure, there is a CAF committee for dress and ceremonial matters. Element commanders are responsible to ensure adherence in their organization to the attributes of the CAF identity system and seeking changes as necessary. Command chiefs and senior commanders indicated that the identity system helps foster group identity, cohesion and operational effectiveness and is, in their opinion, well managed through the committee process.

2.4.3 Pillar 2: Honours and Recognition

2.4.3.1 Honouring the achievements, services and sacrifices of CAF organizations and members

To assess the extent to which the achievements, services and sacrifices of CAF organizations and members are recognized and honoured, the evaluation considered the following indicators:

• stakeholder perception on the thoroughness of process and procedures;
• confidence that the process provides the appropriate means to bestow honours and recognition; and
• compliance and oversight mechanisms.

Key Finding 9: Stakeholders are confident that the honours and recognition process is thorough and appropriate to award honours and recognition.

The system for military honours and awards to individual members of the CAF is part of the larger system for Canadian honours through which the Governor General bestows honours on behalf of the Sovereign.

Military honours and recognition have long been a priority in the DND/CAF. A former CDS observed that “offering appropriate recognition to those who serve in the defence of their country is important to every society around the world. Recognition is a key element of morale building in a military environment, one that fosters esprit de corps and creates role models for others to emulate.”¹⁹

The evaluation found that there is a well-established process in place to oversee military honours recognition and awards as found in the publication “The Heritage Structure of the Canadian Forces.”²⁰ Information is disseminated in writing, as well as electronically in forms designed to facilitate understanding of the relevant policies, criteria, and their application. The evaluation found that the military honours system provides decision makers with ample information.

That said, DH&R is working at capacity. In addition to policy and committee decision-support functions, DH&R receives approximately 100 honours and recognition-related applications per week. DH&R uses a triage approach that prioritizes files of the dead and dying first, files related to deployed operations second and files related to all other domestic operations last. In addition, the directorate has imposed upon itself a service standard for processing applications. That is, 40 working days for CAF members, 90 working days for civilians and 90 working days for retired CAF members. The internal service standard for information and advice requests via email is 10 working days.

The evaluation confirmed that appropriate oversight is provided. A challenge function is exercised by committees and by DH&R that reinforces decisions that follow criteria and are based on evidence with the aim of ensuring consistency and legitimacy of the ultimate decision.

The CAF also maintains a Battle Honours Committee that reports to the CDS as the approving authority. The Committee noted that, “It is of prime importance that past honours retain their value. This can be achieved by ensuring that rules for any award be not only strict, but also strictly applied.”²¹ The Canadian Forces Battle Honours Committee periodically reviewed its guiding principles and eligibility criteria and made changes when needed to reflect new research or a changing understanding of particular components. The evaluation found that DHH was integrally involved in supporting the Battle Honours Committee, providing subject matter expertise and specialist knowledge.

The evaluation conducted interviews with selected representatives of Level 1 commands, specifically people working within personnel structure, chief warrant officers and other senior officers. The focus was on individual honours and recognition, but group and battle honours were also discussed. The interviewees expressed strong satisfaction with the process by which battle honours are bestowed upon units.

The evaluation found that decision makers (from the unit to the CDS) have access to information sufficient to initiate applications, review them and make recommendations as required. In addition, the evaluation found that DH&R acts to support decision making by providing advice with respect to the application of policy and procedures, including supplying precedents. Units celebrate the occasion of an allocation of a new battle honour and unit colours and standards (which bear references to battle honours) as a point of pride.

2.4.4 Pillar 3: Ceremonial Activities

This section evaluates the extent to which ceremonial activities and traditions are preserved and promulgated. This assessment was based on the following indicator:

• evidence the program provides satisfactory support to highly visible GC activities and ceremonies.

2.4.4.1 Canadian military customs and traditions are preserved and promulgated

Key Finding 10: Support to commemorative ceremonies has been a significant area of activity for the DND/CAF in order to meet a government priority.

DCH acts as the lead department for commemorative activities within Canada, while VAC has the lead for those occurring outside Canada. The DND/CAF supports both departments in these endeavours.

The period examined by the evaluation was one of significant activity for the GC in terms of highly visible activities and ceremonies related to military history and heritage. This was outlined in a detailed GC Commemoration Plan, an overview of which is shown in Figure 1.

Figure 1. GC Commemorative Activities 2013 to 2020. This figure demonstrates the Government of Canada’s military commemorations for the period 2013 to 2020.²²

Representatives of DHH, CJOC and SJS were interviewed for the evaluation. All three organizations represent the DND/CAF on an Interdepartmental Commemoration Committee, chaired by DCH. The evaluation noted that while the DND/CAF was one of 23 federal departments and agencies involved in these events, its significant contribution of expertise and resources through its participation in the Interdepartmental Commemoration Committee and its working groups made it a major player in the planning and execution of commemorations. Indeed, the evaluation noted that DCH considered the DND/CAF an essential partner in the fulfillment of its commemoration mandate.

The DND/CAF support to GC ceremonial and commemorative activities has evolved during recent years. As of January 31, 2014, CAF support to the GC national commemoration program has been organized and planned by CJOC in accordance with the directive issued by the CDS and the DM. The national commemoration program receives dedicated funding. The evaluation found evidence that the planning and execution of this support was conducted in an often demanding operating environment with frequently changing requirements. The evaluation noted leadership by SJS on behalf of the CDS to address the often ad hoc approach to ceremonial and public duty activities in the National Capital Region. Likewise, the evaluation noted that for activities conducted as part of Operation DISTINCTION, CJOC is the supported commander, and CJOC J8 tracks and reports costs associated with those activities, providing the DND/CAF with visibility on the costs associated with these commitments. The evaluation did not find unified tracking and reporting for activities conducted outside of Operation DISTINCTION which, as all indications suggest, would still be a significant level of ceremonial and public duty. Arguably, when available, this would give the DND/CAF a more complete view of how resources are being committed and to what ends.

DCH and VAC interviewees stated that they were very satisfied with the support they have received from DND/CAF on the majority of initiatives for commemorative activities and that the level of collaboration has been excellent. Indeed, DCH representatives remarked that information and guidance provided by DHH regarding Canada’s military history and ceremonial/protocol elements had been greatly appreciated. In addition, DND/CAF planning and logistical support was considered crucial by both DCH and VAC. For example, DND/CAF provided support to the VAC guide program at the Vimy Ridge and Beaumont-Hamel memorial sites in France.

2.4.4.2 Military History and Heritage contributes to the military profession and CAF operations

To assess the extent to which the Military History and Heritage program enhances the military profession and CAF operations, the evaluation considered the following indicators:

• the extent to which program outputs are used to foster and develop esprit de corps and unit cohesion;
• the extent to which program outputs are used to foster and develop Canadian military professionalism; and
• the extent to which program outputs are used to support CAF recruiting.

The evaluation frequently heard from personnel at all levels across the DND/CAF that the Military History and Heritage program is instrumental in maintaining customs and traditions. Dress, badges and colours are all seen as unique identifiers across the CAF that forge unit identity and cohesion, and all three of these items are centrally managed by DHH. Museums, also supported by DHH, are used as a tool to show new members the history and heritage of their unit and instill pride in the members. Bands, as the evaluation heard from many CAF members including senior officers, are instrumental in fostering esprit de corps. The battle honours, administered by DHH, are awarded to units of the CAF and displayed on their colours giving CAF members a visual reminder of what their unit has accomplished and what is expected of them. Individual honours, administered by DH&R, can provide motivation and pride, not only for an individual, but they can be a source of pride for an entire unit.

A 2011 report commissioned by Director General Military Personnel Research and Analysis²³ stated that dress, drill, insignia and ceremony are all important parts of custom and tradition, and that they build a sense of community. That sense of community is essential for “commitment, retention and performance as it leads to a sense of belonging.” Despite the noted importance of these elements, the report concluded that CAF members perceived a loss of customs and traditions, and that this should be of concern to the CAF. The evaluation heard similar concerns in interviews with senior non-commissioned officers from across CAF elements. There were numerous causes identified for the loss of customs and traditions, but perhaps the most prominent was the intensity of the CAF involvement in Afghanistan, the focus on maintaining operational tempo and (for the Army, in particular) honing battle skills.

Nevertheless, the program was seen to help generate esprit de corps through honours and awards. In interviews, senior non-commissioned members stated that dress and drill create discipline, and that discipline translates directly to the battlefield. The evaluation also found that by maintaining customs and traditions, and by communicating the culture and heritage of the CAF, members are increasingly made aware of the expectations placed upon them.

Recruiting for the CAF frequently employs references to the past achievements and historic traditions of the CAF. For example, attraction and recruiting was the primary focus of Operation CONNECTION. The Military History and Heritage program was harnessed to this effort, notably CAF bands and commemoration events such as the Royal Nova Scotia Tattoo.

2.4.4.3 The development and sustainment of CAF music capability

To assess the extent to which the Military History and Heritage program supports and develops the CAF music capability, the evaluation considered the following indicator:

• evidence the CAF Music program (DHH7) adequately supports development and sustainment of CAF music capability.

Queens Regulations and Orders (Volume 1, Chapter 32) and DAOD 5040-0 and 5040-1 establish the rationale and the roles for quality music and bands in the CAF. Military musicianship and bands in the Canadian military have a very long tradition, and the evaluation heard repeatedly that both are considered to be integral to efforts of the CAF. DAOD 5040-0 puts it succinctly: “The provision of quality music by CF bands and their musicians supports CF operations, fosters morale and esprit de corps and promotes CF and Canadian aesthetics and values, both nationally and abroad.”

The CDS is the ultimate authority for the employment of CAF bands and their administration and training. Commanders of the RCN, CA and RCAF exercise command over CAF bands and assigned musicians under their authority. Commanding officers of units and formations approve the provision of music support by CAF bands under their authority.

Consistent with the above orders, directives and regulations, the evaluation found that CAF bands are employed for the following purposes:

• public relations and outreach deemed to be in keeping with the dignity or prestige of DND or the CAF;
• recruiting; and
• commemorations and other ceremonial events.

With respect to supporting the development and sustainment of the CAF music capability, the evaluation found that there was generally a high level of satisfaction among all stakeholders with the quality, availability and delivery of the CAF music capability.

As per the DAOD,²⁴ the responsibilities of the DHH (both the Director and DHH-7 as Music Branch Advisor and Supervisor of Music) in relation to supporting the development and sustainment of the CAF music capability are as follows:

• developing policies, plans and procedures to guide the provision of music support (DHH);
• providing staff and technical assistance visits to CAF bands (DHH);
• providing a focal point for the professional identity and functional employment of musicians (DHH-7 as Music Branch Advisor);
• conducting, coordinating and evaluating technical auditions of prospective musicians as part of the recruiting process (DHH-7 as Supervisor of Music);
• reviewing the instrumental configurations of all CAF bands ((DHH-7 as Supervisor of Music);
• validating the technical capability and professional competency of all musicians and CAF bands ((DHH-7 as Supervisor of Music); and
• recommending policy changes to DHH (DHH-7 Supervisor of Music).

In support of personnel generation related to CAF music, the evaluation found evidence that DHH-7 supports commands and commanders to ensure the provision of quality music by CAF bands. DHH7 is the lead for attraction and recruiting of musicians, including conducting auditions. The evaluation also saw evidence that DHH7 supports CAF bands (both Regular Force and Reserve Force) through staff assistance visits.

The evaluation found that DHH, and specifically DHH-7, was performing its designated role with respect to the CAF Music Program, namely supporting musician personnel generation and supporting the delivery of the CAF’s music capability. The evaluation saw that decisions with respect to employing that music capability, in the form of bands or smaller sets of musicians, rests with the commanders of commands and commanding officers.²⁵ That aspect of the Military History and Heritage program is discussed in Annex E of this report.

2.4.5 Pillar 4: Outreach to Canadians

Information regarding this pillar may be found in Annex E of this report.

2.5 Performance—Demonstration of Efficiency and Economy

An assessment was conducted to determine whether the Military History and Heritage program is affordable, achieving value for money and operating efficiently using the following indicators:

• trends in program costs versus the DND budget (affordability); and
• trends in overall program expenditures versus activity (efficiency gains).

2.5.1 Trends in Proportion to Program Budget versus Overall DND Budget

Key Finding 11: Expenditures are small given the priority accorded to program activities. Further, they have declined as a percentage of the overall DND budget.

Costs for The Military History and Heritage program are attributed through the PAA as opposed to being coded directly to activities, thereby making it difficult to measure the cost of specific activities within the program. For the purposes of the evaluation, the total costs for The Military History and Heritage program are the sum total of the costs attributed to line 4.1.1.6 – Honours and Recognition and 4.2.2.0 – History, Heritage, Ceremonial and Protocol under the previous PAA.²⁶ Note that this cost includes the Snowbirds and other air demonstration units. Although these were out of scope, it was not possible to identify, isolate and separate out these costs from within the PAA. Using this data, over the five-year evaluation period, the cost of the Military History and Heritage program averaged about $92.7 million per year, for a total of $463.3 million.

Salaries accounted for 52.3 percent of annual spending, while operations and maintenance (O&M) costs accounted for 30.8 percent. These expenditures, at approximately 0.5 percent of the overall DND budget, do not account for a large proportion of departmental spending. While the program does not directly support the operational aspects of the CAF, such as force employment or readiness, it does contribute directly to one aspect of the Department’s two strategic outcomes, that of promoting Canadian interests and values. In addition, the program plays a key role in ensuring that CAF members honour the principles of military professionalism and conduct that have become part of the Canadian military’s international legacy and reputation.

2.5.2 Trends in Program Costs

Key Finding 12: As the various program areas sought to reduce expenditures, they may have done so at the expense of program capacity as opposed to the achievement of operational efficiencies.

Program expenditures were significantly reduced (a decline of 7.3 percent) throughout the evaluation period, driven mainly by Strategic Review and Deficit Reduction Action Plan initiatives. These reductions occurred despite inflationary pressures, particularly with respect to salaries within all areas. Figure 2 below depicts the changes in the resource allocation for the program over the five-year evaluation period.

The area of increase during the period was the O&M costs of the Snowbird fleet, which saw a 23.7 percent increase in associated national procurement expenditures and an approximate 19 percent increase in aviation fuel prices. Given that military equipment inflation is typically in the range of 5 to 7 percent annually, this increase over the five-year period is reasonable and generally unavoidable. The overall costs attributed to the Snowbirds air demonstration squadron, however, remained relatively stable from 2011/12 to 2014/15 and in fact declined slightly, by 2.3 percent from $47.3 million to $46.2 million.

Figure 2. Military History and Heritage Expenditures in Millions of Dollars. This figure demonstrates the trends in total military history and heritage expenditures, by category for FYs 2009/10 to 2013/14. The data is summarized in Table 2.

Table 2. Military History and Heritage Expenditures in Millions of Dollars. This table indicates the trends of military history and heritage by category for FYs 2009/10 to 2013/14.

Aside from reduced yearly flying hours for the Snowbirds, the overall reductions were achieved by containing salary expenditures to only a three-percent increase (representing a seven-percent reduction in overall full-time equivalents over the period), and by severely reducing O&M costs by 38.7 percent. The O&M reduction was largely realized through significant reductions in the MDF.

Program expenditures by DHH and DH&R decreased by 32.4 percent from approximately $10.6 million to $7.2 million during the evaluation period. This decrease was primarily driven by a lower amount granted through the MDF and a significant reduction in staff. Note that the funding to the MDF can fluctuate greatly from year to year, as shown in Table 3.²⁷

The reduction in expenditures did not represent improved efficiency. One of the key outputs of the program is to deliver financial support to military museums. Reducing the amount of financial support to museums while maintaining the organization that delivers it largely intact (despite the slight reduction in full-time equivalents), reduces the program’s overall impact on its “history” component. Consequently, the reductions in expenditures have had a negative effect on the program deliverables, which are likely greater than the value of the reductions.

Table 3. MDF expenditures for FYs 2009/10 to 2013/14. This table shows the annual expenditures for the MDF for FYs 2009/10 to 2013/14.

One area that has seen a marked improvement in efficiency is the support to the GC commemorative ceremonial activities. Although performance data indicating the effort involved is lacking, the number of events supported over the evaluation period increased dramatically,²⁸ while the resources available to support them remained static. There is also no sign of a decrease in the workload over the next five years, as Canada enters into a busy time for the celebration of military history and heritage, with more 100^th anniversaries of First World War, and 75^th anniversaries of Second World War events. In fact this is more likely to cause an increase in workload for SJS, CJOC and DHH, who support ceremonies and commemorative events. There is also a very real possibility of a sharp increase in the workload of the CIP. Given the reduction in resources, DHH may not be able to sustain all of its mandated activities to the current output level if its workload suddenly increases in the coming years.

Annex A—Management Action Plan

Strategic Governance

Management Action

Action 1.1: CMP will identify requirements for a cohesive and long-term vision for Military History and Heritage. As a first step, CMP will work with the commands, consistent with existing orders and directives, to explore ways and means to enhanced performance information related to those activities. (March 2018)

Action 1.2: In December 2015, the CAF endorsed the Director Personnel Generation Requirements Music Branch Occupation Analysis recommendations for sustainable military music capability aligned to CAF/GC requirements. CMP will work with the CAF Music Branch Advisor and Supervisor of Music in collaboration with the commands/groups to implement these CAF-endorsed recommendations through the Military Employment Structure Implementation Plan to be released in June 2016. Concurrently, the CAF Music Branch Advisor and Supervisor of Music will continue to engage with the commands and SJS to advise them on the strategic employment of CAF ceremonial and musical capabilities. (September 2017)

OPI: CMP

Target Dates: March 2018; September 2017

Digitization, IM/IT and CMHG

Management Action

Action 2.1: CMP will work with departmental partners to develop a digital strategy to firmly ground its mandated acquisition, preservation and access functions in the digital era. This digital strategy will build on DHH transformation plans and reflect the DND/CAF and GC transition to an integrated digital environment. (September 2017)

Action 2.2: CMP will engage partners in the Military History Gateway to renew the MOU and the project charter and identify requirements to enhance its operation. (March 2017)

Action 2.3 In collaboration with CMP, key IM/IT enablers—ADM(IM) and Share Services Canada—will develop implementation plans for the most cost-effective solution to modernize the Military History Gateway that meets departmental commitments and GC requirements. (December 2017)

OPI: CMP

Target Dates: September 2017; March 2017; December 2017

Military Museums

Management Action

Action 3.1: As a means to achieve stable and predictable funding for both Vote 1 and Vote 5 submissions, CMP will work with CAF museums and departmental partners to forecast museum investment needs in combination with an investment plan that establishes priorities to resolve specific needs and problems. Stable and predictable funding managed by DHH in collaboration with the CAF Museum Committee (comprising heritage officers from the RCN, CA and RCAF), Assistant Deputy Minister (Infrastructure and Environment) and Non-Public Property would afford greater opportunity to address systemic/global museum issues and concerns. (March 2018)

Action 3.2: CMP will procure and manage the implementation of The Museum Software to replace CFAMS in all CAF museums. In collaboration with ADM(IM), DHH managed the testing of this software by ten CAF museums in January 2016. The results were positive. Through negotiation, future technical support will be provided through the museums’ chains of command by ADM(IM), Shared Services Canada and the vendor. DHH will maintain oversight to ensure the least intrusive implementation. (April 2017)

Action 3.3: CMP will build on its longstanding relationship with the CWM and explore the possibility of its expansion on behalf of the CAF museum system as a whole. CMP will also reach out to other national institutions like the Canada Aviation Museum where there is a commonality of interest and purpose. (March 2017)

OPI: CMP, ADM(IM)

Target Dates: March 2018; April 2017; March 2017

Annex B—Evaluation Methodology and Limitations

1.0 Methodology

1.1 Overview of Data Collection Methods

The evaluation findings and conclusions have been based on the objective analysis and triangulation of multiple lines of evidence (both quantitative and qualitative) using the following research methods:

• document review
• literature review
• key informant interviews
• financial analysis.

1.2 Details on Data Collection Methods

1.2.1 Document and Literature Review

Document review included stakeholder documentation specific to The Military History and Heritage program including MOUs, policies, directives and orders. In addition to program documents, other more general documentation that made specific reference to Military History and Heritage was reviewed as well, including departmental performance reports, the Speech from the Throne and documentation from other government departments. These documents identified key issues, trends and requirements that related to the relevance, effectiveness and efficiency of the program.

A literature review was conducted to identify issues and trends relating to Military History and Heritage. The review included external documents including academic research, public opinion research and media coverage.

1.2.2 Key Informant Interviews

Preliminary interviews were conducted with members of DHH and DH&R to obtain information on the core Military History and Heritage program and to identify the scope and key issues for the evaluation. These preliminary interviews also aided the creation of the logic model (Annex C) and evaluation matrix (Annex D).

Further interviews were conducted during the conduct phase of the evaluation. These interviews contributed to the relevance, performance, and economy and efficiency portions of the evaluation. Interviews were conducted with the following:

• DHH staff
• DH&R staff
• CJOC staff
• SJS staff
• ADM PA staff
• ADM IM staff
• command historians
• command heritage officers
• CAF senior officers
• CAF senior non-commissioned members
• representatives from VAC and DCH
• representatives from CAF museums
• representatives from the CWM.

1.2.3 Financial Analysis

The financial data for the Military History and Heritage program was reviewed in order to determine the degree of efficiency and economy of its activities and outputs. The data, covering five years from 2009/10 to 2013/14, was extracted from the PAA. Comptrollers at multiple Level 1 organizations were contacted to provide contextual information about the data. Financial data for Military History and Heritage in other countries was not conducted as the programs are considered to be too different for such a comparison.

2.0 Limitations

Table B-1 lists the limitations found during the evaluation and the mitigation strategies employed.

Table B-1. Evaluation Limitations and Mitigation Strategies. This table lists the limitations of the evaluation and the corresponding mitigation strategies.

Annex C—Logic Model

Figure C-1. Logic Model for the DEP. This flow chart shows the relationship between the program’s inputs, main activities, outputs and expected outcomes.

Starting Point A

Inputs:

• Policies, doctrine, directives, personnel, infrastructure, funding and equipment
• Requests for services (information, guidance, products, etc.) that require a promulgation

Activities (management/administrative activities):

• History and Archives (DHH 2)
  • Exercise intellectual control over the historical record of DND
  • Write “official,” popular and commemorative CAF histories
  • Support historical research
• Military Heritage (DHH 6)
  • Preserve, interpret and disseminate Canadian military heritage
  • Manage the Canadian Military History Gateway web portal
  • Conduct casualty identification for repatriation or interment
• CF Dress and Ceremonial (DHH 3)
  • Maintain CAF dress, insignia, customs and ceremonial policies
  • Assist VAC with CAF participation with pilgrimages and commemorations of Canada’s battles overseas
• CF Museums (DHH 5)
  • Collect, preserve, and interpret Canada’s military heritage to create a sense of history, identity and pride
  • Support the training and development of museology across CAF museums
• CF Music (DHH 7)
  • Music Branch functional authority
  • Select Musician candidates
  • Oversee training standards for CAF bands
• Directorate of Honours and Recognition (DH&R)
  • Administer CAF nominations with merit for specified bravery / gallantry decorations
  • Distribution of medals, decorations, honours and awards
• Elements/CMP
  • Demonstration units
  • Special task forces

Outputs: The first five activities lead to the following outputs of all activity areas:

• Policies and standards
• Publications
• Advice
• Support
• Tools

Immediate Outcomes:

• Canadian military history and heritage are preserved and promulgated
• Canadian military customs and traditions are preserved and promulgated
• The achievements, services and sacrifices of CAF organizations and members are recognized and honoured
• DND History, Heritage, Honours and Ceremonial activities are well governed (managed and supported)

Intermediate Outcomes:

• The proficiency, professionalism and sacrifice of the Canadian military are promoted, celebrated and recognized at home and abroad
• Military history, heritage, honours and ceremonial activities enhance the military profession and CAF operations

Strategic Outcome:

• DND/CAF reputation for reflecting and promoting Canadian values is established and maintained

Starting Point B

Inputs:

• Policies, doctrine, directives, personnel, infrastructure, funding and equipment
• Requests for services (information, guidance, products, etc.) that require a promulgation

Activities (management/administrative activities):

• History and Archives (DHH 2)
  • Exercise intellectual control over the historical record of DND
  • Write “official,” popular and commemorative CAF histories
  • Support historical research
• Military Heritage (DHH 6)
  • Preserve, interpret and disseminate Canadian military heritage
  • Manage the Canadian Military History Gateway web portal
  • Conduct casualty identification for repatriation or interment
• CF Dress and Ceremonial (DHH 3)
  • Maintain CAF dress, insignia, customs and ceremonial policies
  • Assist VAC with CAF participation with pilgrimages and commemorations of Canada’s battles overseas
• CF Museums (DHH 5)
  • Collect, preserve, and interpret Canada’s military heritage to create a sense of history, identity and pride
  • Support the training and development of museology across CAF museums
• CF Music (DHH 7)
  • Music Branch functional authority
  • Select Musician candidates
  • Oversee training standards for CAF bands

Output: The activity “CF Museums (DHH 5)” also leads to the following output:

• Grants

Immediate Outcome: The “Grants” output leads to the following immediate outcome:

• Canadian military history and heritage are preserved and promulgated

Intermediate Outcomes:

• The proficiency, professionalism and sacrifice of the Canadian military are promoted, celebrated and recognized at home and abroad
• Military history, heritage, honours and ceremonial activities enhance the military profession and CAF operations

Strategic Outcome:

• DND/CAF reputation for reflecting and promoting Canadian values is established and maintained

Starting Point C

Inputs:

• Policies, doctrine, directives, personnel, infrastructure, funding and equipment
• Requests for services (information, guidance, products, etc.) that require a promulgation

Activities (management/administrative activities):

• History and Archives (DHH 2)
  • Exercise intellectual control over the historical record of DND
  • Write “official,” popular and commemorative CAF histories
  • Support historical research
• Military Heritage (DHH 6)
  • Preserve, interpret and disseminate Canadian military heritage
  • Manage the Canadian Military History Gateway web portal
  • Conduct casualty identification for repatriation or interment
• CF Dress and Ceremonial (DHH 3)
  • Maintain CAF dress, insignia, customs and ceremonial policies
  • Assist VAC with CAF participation with pilgrimages and commemorations of Canada’s battles overseas
• CF Museums (DHH 5)
  • Collect, preserve, and interpret Canada’s military heritage to create a sense of history, identity and pride
  • Support the training and development of museology across CAF museums
• CF Music (DHH 7)
  • Music Branch functional authority
  • Select Musician candidates
  • Oversee training standards for CAF bands
• Directorate of Honours and Recognition (DH&R)
  • Administer CAF nominations with merit for specified bravery / gallantry decorations
  • Distribution of medals, decorations, honours and awards
• Elements/CMP
  • Demonstration units
  • Special task forces

Output: The activity “CF Music (DHH 7)” also leads to the following output:

• Qualified musicians/bands

Immediate Outcome: The “Qualified musicians/bands” output leads to the following immediate outcome:

• Canadian military customs and traditions are preserved and promulgated

Intermediate Outcomes:

• The proficiency, professionalism and sacrifice of the Canadian military are promoted, celebrated and recognized at home and abroad
• Military history, heritage, honours and ceremonial activities enhance the military profession and CAF operations

Strategic Outcome:

• DND/CAF reputation for reflecting and promoting Canadian values is established and maintained

Starting Point D

Inputs:

• Policies, doctrine, directives, personnel, infrastructure, funding and equipment
• Requests for services (information, guidance, products, etc.) that require a promulgation

Activity (management/administrative activities):

• Directorate of Honours and Recognition (DH&R)
  • Administer CAF nominations with merit for specified bravery / gallantry decorations
  • Distribution of medals, decorations, honours and awards

Output:

• Honours awarded

Immediate Outcomes:

• The achievements, services and sacrifices of CAF organizations and members are recognized and honoured
• DND History, Heritage, Honours and Ceremonial activities are well governed (managed and supported)

Intermediate Outcomes:

• The proficiency, professionalism and sacrifice of the Canadian military are promoted, celebrated and recognized at home and abroad
• Military history, heritage, honours and ceremonial activities enhance the military profession and CAF operations

Strategic Outcome:

• DND/CAF reputation for reflecting and promoting Canadian values is established and maintained

Starting Point E

Inputs:

• Policies, doctrine, directives, personnel, infrastructure, funding and equipment
• Requests for services (information, guidance, products, etc.) that require a promulgation

Activity (management/administrative activities):

• Elements/CMP
• Demonstration units
• Special task forces

Output:

• Events/shows

Immediate Outcomes:

• Canadian military customs and traditions are preserved and promulgated
• DND History, Heritage, Honours and Ceremonial activities are well governed (managed and supported)

Intermediate Outcomes:

• The proficiency, professionalism and sacrifice of the Canadian military are promoted, celebrated and recognized at home and abroad
• Military history, heritage, honours and ceremonial activities enhance the military profession and CAF operations

Strategic Outcome:

• DND/CAF reputation for reflecting and promoting Canadian values is established and maintained

Annex D—Evaluation Matrix

Table D-1. Evaluation Matrix—Relevance. This table indicates the data collection methods used to assess the evaluation issues/questions for determining the relevance of Military History and Heritage.

Table D-2. Evaluation Matrix—Performance (Effectiveness). This table indicates the data collection methods used to assess the evaluation issues/questions for determining the program’s performance in terms of achievement of outcomes (effectiveness).

Table D-3. Evaluation Matrix—Performance (Efficiency and Economy). This table indicates the data collection methods used to assess the evaluation issues/questions for determining the program’s performance in terms of efficiency and economy.

Annex E—Outreach Activities

Pillar 4 : Outreach to Canadians

The evaluation found that the DND/CAF have been focused for many years on enhancing the overall awareness and understanding of the role and mandate of the Canadian military within Canada and abroad; making the military more attractive to Canadians; broadening and deepening the connection between Canadian society and CAF members, military families and civilian DND employees; and fostering a better appreciation of the issues and challenges facing members of the Defence community. These objectives are far-reaching and in order to achieve them, the Department and the CAF deploy a variety of CAF assets to outreach events across the country.

The evaluation noted that that principal means of outreach are CAF bands and demonstration units. However, all Defence units, formations and assets can be, or have been, employed in such activities. Calculating the full costs associated with outreach and evaluating the impact and efficiency of these activities require an evaluation on its own and will be the subject of a future evaluation. This annex therefore presents information gathered in the course of the evaluation of the Military History and Heritage program.

The evaluation found that responsibility for significant portions of outreach activities also reside with many commanders, as shown in the following examples:

• The Chief of the Air Service for CF-18 demonstration aircraft, the Commander of the CA for the Skyhawk parachute team, and the Commander of the RCN for Her Majesty’s Canadian Ship Oriole;
• CAF bands, Regular and Reserve Force, are under the authority of the Commanders of the RCN, the CA and the RCAF, and the Vice Chief of Defence Staff (for the Central Band of the CAF); and
• Military history and heritage-themed strategic communications, advice, guidance, services and products in support of GC and Defence priorities are under the authority of the Assistant Deputy Minister (Public Affairs).

The reception, assessment, synchronization, capture of financial costs and tasking of resources to execute CAF participation in approved events (e.g., covered by CDS directives) domestically and internationally is accomplished by the collaborative work of SJS and CJOC.

With the exception of ceremonial events of national and international significance managed by SJS and CJOC, there are many activities that fall into the broad category of outreach. For example, each year the CAF engages or participates in promotional events across Canada and abroad. A principal attraction is the various demonstration teams, such as the CAF Skyhawks Parachute Team, the CF-18 Demonstration Team and the RCAF Snowbirds (431 Air Demonstration Squadron). Each year 431 Air Demonstration Squadron makes upwards of 50 to 60 appearances across Canada and the USA and conducts numerous “fly pasts” to audiences ranging in size from 5,000 to 150,000. With television coverage of some of these “fly pasts,” audiences can presumably be much larger. The public outreach impact for the Snowbirds emphasizes events on the ground and in the community, with the goal of establishing positive public perceptions of the CAF in general and the RCAF in particular.

As such, the Snowbirds are integrated into the general CAF outreach and recruiting effort. The RCAF links air demonstration to sustainment (i.e., recruitment) efforts. Promotional material produced for the Snowbirds is displayed and distributed at events attended by 431 Squadron and explicitly includes recruiting messages for the CAF. There is a liaison between the squadron’s public affairs effort and the Canadian Forces Recruiting Group in this regard. The intent is to deliver CAF recruiting messages to people who come out to see the Snowbirds, recognizing that the Snowbirds can draw very large crowds.

The actual impact and benefit of these events, as well as other Military History and Heritage program activities, are difficult to measure as they represent but one of many factors influencing people to join the CAF. Evidence gathered from Canadian Forces Recruiting Surveys²⁹ indicate that air shows, band performances, visits to military museums and solemn military commemorations (e.g., Remembrance Day) all play a role in attracting people to the profession of arms. Staff at CAF museums, particular those closely tied to units, have also indicated that they feel that they play a role in attraction, recruiting and retention.

The annual Canadian Forces Recruiting Surveys indicate that in over 50 percent of instances, the decision to join the CAF was strongly influenced by the impact of CAF air shows and other exhibits, highlighting the positive impact of such activities on recruiting. Recruitment aside, public opinion research is limited with respect to the impact of the Snowbirds, but what does exist establishes the link between the work of the demonstration units and opinions and attitudes among the public. It would appear that while they are significant drivers of positive public awareness for the CAF, they are one among several.

EKOS Research Associates Inc.’s Canadians’ Views of the CAF and Its Elements – 2012 Final Report noted that in response to an open question, “What image or impression comes to mind when you think specifically of the RCAF?,” 10 percent identified the Snowbirds. In 2009, 15 percent of the survey sample identified the Snowbirds in response to the same question. Air Shows were the image that came to mind for a further 5 percent of respondents in 2012, the same as in 2009.

Footnotes

___________________________________________________________________________________________________________________________

Footnote 1 TBS-approved 2009 version.

Footnote 2 Source: DHH.

Footnote 3 Source: DHH.

Footnote 4 Source: DH&R.

Footnote 5 Source: DHH

Footnote 6 DND/CAF Strategic Public Affairs Plan 2014-16. The message from the CDS and DM states: “All our future successes are directly attributable to the efforts, lessons and sacrifices of our past. Accordingly, in the coming years, commemoration activities will be an important commitment for Defence. We will salute Canada’s strong military tradition, including the centennial of the First World War, the 75th anniversary of the Second World War and the Canadian Armed Forces’ contributions in Afghanistan.”

Footnote 7 Coordinated CDS/Chief of the Land Staff/Canada Command Tasking Order: CF Ceremonial Support to Veterans Affairs Canada Commemorative Events 2011.

Footnote 8 The 2013-14 PCH Report on Plans and Priorities. http://www.pch.gc.ca/DAMAssetPub/DAM-verEval-audEval/STAGING/texte-text/rpp-2013-14_1364313938423_eng.pdf?WT.contentAuthority=18.0. Last consulted on June 27, 2016.

Footnote 9 Speech from the Throne to open the Third Session Fortieth Parliament of Canada (2010) http://www.parl.gc.ca/Parlinfo/Documents/ThroneSpeech/40-3-e.html. Last consulted on June 27, 2016.

Footnote 10 Speech from the Throne to open the First Session Forty-First Parliament of Canada (2011) http://www.parl.gc.ca/Parlinfo/Documents/ThroneSpeech/41-1-e.html. Last consulted on June 27, 2016.

Footnote 11 Speech from the Throne to open the Second Session Forty-First Parliament of Canada (2013) http://www.parl.gc.ca/Parlinfo/Documents/ThroneSpeech/41-2-e.html. Last consulted on June 27, 2016.

Footnote 12 No amounts were allocated in the 2014 Budget Plan http://www.budget.canada.ca/2014/docs/plan/pdf/budget2014-eng.pdf. Last consulted on June 27, 2016.

Footnote 13 As per the TBS Policy on Management of Materiel, “Collections of art, historical artifacts, archaeological artifacts and archival collections that are of artistic, historical, ceremonial, documentary, technological or associative importance and that are owned by federal departments (excepting those managed by Parks Canada under its legislative mandate). New objects of potential heritage value are also considered to be valid cultural property.” http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=12062&section=text#6. Last consulted on June 27, 2016.

Footnote 14 Section 2 of DAOD 5040-0, Military History and Heritage. http://www.forces.gc.ca/en/about-policies-standards-defence-admin-orders-directives-5000/5040-0.page. Last consulted on June 27, 2016.

Footnote 15 Allan D. English, Angus Brown and Paul Johnston. “Are we losing our memory?: Decision-making in DND,” section IV, page 473 of Proceedings of the Canadian Military History Conference 2000, http://www.cmp-cpm.forces.gc.ca/dhh-dhp/his/docs/cmhc_2000.pdf. Last consulted on June 27, 2016.

Footnote 16 See the Forward to the Constitution of the National Air Force Museum (2011) for an example. http://airforcemuseum.ca/en/uploads/2010/08/Constitution.pdf. Last consulted on June 27, 2016.

Footnote 17 CFAO: Canadian Forces Administrative Order.

Footnote 18 Canadian Forces Museums – Operations and Administration, Part 1. https://www.cfmws.com/en/AboutUs/Library/PoliciesandRegulations/Finance/Documents/Museum%20Ops%20and%20Adm%20Manual%20CFP%20266_b.pdf. Last consulted on June 27, 2016.

Footnote 19 Preface to The Canadian Forces’ Decoration, January 12, 2011. http://www.cmp-cpm.forces.gc.ca/dhr-ddhr/pub/cfd-dfc-eng.asp. Last consulted on June 27, 2016.

Footnote 20 Chapter 7 – Anthems, Marches and Calls. May 15, 2008.

Footnote 21 Ibid. The principles and eligibility criteria for the award of Canadian Forces Battle Honours as amended by the Battle Honours Committee on October 18, 2011.

Footnote 22 Acronyms in Figure 1 are as follows: UNEF II = Second United Nations Emergency Force, PPCLI = Princess Patricia’s Canadian Light Infantry, R22eR = Royal 22nd Regiment, RNCVR = Royal Naval Canadian Volunteer Reserve, UK = United Kingdom, NATO = North Atlantic Treaty Organization, UNESCO = United Nations Educational, Scientific and Cultural Organization, CCTM-A = Canadian Contribution to the Training Mission in Afghanistan, UN = United Nations, CFB = Canadian Forces Base, UNEF I = First United Nations Emergency Force, POW = Prisoner of War, WRCNS = Women’s Royal Canadian Naval Service, NHL = National Hockey League, VE = Victory in Europe, VJ = Victory in Japan.

Footnote 23 Task #170, March 31, 2015. Human Resource Systems Group, Ltd. “Organizational Strategies for Member Identity, Loyalty, and Commitment: Understanding Current and Future Impacts of Military Customs and Traditions.” Produced for Director General Military Personnel Research and Analysis.

Footnote 24 DAOD 5041-1 – Canadian Forces Music Support. http://www.forces.gc.ca/en/about-policies-standards-defence-admin-orders-directives-5000/5041-1.page. Last consulted on June 27, 2016.

Footnote 25 DAOD 5041-0 – Canadian Forces Music. http://www.forces.gc.ca/en/about-policies-standards-defence-admin-orders-directives-5000/5041-0.page. Last consulted on June 27, 2016.

Footnote 26 To ensure consistency in trending and costing methodologies, for FY 2009/10, the amount of military pay expenditures attributed to PAA 4.2.2.0 (History, Protocol, and Heritage Ceremonial Activities) was assumed equivalent to FY 2010/11.

Footnote 27 Source: Program staff and the Defence Resource Management Information System.

Footnote 28 Source: Interviews, document review and media coverage.

Footnote 29 This is a voluntary survey administered in CAF recruiting centres and detachments to prospective recruits at the end of the recruiting process . It assesses satisfaction with recruitment services and recruitment materials, factors influencing prospective recruits to contact the CAF for employment