Evaluation of Canada’s Membership in the International Holocaust Remembrance Alliance
2. Methodology
2.1 Evaluation scope and data collection methods
The evaluation examined Canada’s participation in and contribution to the IHRA, including the extent to which Canada’s experience in building an integrated and socially cohesive society has influenced the IHRA, and how membership in the IHRA has affected Holocaust education, remembrance and research in Canada. In particular, the evaluation assessed the extent to which Canada met the objectives for its Chair Year. The evaluation did not examine the IHRA itself, the specific activities funded by the IHRA, or the program initiatives undertaken in Canada.
In accordance with the requirements of the Treasury Board Secretariat (TBS) Directive on the Evaluation Function,Footnote 17 the evaluation assessed the relevance and performance of Canada’s membership in the IHRA from 2009-10 to 2013-14. As shown in the logic model, the expected outcomes of Canada’s membership in the IHRA are that:
- Canadians are aware of the Holocaust, understand its causes and reflect on its consequences;
- Canada's approach to building an integrated and socially cohesive society is conveyed internationally; and
- Canada demonstrates national and international leadership in Holocaust education, remembrance and research.
Table 3: Summary of evaluation issues and questions
Relevance – Continued Need
- Is there a continued need for Canada to belong to the IHRA?
Relevance – Alignment with Government Priorities
- Is membership in the IHRA aligned with departmental and government-wide priorities?
Relevance – Alignment with Federal Roles and Responsibilities
- To what extent is membership in the IHRA aligned with federal roles and responsibilities?
Performance – Achievement of Expected Outcomes
- How has Canada’s membership in the IHRA affected Holocaust education, remembrance and research in Canada?
- Has awareness and understanding of the Holocaust and its causes and consequences improved in Canada?
- To what extent did Canada meet its objectives for its Chair Year?
- How has Canada contributed to the IHRA?
- How has Canada’s approach to building an integrated and socially cohesive society been conveyed internationally through the IHRA? Has it influenced the IHRA?
- To what extent has Canada’s membership in the IHRA helped demonstrate Canada’s leadership in Holocaust education, remembrance and research in Canada and internationally?
- Has Canada’s membership in the IHRA had any unintended consequences (positive or negative)?
Performance – Demonstration of Efficiency and Economy
- Could Canada contribute to the IHRA more effectively/efficiently?
- Are there alternative ways in which Canada could participate in and support Holocaust education, research and remembrance?
The methodology and level of effort for the evaluation was calibrated in recognition of its low complexity, and the relatively low level of materiality of the contribution. As a result, the evaluation relied on three lines of evidence to examine the relevance of Canada’s IHRA membership and performance relative to intended outcomes over the five-year period from 2009-10 to 2013-14.
Interviews: A total of 15 interviews were completed for the evaluation and were undertaken with three key stakeholder groups including: experts and Canadian IHRA delegates (9), CIC staff (5) and IHRA staff (1). Interviews were conducted both in-person and by telephone. Interview guides were developed for each stakeholder group and available in the technical appendix.
Document review: Documents reviewed included CIC documents (plans, reports and foundation documents), government documents (legislation, plans, reports, budgets, Speeches from the Throne and press releases), and speaking notes for the Minister. A complete list of documents reviewed is available in the technical appendix.
Literature review: Published literature on trends in hate crimes, anti-Semitism and awareness and understanding of the Holocaust was reviewed. In addition to an internet search for literature, expert interviewees were asked to identify relevant literature.
2.2 Limitations and mitigation
Due to the relatively low level of materiality of the contribution and its low complexity, the level of effort and approaches for the evaluation were calibrated accordingly. As a result, it relied on three qualitative lines of evidence and relatively few key informant interviews, which made it difficult to determine the significance and importance of interviewee perspectives in some cases. However, a variety of stakeholder perspectives were represented, and as a result, these limitations did not have an impact on the findings.