Evaluation of the Multiculturalism Program

Executive summary

Purpose of the evaluation

This report presents the findings of the evaluation of Citizenship and Immigration Canada’s (CIC) Multiculturalism Program. The evaluation of the Multiculturalism Program was designed to address three broad themes: relevance, design and delivery, and performance. In keeping with the requirements of the Directive on the Evaluation Function (Treasury Board of Canada Secretariat, 2009), program relevance was assessed in terms of: (1) continued need; (2) consistency with respect to federal roles and responsibilities; and (3) alignment with government and departmental objectives and priorities. Program performance was assessed by examining program results in terms of: (4) effectiveness; and (5) efficiency and economy.

The evaluation was conducted by CIC’s Research and Evaluation Branch (R&E) between April and October 2011.

The Multiculturalism Program

Canada has a long history of multiculturalism programming, which is founded in the 1971 Multiculturalism Policy and the 1988 Multiculturalism Act. In July 2009, Cabinet approved three new objectives for the Multiculturalism Program, which were formally implemented on April 1st, 2010:

  1. to build an integrated, socially cohesive society;
  2. to improve the responsiveness of institutions to meet the needs of a diverse population; and
  3. to actively engage in discussions on multiculturalism and diversity at an international level.

The Multiculturalism Program is comprised of four key components: provide funding to organizations to undertake multiculturalism projects and events (called Inter-Action) supporting the three program objectives, undertake public education and promotion initiatives, provide support to federal institutions, and conduct international engagement activities. The Multiculturalism Program is the responsibility of a number of sectors and branches within CIC.

Methodology

The evaluation used multiple lines of quantitative and qualitative evidence to ensure the strength of results:

  • interviews;
  • project and event participant feedback forms;
  • project evaluations;
  • telephone survey with project funding recipients and non-recipients;
  • administrative data review;
  • Multiculturalism Champions Network questionnaire;
  • literature review; and
  • document review.

The scope of the evaluation included activities undertaken, and outputs produced, between fiscal years 2008-09 and 2010-11. Consequently, the Gs&Cs projects reviewed for the evaluation included some that had been funded under both the old and new program objectives.

Limitations

Although the evaluation included a good balance of quantitative and qualitative lines of enquiry, and allowed for the triangulation of results, there were four key limitations to the methodology, which should be considered when reviewing the evaluation results.

  • There are inherent challenges with measuring the outcomes of social programs, particularly with respect to the attribution of outcomes.
  • There is little on-going performance measurement in place to gather information on project outcomes. Consequently, the evaluation relied on data collected at the time of the evaluation in order to assess program outcomes.
  • Where data are available with respect to outcomes, they are not representative of all program stakeholders (i.e., all multiculturalism champions, all project funding recipients and non-recipients), nor are they representative of all projects and events funded.
  • The evaluation was conducted one year following the implementation of new program objectives and therefore only a few projects funded under the new objectives had been completed at the time of the evaluation.

Evaluation findings

The main findings associated with each of the evaluation questions are presented below.

Relevance

  • Given the increasing ethnocultural and religious diversity of the Canadian population, and the continued presence of prejudice, racism and discrimination in Canadian society, there is a need for multiculturalism programming in Canada. The Multiculturalism Program’s approach, which facilitates interaction among different communities in order to increase mutual awareness and understanding, has been found by a variety of academic research to be an effective means to promote social cohesion.
  • Multiculturalism programming, with its basis in federal legislation, is clearly aligned with federal roles and responsibilities, although provinces, municipalities and other organizations such as non-profit and businesses also have a complementary role to play. The federal role, according to interviewees, is to provide leadership, promotion and education in relation to multiculturalism, and to support the delivery of consistent and best practices across the country.
  • CIC’s program activity architecture and strategic goals have been revised to include multiculturalism programming. However, the way in which this addition will influence, or be influenced by, other programs and services has not yet been fully determined. The majority of interviewees did not think that multiculturalism is a federal priority. Some key federal government documents, such as Speeches from the Throne, refer to diversity, but do not identify multiculturalism programming as a policy priority.

Design and delivery

  • While the program objectives were modified slightly in 2010, the program activities and target groups remained largely the same as under the previous objectives. Some notable changes were made with respect to the delivery of the grants and contributions component, including the implementation of a call for proposals (CFP) process with associated tools, and the addition of an events stream.
  • Program responsibilities are shared among many sectors, branches, directorates and units, and there have been reorganizations of the program since its transfer from Canadian Heritage to CIC in October 2008. This has made effective program governance a challenge, particularly with respect to communication, coordination and shared decision-making. There is a lack of clarity with respect to the responsibilities of the various units involved in the Multiculturalism Program and some decisions have been undertaken without appropriate input from both the policy and program units.
  • The new CFP process added consistency and transparency to the way in which project priorities were defined and proposals were assessed, which also brought the Multiculturalism Program in alignment with other CIC Gs&Cs programming. However, due to the intentional broadness of the language in the CFP, the dollar value of applications received far outweighed the funds available for projects. In addition, the approval process was found to be lengthy and not sufficiently transparent.
  • An appropriate performance measurement strategy has not been put in place to collect data on an on-going basis and available performance measurement data are largely at the output level. Some of these output-level data are incomplete, inconsistent, and unreliable.

Performance

  • The Gs&Cs and public education components of the Multiculturalism Program have the intended outcomes of increasing participants’ civic memory and pride, respect for core democratic values, and intercultural / interfaith understanding. While information related to the actual outcomes of projects and events is limited, participant feedback from two projects funded under the new objectives suggests that the program is contributing to these outcomes. Public education and promotion initiatives have been widely promoted using a variety of methods and there appears to have been public interest in these initiatives; however, data related to the achievement of the broader intended outcomes is not currently collected for these activities.
  • The second intended outcome of the Multiculturalism Program is that the programs, policies and services of federal institutions are responsive to the needs of a diverse society. The related program activities are highly administrative in nature and, while they meet the reporting requirements of the Multiculturalism Act, they are inadequate to bring about this expected outcome. This issue is exacerbated by the limited resources dedicated to this program component.
  • Canada has shared best practices related to multiculturalism internationally. However, there was limited evidence on whether any information gained internationally is used by CIC or shared with other federal institutions.
  • The overall efficiency of the Multiculturalism Program has been affected by the length of time taken to make decisions on project proposals. Consequently, the program lapsed funds in each of the years covered under the scope of the evaluation, although the amount of these lapses has diminished significantly each year, from 75% of the budget allocation in 2008/09, to 37% in 2010-11. This is expected to be further reduced, to approximately 23%, in 2011-12.

Conclusions

  • There is a need for multiculturalism programming in Canada and the federal government has a role to play in that programming. There is substantial academic research to support the approach used by the Multiculturalism Program to promote an integrated society.
  • CIC is, in many ways, the appropriate department to assume the lead for federal responsibilities related to multiculturalism. However, inclusion of the Multiculturalism Program within CIC has broadened the departmental mandate (to include longer-term integration issues), and clientele (to comprise all Canadians). The impact this will have on CIC policies and programs has yet to be determined.
  • The Multiculturalism Program objectives are very broad. While this means they are sufficiently flexible to allow the program to be responsive to the needs of different communities, this breadth also results in a lack of focus with respect to the types of activities that might best support the program objectives. These objectives are also larger than what can reasonably be achieved, given current program resources and activities.
  • There are three key factors with respect to the design and delivery of the program that have hindered its successful implementation. These include governance, performance measurement, and the approval process:
    • insufficient communication, coordination and shared decision-making between the different organizational units responsible for the program;
    • a lack of basic performance measurement data, with which to assess how well the program as a whole, or individual projects and events, are performing; and
    • the timeliness and lack of transparency of the approval process.
  • Given the challenges with performance measurement, there is currently limited evidence to demonstrate to what extent the Multiculturalism Program is achieving its expected outcomes. There is some recent performance measurement information to suggest that projects are having a positive impact with respect to increased civic memory and pride, respect for core democratic values, and intercultural / interfaith understanding.
  • The overall efficiency of the program has been affected by the length of time it has taken to make decisions on project proposals. Consequently, the program lapsed a substantial amount of Gs&Cs funding in 2008-09 and 2009-10, although the amount lapsed diminished in 2010-11 and is expected to decrease further in 2011-12. The fact that program resources were not fully utilized limits the potential impact of the program.

Recommendations

  1. Given that the Multiculturalism Program has broadened CIC’s mandate (to include longer-term integration) and its clientele (to include all Canadians), CIC should ensure that multiculturalism is fully integrated into CIC policies and programming.
  2. With the relatively small amount of funding available for CIC’s Multiculturalism Program, the objectives and expected outcomes of the program need to be better aligned with available resources and strategically focused on core priorities and needs. The department needs to assess how best it can do this.
  3. Further efforts are required to improve the transparency and timeliness of the approval process for projects and events.
  4. The governance for the Multiculturalism Program needs to be improved to support better communication and coordinated decision-making among the responsible branches and units for the program.
  5. Given the issues identified with respect to performance measurement, the program needs to implement a robust performance measurement strategy. This will require:
    • a review of, and possible revisions to, the performance measurement strategy framework developed during the planning phase for this evaluation;
    • improvements to the present data collection system;
    • a review of the current requirement for funding recipients to submit a project evaluation, to determine how it can be used to compile consistent and comparable data on CIC’s program outcomes; and
    • implementation of a process for ensuring that the project and event feedback forms remain up-to-date, and are regularly compiled and analysed to assist with the assessment of project and event outcomes.

Page details

Date modified: