Language selection

Government of Canada / Gouvernement du Canada

Search

Evaluation of Canada’s Action Plan Against Racism

3. Evaluation findings

This section presents the findings of the evaluation, organized by the three major evaluation areas (relevance, design and delivery, and performance) and by the major identified themes.

3.1. Relevance

3.1.1. Continued need for initiatives to combat racism and discrimination

Finding: Evidence shows that diversity in Canada will continue to increase and that intolerance remains an issue for Canadians. As such, the rationale for initiatives to combat racism and discrimination has not changed since CAPAR began.

Context and prevalence of racism and discrimination in Canada

Canada is home to more than 200 ethnic groups, with 16% of its population (over five million individuals) identifying as a visible minority Footnote 8. Information from the Census showed that Canada’s visible minority population grew 27% from 2001 to 2006, five times faster than the population as a whole Footnote 9. The diversity of Canada’s population is expected to continue to increase over the next two decades Footnote 10. According to projections by Statistics Canada (STC), up to 14.4 million Canadians (or about one-third of the population) will be members of a visible minority by 2031 Footnote 11. The religious composition of the country is also changing, with some of the largest increases seen in Muslim, Hindu, Sikh and Buddhist denominations Footnote 12.

Within the context of these demographic changes, there are signs of intolerance. The 2002 Ethnic Diversity Survey (EDS) found that nearly one-half of Blacks and one-third of South Asian and Chinese respondents reported having been the victim of discrimination and unfair treatment in the previous five years Footnote 13. Another source found that almost one-quarter of visible minority workers reported having experienced racial harassment or discrimination in the workplace Footnote 14. A 2003 Ekos survey found that 46% of Aboriginal people living off-reserve felt they had been a victim of racism or discrimination over the previous two years Footnote 15. A 2005 poll from Ipsos-Reid showed that Canadians believe the groups most targeted by racism are Muslim/Arabs, Aboriginal people, Blacks, and East Indians Footnote 16. A more recent poll undertaken by Angus Reid Strategies (2009) found that some Canadians hold unfavourable views towards some non-Christian religions, including Islam and Sikhism Footnote 17.

Further, research on hate-motivated crimes has shown that groups most at risk of being victimized by hate and bias activity were racial/ethnic minorities and religious minorities Footnote 18. In 2008, police services in Canada reported 1,036 hate-motivated crimes, with 55% motivated by race/ethnicity and 26% motivated by religion Footnote 19.

Studies also demonstrate that Aboriginal people, visible minorities and immigrants are particularly vulnerable to unemployment, underemployment, lower incomes and social segregation Footnote 20. A 2004 study found that the wage gap between visible minorities and the rest of the Canadian population had increased from 11% to 14.5% in the ten years from 1991 to 2000 Footnote 21. Among Canadian-born men, the three largest visible minority groups—Blacks, Chinese and South Asians—had significant wage gaps compared to their Caucasian counterparts Footnote 22. The 2006 Census revealed that the unemployment rate of visible minority workers with university degrees in 2006 stood at 7.4% compared to 3.7% among non-visible minority workers with university degrees Footnote 23. The incidence of poverty among immigrants, almost half of whom are visible minorities, has been shown to be increasing in Canada Footnote 24.

Interviewed stakeholders were also all of the opinion (24 of 24) that there is a need for initiatives to counter racism and discrimination. Reasons given included that racism exists in Canada (15 of 24) and because of the trend of increasing diversity in Canada (8 of 24). The evaluations of the CAPAR-funded initiatives also cite these issues in the context of examining their own continued need. The various evaluation reports highlight issues such as Canada's increasing diversity; evidence of continued intolerance/discrimination against members of visible minority groups; and the significant labour/workforce inequalities faced by Aboriginal people and members of visible minorities.

3.1.2. Alignment with federal roles and responsibilities

Finding: CAPAR is aligned with federal legislative responsibilities articulated in the Canadian Charter of Rights and Freedoms, the Canadian Human Rights Act, the Multiculturalism Act, and the Employment Equity Act. CAPAR also aligns with international initiatives and GoC commitments.

Alignment with federal legislative responsibilities

Canada has a long history of policies on human rights, equality and multiculturalism (as previously shown in Figure 1). A review of documents showed that CAPAR is aligned with federal roles and responsibilities. More specifically, CAPAR and its funded initiatives support federal legislative responsibilities such as the Canadian Charter of Rights and Freedoms, the Canadian Human Rights Act, the Multiculturalism Act, and the Employment Equity Act. These pieces of legislation call for departments to place emphasis on inclusion, equality, and access for all Canadians. Most interviewees (16 of 18) agreed that CAPAR was aligned with federal responsibilities, citing these legislative requirements. Note that the two remaining interviewees (both Multiculturalism Champions) were uncertain with respect to this alignment.

The CAPAR-funded program evaluations also noted that the initiatives were consistent with federal roles, suggesting that the federal government is well-placed to provide coordination and leadership on anti-racism initiatives. For example, the Data Collection Strategy is well situated within the federal government, which can coordinate data compilation and other activities involving all provinces and across the various jurisdictions (e.g., municipal, provincial, First Nation, federal).

The evaluations also provide examples of how the individual CAPAR-funded initiatives link to federal responsibilities, including:

  • CIC is responsible for administering the Multiculturalism Act and is also responsible for assisting in the integration of newcomers, so it is appropriate that it has responsibility for WCI;
  • RFWS focuses on employers covered by the Employment Equity Act, the Federal Contractors Program and the Legislated Employment Equity Program and, therefore, aligns well with federal government roles and responsibilities in relation to the discrimination faced by members of visible minorities and Aboriginal peoples in the workplace (as per the Employment Equity Act).

Note that, prior to the program’s cancellation, the evaluation of the Law Enforcement and Aboriginal Diversity (LEAD) initiative had found that that program was not consistently aligned with federal roles and responsibilities, particularly with respect to its police training component.

The evaluation found that there does not appear to be any other federally coordinated efforts to combat racism. This is supported by a review completed by the Citizenship and Multiculturalism Branch on the status of anti-racism efforts, which found that there was little other federal programming specifically on anti-racism. Federal programs were more focused on diversity, ethno-cultural programming, anti-discrimination, and diversity. The evaluation identified other initiatives in place to counter racism and discrimination in both provincial and municipal jurisdictions (e.g., Stop Racism Program in Ontario Schools, Manitoba’s 2-Way Street Program). At this time, it appears that there is little coordination between the federal, provincial, and municipal programs.

Alignment with international initiatives and commitments

CAPAR is in alignment with numerous international initiatives and commitments. For example, the UN has organized three WCARs, which were held in 1978, 1983 and 2001. Prior to the 2001 WCAR, the UN noted that the goals of previous conferences had not been attained and emphasized the importance of action by attending states. Canada was a participant at the 2001 WCAR in Durban, South Africa where the Durban Declaration and Programme of Action (DDPA) was adopted by consensus, and called for member states to implement a national action plan against racism.

In addition, Canada is a signatory to ICERD. The Convention aims to eliminate racial discrimination and promote tolerance by requiring its parties to outlaw hate speech and criminalize membership in racist organizations Footnote 25. Canada has a commitment to report to the UN on its progress on the implementation of ICERD. CAPAR also responded to the UN Special Rapporteur’s report on contemporary forms of racism, racial discrimination, xenophobia and related intolerance, based on his visit to Canada in 2003 Footnote 26. The report recommended that a national programme against racism based on the DDPA be launched in a coordinated and coherent framework.

Finally, Canada is also a participating state of the Organization for Security and Co-operation in Europe (OSCE), in which the Office for Democratic Institutions and Human Rights (ODIHR) works with members to combat intolerance and discrimination. All of these international initiatives were also raised by a few interviewees (4 of 18) and in the initiative evaluations, which particularly noted Canada’s commitments under ICERD.

3.1.3. Alignment with federal and departmental priorities

Finding: At the time of the inception of CAPAR, the issue of combating racism was a clear GoC priority. Since that time, stated priorities have evolved to focus on furthering social cohesion and access to economic opportunity. CAPAR is in alignment with the priorities of the funded departments.

Alignment between CAPAR and GoC priorities

Evidence shows that the issue of anti-racism and discrimination was a clear priority for the GoC in the early- to mid-2000s. Between 2000 and 2001, in preparation for WCAR, the GoC carried out extensive consultations with national organizations and institutions in civil society, various levels of government, Aboriginal organizations, and human rights commissions. In the October 2004 Speech from the Throne, the GoC pledged to "take measures to strengthen Canada's ability to combat racism, hate speech and hate crimes” Footnote 27. The GoC’s commitment to the issue was further demonstrated in the 2005 Federal Budget, which included a five-year investment of $56 million for CAPAR Footnote 28.

More recent speeches from the throne suggest the GoC has shifted its priorities from anti-racism and discrimination, specifically, to furthering social cohesion and access to economic opportunity. For example, the November 2008 Speech from the Throne committed the GoC to reducing “barriers that prevent Canadians from reaching their full potential” and ensuring that “all Canadians share in the promise of this land, regardless of cultural background, gender, age, disability or official language.” Footnote 29 The March 2010 Speech from the Throne stated that Canada demonstrated that “people drawn from every nation can live in harmony.” Footnote 30

This is supported by CIC and CAPAR-funded program interviewees, many of whom suggested that CAPAR (i.e., anti-racism initiatives specifically) was no longer a priority for the government (2 of 10) or that the priorities of the government had evolved over time (8 of 10). The latter interviewees suggested that current GoC priorities have shifted from a focus on anti-racism, to anti-Semitism, social cohesion, and supporting communities and families.

Alignment between CAPAR and departmental priorities

The objectives of CAPAR remain in alignment with the objectives of CIC’s Multiculturalism Program, which, as of 2009, include: building an integrated, socially cohesive society; and improving the responsiveness of institutions to the needs of a diverse population. While CAPAR’s objectives are specifically focused on reducing racism and discrimination, this is congruent with the Multiculturalism Program’s overall focus on social cohesion, equality of opportunity, and responsiveness to diversity in the public sector.

All CIC and CAPAR-funded interviewees (11 of 11) indicated that CAPAR is aligned with their departmental objectives, particularly with respect to integration, equality, and removing barriers to participation (8 of 11) and to anti-racism (5 of 11). All Multiculturalism Champions who responded to this question (5 of 5) also believed that CAPAR was aligned with their departmental priorities.

Information from the initiative evaluations suggested that the funded initiatives were consistent with the mandates/priorities of the responsible departments. For example: CIC is responsible for assisting in the integration of newcomers, which aligns with the objectives of WCI; and RFWS complements HRSDC's employment equity programs. The DoJ evaluation report similarly noted that addressing race-based issues in the justice system is a departmental priority.

3.2. Design and delivery

CAPAR was the first federal initiative on racism and anti-discrimination to employ a horizontal approach. In order to assess the effectiveness of its design and delivery, the evaluation examined different elements of CAPAR in the context of findings from the literature review on effective practices and success factors in horizontal initiatives.

3.2.1. CAPAR design

Finding: CAPAR was intended to be a horizontal initiative encompassing many government departments and agencies, building on existing programs and filling programming gaps. The design of CAPAR, however, did not meet this intent. More specifically, the original rationale for managing CAPAR as a horizontal initiative was not clear and the actual depth and breadth of CAPAR’s horizontality was limited, with only the nine CAPAR-funded initiatives included in any joint activities, and with collaboration largely focused on performance measurement and basic information-sharing. Furthermore, the limited focus of the CAPAR design on the funded initiatives was not aligned with the nature of racism as a broad-based issue, and appears to have resulted in a lack of cohesion among the programs for the horizontal initiative.

Origin of CAPAR and intended design

As discussed in section 3.1.2 (Alignment with federal roles and responsibilities) CAPAR was developed in response to a number of international events. In addition to these events, information from the RMAF/RBAF indicates that a number of other events and research fed into the design of CAPAR. The consultations completed prior to WCAR identified a need for domestic action to combat racism. A national forum on policing identified the need for policing authorities to incorporate diversity in their policies and programs and the need to establish rigorous data collection on hate crime. Information from the 2001 Census and the 2002 EDS confirmed the scope of racism in Canada. The Multiculturalism Program consulted departments to identify measures undertaken to combat racism and significant gaps in federal programming. Finally, CAPAR was developed in response to the UN Special Rapporteur’s 2004 report that called for Canada to develop an action plan.

The foundation documents for CAPAR advanced this work and committed the GoC to developing a coordinated, horizontal approach across federal departments to strengthen social cohesion by addressing significant gaps in key sectors of federal programming. Further documents note that the Action Plan is a deliberate and concerted effort by the GoC to develop a coordinated framework to combat racism and build on existing measures that strengthen Canadian citizenship. It was organized around six key priority areas:

  • Assist victims and groups vulnerable to racism and related forms of discrimination;
  • Develop forward-looking approaches to promote diversity and combat racism;
  • Strengthen the role of civil society;
  • Strengthen regional and international cooperation;
  • Educate children and youth on diversity and anti-racism; and
  • Counter hate and bias.

As the lead department, PCH was tasked with establishing a strong coordination and monitoring framework to measure progress on the implementation of the Action Plan and to ensure that existing gaps in the federal framework to combat racism are effectively addressed in a timely manner. The Action Plan itself characterized CAPAR as a collaborative effort aimed at eliminating racism in Canada (seeking) to enhance policies, programs and actions across federal departments.

This suggests that the GoC’s intention was to develop a broad-based, all encompassing action plan, involving many different departments and agencies and aimed towards a wide variety of target audiences. CAPAR’s subsequent design, however, did not meet these intentions. The 2006 Evaluation Assessment of CAPAR acknowledged that “although CAPAR has a horizontal vision, this is not easily or well-translated into a horizontal program design.” Footnote 31 The following sections provide further evidence that the design of CAPAR did not match its intent.

Rationale for managing CAPAR as a horizontal initiative

As noted in the literature review, the need for horizontal coordination is not new. Given that the government is structured into separate departments and agencies, there is a need to find ways to address, in a coordinated manner, policy issues that cross departmental boundaries. The literature identified a number of reasons for managing horizontally, which include: promoting policy coherence; promoting cost-effective program and policy interventions; addressing the needs of particular groups in an integrated manner; addressing local and regional issues in an integrated manner; improving service delivery; improving operational efficiency; building capacity; supporting a more unified government approach; and improving the image of government Footnote 32.

The original rationale for managing CAPAR as a horizontal initiative was not clearly articulated in program documentation. While the plan itself refers to CAPAR as a “collaborative effort” and foundation documents refer to CAPAR as a “horizontal approach,” the available documents do not describe the precise rationale for designing CAPAR as a horizontal initiative. Interviewees most often suggested that CAPAR was designed as a horizontal initiative because anti-racism and discrimination is an issue that cuts across many departments, thus requiring federal coordination (9 of 13). Some interviewees also suggested that Canada’s international obligations with respect to anti-racism required a coordinated approach (4 of 13), or that horizontality was a trend in government management at the time of CAPAR’s development (2 of 13). Overall, these comments suggest a horizontal approach was implemented to support a more unified government approach—one of the reasons suggested in the literature for implementing horizontal initiatives—but this is not definitively clear.

The lack of a clearly articulated rationale for horizontal management likely presented a challenge to successful horizontal management from the Initiative’s outset. As concluded in the literature review, “a sound understanding of the rationale and benefits expected from the [horizontal] initiative will […] help to determine the areas where collaboration is most needed, which organizations need to be involved in each case and the depth of collaboration required.” Footnote 33

Nature and scope of collaboration

The literature review identified a number of typologies of horizontal initiatives, ranging from informal to formal. CAPAR was analysed to determine where it was situated within these typologies, using in particular the Organization for Economic Cooperation and Development’s Policy Coordination Scale (1998), and Consulting and Audit Canada’s Continuum of Escalating Options (2002) (see Appendix G for the full assessment) Footnote 34. The analysis concluded that along a continuum of informal to formal, CAPAR can be classified as a relatively informal initiative. Its horizontality was limited to:

  • addressing elements related to communication, consultation, dialogue and information- sharing with other departments;
  • ensuring that the government spoke with one voice on the issue; and
  • undertaking joint projects in the areas of performance measurement and reporting.

The only vehicle for inter-departmental collaboration in CAPAR was the IWG. The Terms of Reference (ToR) developed by the IWG outlined four roles for the group: performance measurement, reporting, evaluation, and information exchange. Information from the interviews confirmed that the IWG’s role remained focused on information exchange and integrated performance measurement and reporting. Neither the foundation documents for the horizontal design, nor the ToRs for the IWG included any reference to other forms of collaborative activities, such as joint planning and priority setting, joint research projects, exchange of expertise, mutual strategic adjustment or joint implementation. Furthermore, the IWG was not designed to include such formal elements as central decision-making and policy setting—elements which would have been more in-line with a more formal horizontal approach. Based on the foundation documents for CAPAR and its intentions, it can be concluded that CAPAR was intended to be a more formal horizontal approach; however, it evolved into something more informal.

It was suggested in interviews that the depth of inter-departmental collaboration could potentially have been expanded, such as through joint development of publications, further developing of working relationships, and joint creation of a communications strategy to raise the profile and visibility of CAPAR.

Participation in the IWG was limited to the nine (and ultimately fewer) CAPAR-funded initiatives. Although the original action plan listed over 50 initiatives in CAPAR, there was no involvement in the initiative’s horizontality by the initiatives not directly funded by CAPAR. It was suggested in interviews that the breadth of coordination could have been increased by including more departments in the IWG. However, it was noted that involving all of the CAPAR initiatives originally listed as part of CAPAR was likely unrealistic and that further clarification of terminology and objectives would have been needed to determine the most appropriate members. More generally, it was also suggested that the level of collaboration in CAPAR could have been improved by actively involving other levels of governments, community groups and NGOs.

Cohesiveness

The literature review identified the degree to which the various initiatives demonstrate some form of overall cohesion as one of the factors that can aid or impede the success of a horizontal initiative. The component parts of a horizontal initiative should respond to a specific policy or operational issue in a coherent manner. The design of CAPAR lacked this cohesion. The evidence from the evaluation suggests that this was the result of two factors: the choice of initiatives included in the horizontal management of CAPAR and the nature of racism itself.

There is little available information on how the initiatives selected for CAPAR funding were chosen. Interviewees and the RMAF indicated that a formal gap analysis was undertaken prior to CAPAR to examine where additional federal programming related to anti-racism would be beneficial. Interviewees stated that this gap analysis then formed the basis for the selection of initiatives that would receive CAPAR funding. Note that this gap analysis was not available for review by the evaluation team.

While this selection process may have related to the original six priority areas of CAPAR and responded to identified needs for additional programming, it does not appear to have factored in the compatibility of the funded programs, including their potential ability to meaningfully collaborate. The funded programs included a wide variety of objectives and target audiences, and ranged from funding the development of programming for immigrant youth (WCI), to policy research related to race-based issues in the justice system (the DoJ component). While some programs appear to have had common elements or stakeholders (such as, most obviously, the hate crime-related activities undertaken by DoJ and the Data Collection Strategy of CIC), they were generally dissimilar and, arguably, fairly incompatible in terms of potential collaboration.

Interviewees agreed (9 of 10) that the funded initiatives largely lacked cohesion, primarily because they did not have common objectives, other than at the broad level. This was also noted in a number of previous reviews, which stated that “the outcomes identified in the logic model do not consistently align with the activities and outputs of the nine initiatives” Footnote 35 and that “Initiatives saw linkages between their work and CAPAR only at a very high level and their efforts were not well reflected in the horizontal logic model.” Footnote 36

The challenge with cohesiveness is likely, in part, a consequence of the nature of racism as an issue. Since racism can have an impact on so many different areas—including the integration of immigrants, youth in schools, the workplace, the criminal justice system, and others—there is a great deal of policy scope for initiatives that do not complement each other in any natural way. Further, the fact that CAPAR activities were limited to only the funded initiatives (i.e., the identified gaps) affected the possible level of cohesion across the whole Initiative.

3.2.2. Effectiveness of CAPAR management and coordination

Finding: A governance and management structure for CAPAR was put in place; however its effectiveness was limited in terms of both coordination and information-sharing.

The effectiveness of the CAPAR governance structure was assessed by examining: the level of senior management support; the clarity of roles and responsibilities; performance measurement; and the effectiveness of coordination and information-sharing.

Governance and management structure and support

According to the information from the literature review, an effective and appropriate management structure is vital to the success of any horizontal initiative. While there are different types of possible governance structures, they typically consist of one or more interdepartmental committees and working groups, supported by a secretariat. The governance and management structure for CAPAR, established in the foundation documents, was aligned with this practice. The structure included a lead department—originally PCH and then CIC—and an IWG, to be composed of lead officials from the nine funded initiatives and evaluation officers from the four funded departments. Finally, the structure included a Secretariat, housed in the lead department, to provide leadership and support to the IWG and, in effect, to carry out the lead department’s responsibilities under CAPAR.

The literature review also noted that it is “common to have a steering committee in place at the appropriate level to provide strategic direction and to oversee the initiative; and a Management Committee to organize the people, tasks, relationships, information and technology to get the work done and to formulate recommendations for the Steering Committee.” Footnote 37 There was no senior management body in place for CAPAR that could have identified potential areas of collaboration, which was often identified by interviewees as a gap in the design. When asked to rate the overall effectiveness of the management of CAPAR (on a scale of 1 to 5, 1 being not effective, 5 being very effective), many CIC and CAPAR-funded interviews rated it a three (8 of 11), while two rated it a four or five and one rated it a one or two. In explaining their responses, interviewees were often critical of the fact that no management body was in place for CAPAR.

Interviewees often noted an overall lack of sustained commitment from senior management (10 of 13 interviewees said commitment was lacking or that commitment waned over time). As outlined in the literature review, horizontal initiatives require ongoing political and senior management support and leadership to maintain their legitimacy…and to operate efficiently and effectively Footnote 38 . This is consistent with the 2008 Management Review which suggested that most federal departments with horizontal initiatives similar in organization to CAPAR have an ADM-level interdepartmental steering committee supported by working level committees. The findings of the Management Review noted that this issue had been raised with the IWG in 2006, however it was decided at that time that the working group was sufficient to guide CAPAR.

It was also noted during interviews that CAPAR lacked sustained political interest. It was suggested that the Minister of State (Multiculturalism and Status of Women) was a strong champion of the initiative at the time of its launch, but CAPAR lacked a senior political champion during its implementation. As a result, following its launch, the initiative lacked visibility within the federal government and among its stakeholders.

Roles and responsibilities of secretariat and IWG

As previously discussed, the foundation documents for CAPAR outlined specific responsibilities for the lead organization, the IWG and the Secretariat. The responsibilities of the IWG and the Secretariat were further defined in program documentation. Generally, the Secretariat was responsible for: coordinating the IWG; organizing stakeholder consultations; coordinating performance measurement and reporting, and developing the annual report (Table 3-1 ). The role of the IWG was to assist with the planning of the annual stakeholder consultations, and the development and collection of baseline and performance information.

Table 3-1: Summary of expected responsibilities of the IWG and Secretariat
Key Roles and Responsibilities
Secretariat
  • Coordinate Interdepartmental working group meetings, to be held approximately four times a year (January, April, June and October).
  • Organize annual stakeholder consultations.
  • Coordinate performance measurement and reporting (e.g., RMAF, reporting framework, baseline information, evaluation).
  • Analyze input, draft the Annual Report and all related material.
  • Manager of the Action Plan Unit will chair the meetings.
Working Group
  • The group will meet a minimum of four times a year until the final evaluation for CAPAR is completed (to discuss annual report preparations, engagement sessions, and evaluation).
  • Participate in planning and attend annual stakeholder consultations.
  • Cooperate with the Action Plan Unit on developing and collecting baseline information, performance information, and evaluation processes.
  • Provide comments on the Annual Report drafts.

Source: Terms of Reference for the IWG; 2006 CAPAR Reporting Framework.

Some Secretariat and CAPAR-funded interviewees believed the roles were clear (3 of 5), while others suggested they were not (2 of 5). Information from the evaluation suggests that perhaps the issue was not the clarity of the roles, but the expectations of stakeholders. The roles and responsibilities of the Secretariat and IWG were perhaps too narrowly defined for some. Information from interviewees suggests that much more was expected (for more on this see section below on coordination and information-sharing).

Performance measurement

The literature review identified performance measurement as one of the key success factors in the management of a horizontal initiative. It suggests that it is “necessary to establish a process whereby participating organizations provide performance data, usually to the secretariat, on a regular basis in an agreed-upon format” Footnote 39 and that the ideal is to “establish an evaluation framework (such as an RMAF) and collect good baseline data at the outset.” Footnote 40 The CAPAR Secretariat put in place a process for identifying performance indicators and collecting performance information. Information from interviews and the document review shows that an RMAF was developed for CAPAR, which was created in collaboration with the IWG. In addition, the requirements and processes for joint monitoring and reporting were clearly outlined in a CAPAR Reporting Framework (2006). The reporting framework was established to ensure that responsibilities and timelines for reporting were clear and that the collection of data was standardized.

On an annual basis, CAPAR-funded departments provided performance reports to the Secretariat using a standard template. A review of these templates showed that the first portion of the template was narrative and captured information such as: description and objectives of the initiative, partners and stakeholders involved, changes to the initiative, consultations held, and challenges encountered. The second portion of the template contained a performance section that included a table of performance measures on which initiatives were to report. In reviewing the annual reports submitted, the evaluation found that initiatives used different performance measures and not all initiatives reported on them. The lack of common performance measures means that it would have been very difficult to provide any reporting on the achievement of common CAPAR outcomes. This is evident from the Annual Reports to Parliament on the Implementation of the Canadian Multiculturalism Act, which featured very little information on CAPAR Footnote 41.

Previous studies and reviews confirmed that there were challenges with performance measurement. For example, the Evaluation Assessment of CAPAR (2006) noted that there is often a significant gap between the description of objectives and outcomes and associated performance indicators. In addition, the Baseline Review conducted in 2007 identified challenges with respect to: integrating baseline and performance information into the reporting structure; ensuring that indicators align with activities, outputs and outcomes; and accommodating the variety of indicators and baseline information available. These findings are consistent with the information from the interviews. Two of three CAPAR-funded program representatives were critical of the reporting structure, suggesting that it was not useful because the measurements were not meaningful at the program level (for further discussion on the challenges of performance measurement, see 3.3 Performance).

Effectiveness of coordination and information-sharing

Information from the literature review identified coordination and information-sharing as a key success factor in the implementation of horizontal initiatives. The CAPAR logic model itself contained an activity related to “networking and information-sharing” and therefore was a planned component of the horizontal management.

The main mechanisms in place for coordination and sharing information were the Secretariat and IWG. The literature review stressed the importance of having a strong Secretariat in place, noting that “an effective secretariat can make a major contribution to the success of any horizontal initiative” and that it “can also play a key role in facilitating effective communication, not only by ensuring consistent messaging, but by ensuring that the various working groups and committees are kept fully up-to-date on what is happening and are given timely opportunities to provide input.” Footnote 42

The evaluation found that the Secretariat was not effective in ensuring coordination and information-sharing. With minimal staff, it was not able to undertake all of its responsibilities. The Management Review completed in 2008 indicated that CAPAR-funded representatives were mixed in their opinion on the horizontal coordination of CAPAR: two initiatives were somewhat unsatisfied; one was satisfied; one very satisfied; and one provided no response. Those interviewed for the evaluation suggested that the Secretariat did not provide sufficient coordination, communication, and information-sharing, primarily because there was too much turnover in Secretariat staff, too few meetings, and a lack of leadership. CAPAR-funded representatives were interested in having more opportunities for information-sharing among the departments. This was also noted in the Management Review, which concluded that there was general interest in increasing the level of information-sharing among initiatives, having the Secretariat share more information, and in changing the structure of the working group meetings.

While Secretariat and CAPAR-funded interviewees believed that the membership of the IWG was appropriate, there was some criticism that the group was too operational and lacked decision-making authority. Interviewees also suggested that the IWG was not effective because it did not meet frequently enough, lacked a clear mandate, had significant turnover in members, did not include non-funded departments, and lacked leadership or direction. Limited documentation was available for IWG meetings, although it confirmed that three meetings were held between 2008 and 2009 (January 2008, May 2008, and September 2009). Information from program representatives indicate that other meetings did occur, however, no records of those meetings were kept. Note that keeping records of minutes was a recommendation in the 2008 Management Review.

Overall, the effectiveness of both the Secretariat and the IWG were limited by the design of CAPAR in that it focused solely on performance measurement and reporting, which, as discussed previously, was not in line with its original intent.

3.2.3. Federal partner and stakeholder engagement

Finding: Results of consultations completed for Canada’s participation in WCAR fed into the development of CAPAR, however, little consultation was done specifically on CAPAR and stakeholders believe that consultation was insufficient. Funded departments undertook consultations following the launch of CAPAR in support of the development of their initiatives.

Consultation prior to CAPAR

Extensive consultations were undertaken leading up to Canada’s participation in WCAR. Records of the Proceeding for WCAR consultations indicate that a total of seven regional consultation sessions were held across Canada during fall 2000 and spring 2001 Footnote 43. The consultations involved members of civil society, including representatives of labour, business, academia, ethnocultural and non-governmental organizations, professionals, and race-relations groups. The regional meetings culminated in a National Consultation, held in Ottawa in February 2001. Again, this national consultation brought together various members of civil society and was held to obtain national perspectives on WCAR themes, draft documents, and national priorities.

The information gathered through the WCAR consultations was one source of information that fed the development of CAPAR. However, this was not the common perception among interviewed stakeholders. When asked about the consultations completed for CAPAR, seven of 12 interviewees that commented on this question suggested that no consultations were completed and one suggested that consultation was limited. The four remaining interviewees indicated that consultations were done in preparation for WCAR. Only nine interviewees commented on the sufficiency of the consultations: six said it was insufficient; three said it was sufficient.

Consultation during CAPAR

The original action plan stated that the lead department would consult stakeholders annually on the progress of CAPAR’s implementation. Information from the literature review suggests that regular consultation with key players is an important component of a horizontal initiative, as it can provide a source of leveraging and support, can further communication of the initiative and can keep the initiative on the political agenda Footnote 44.

Information from the evaluation shows that very limited consultation took place following the launch of CAPAR and that it did not focus on the progress of implementation. A total of six consultation sessions were held across Canada between 2005 and 2007 Footnote 45. Individuals representing community, education, academic, municipal and police organizations attended these sessions. The sessions focused on raising awareness of CAPAR and discussing performance measurement challenges. Few interviewees were able to comment on the sufficiency of these consultations; all nine of those who did comment indicated that these consultation sessions were insufficient. Some specifically noted that the provincial governments were not adequately involved in consultations, although available documentation did not provide any information on the level of involvement from the provinces.

While there was limited consultation for CAPAR overall, it should be noted that information from the document review and the initiative evaluations demonstrated that consultations were undertaken with stakeholders for CAPAR-funded programs. For example:

  • LEAD grew out of discussion at two policing in multicultural society forums;
  • WCI and III conducted consultations with stakeholders to gain insight on how to best develop their initiatives;
  • in developing the Data Collection Strategy, much work was done to develop ongoing relationships with policing services through attending relevant conferences and meetings; and
  • DoJ conducted consultations with respect to hate on the internet.

3.3. Performance

Finding: The nature and scope of CAPAR made it difficult to assess its overall impact, although activities undertaken in the funded initiatives have resulted in some progress at the immediate outcome level. Performance was affected by the fact that not all initiatives were implemented as intended.

Context

A logic model for CAPAR, developed for the evaluation Footnote 46, outlined the expected activities, outputs, and outcomes of the CAPAR-funded initiatives, overall. The logic model grouped the funded programs into two streams of programs:

  1. Inclusion: which consisted of WCI, RFWS, and III; and
  2. Law, Justice and Hate Crimes: which included ARTCI, the LEAD Network, the DoJ and the Data Collection Strategy.

The logic model illustrates the difficulty in assessing the overall results of CAPAR. Even within the two streams of programs, the funded initiatives involved a number of different target groups and types of activities. The Multiculturalism Program originally had a plan in place (with the Social Sciences and Humanities Research Council of Canada) to “identify indicators to monitor progress in the fight against racism.” Footnote 47 However, no such indicators or data were identified, and an assessment of CAPAR’s performance is therefore reliant on examining the results of the individual initiatives. In developing the logic model for the evaluation, it was difficult to formulate a cohesive set of outcomes that could encompass all the funded initiatives. As a result, the CAPAR outcomes are very broad and diffuse. This presents a barrier to performance measurement of CAPAR as a single initiative.

This is an inherent challenge for a horizontal initiative that includes multiple departments with very different mandates. While CAPAR was always envisaged as “dynamic and inclusive”, inviting “all sectors of society […] to embrace action against racism,” Footnote 48 its heterogeneity is a barrier to establishing the overall impact. Further, as pointed out in the evaluation assessment of CAPAR, anti-racism initiatives generally present severe difficulties for evaluators with respect to the attribution of changes in attitudes and behaviours Footnote 49.

In addition to these measurement challenges, CAPAR experienced some issues that negatively affected the achievement of results. Most fundamentally, some planned initiatives did not begin at all, were cancelled or were not implemented:

  • following initial conceptualization activities, ARTCI was cancelled in 2006;
  • III and the LEAD Network were cancelled in 2008; and
  • DoJ’s activities related to combating Internet-based hate crime were not implemented following some initial consultations and policy development activities.

In total, four of the original nine funded initiatives did not go forward as originally planned.

Program evaluations also noted significant delays in the initial implementation of some funded initiatives, including the DoJ component and RFWS. The reasons for these delays appear to be largely related to limited administrative capacity and slow administrative processes within the funded departments. Despite these challenges, the evaluations did document results achieved by CAPAR. The evaluations examined outcomes at the immediate outcome level, with few findings related to intermediate or long-term (ultimate) outcomes. CAPAR was intended to make progress related to the following two immediate outcomes, one for each of the two streams of programs Footnote 50:

  • federal partners and stakeholders have knowledge, resources and tools to promote inclusion and address issues of racism and discrimination in their milieu (Inclusion); and
  • federal partners and stakeholders have an understanding of culturally competent policing, hate crimes, racial profiling and the problem of the overrepresentation of ethnocultural groups in the justice system (Law, Justice and Hate Crimes).

The following sections describe the target audiences reached by the CAPAR-funded initiatives, and provide highlights from the documented results.

3.3.1. Knowledge, resources and tools to promote inclusion and address issues of racism and discrimination

Target groups

CIC’s WCI was intended to follow established models for settlement program delivery and support ongoing anti-racism activities. WCI provides services directly to immigrant-serving organizations and other community-based organizations, who in turn provide services to newcomers. Target groups for the WCI, therefore, included:

  • settlement organizations;
  • receiving communities, including community-based organizations/groups, educators, and the Canadian public; and
  • newcomers to Canada, with a particular focus on youth.

RFWS was a complement to HRSDC’s employment equity program, which targets employers covered by the Employment Equity Act, the Federal Contractors Program and the Legislated Employment Equity Program. III was intended to target all federal institutions, with an initial focus on 15 specific departments and agencies.

Results

WCI funded 56 projects (between 2006-2007 and 2008-2009), which included a diverse range of activities to promote the successful integration of newcomers to Canada, by reducing discriminatory barriers, in order to maximize the economic, social and cultural benefits of immigration. The evaluation of WCI grouped the projects into six themes:

  • Awareness Raising and Education, including outreach activities, events, workshops, presentations and conferences (12 projects);
  • Newcomer Participation and Integration, including mentoring, events, and developing special supports (11 projects);
  • Anti-Racism and Diversity Training, which consisted of the development and delivery of training materials and a pilot training program (8 projects);
  • Development of Tools and Resources, which involved the development of training materials and other tools, as well as conducting research, workshops and presentations (7 projects);
  • Research and Policy Development (6 projects); and
  • Community / Settlement Services Expansion / Enhancement, which included support for non-settlement organizations and the expansion of CIC’s Settlement Workers in Schools and Host programs (12 projects).

With the possible exception of the last, all of these themes are likely to result in stakeholders having tools and resources to promote inclusion and address issues of racism. At the time of the evaluation, an estimated 32,000 individuals had used WCI products or participated in activities undertaken through WCI projects during the previous two years.

WCI projects appear to have had a positive impact on the capacity of newcomers, particularly youth, to deal with issues related to prejudice and discrimination and to integrate into their communities. Some WCI projects reached out to newcomers directly by providing information and educational activities through workshops and education seminars or by engaging them in discussions about racism and discrimination. Examples included sessions on prejudice and discrimination in English as a Second Language classes, training students as part of settlement supports in schools, and training newcomers to become ambassadors of anti-racism among their peers.

WCI projects also appear to have had a positive impact on receiving communities, enabling them to become more aware of issues related to racism and discrimination. For example, many WCI projects involved non-settlement organizations, such as businesses and Canadians, and have included workshops, presentations, and resources to educate on the experience of immigrants.

RFWS also completed an extensive range of activities related to promoting inclusion and addressing racism/discrimination in the workplace. According to preliminary findings from its summative evaluation, 827 individuals representing 490 organizations participated in RFWS-supported activities. In follow-up interviews undertaken with participants, 76% reported attending racism prevention workshops and/or training sessions; 39% used tools and resources (including slide shows, training materials, and handouts) provided by RFWS; and 26% were visited by a Racism Prevention Officer. Most participants communicated the knowledge they gained from RFWS activities back to their respective organizations.

While there is limited information available on III, primarily because it was in operation for only two years, existing documentation indicates that a total of five projects received funding covering activities such as: increasing departmental capacity to provide culturally relevant services; enhancing cultural competency of the department in dealing with ethno cultural/racial clients; creating linkages between the correctional system and the community; improving quality of translation services; and commissioning a study on barriers to ethno-cultural/racial communities in dealing with the Federal Government. No information was available regarding the progress or results of these projects.

3.3.2. Understanding of culturally competent policing, hate crimes, racial profiling, and the problem of the overrepresentation of ethnocultural groups in the justice system

Target groups
  • The target groups for the Law, Justice and Hate Crimes stream of CAPAR included the following:
  • Police services and other law enforcement agencies (for the LEAD Network, the Data Collection Strategy, and the DoJ component);
  • Policy makers and researchers within government and academia (for the Data Collection Strategy and the DoJ component);
  • Legal experts, industry partners and educators, as well as Black youth in the criminal justice system (for the DoJ component);
  • Community groups (for the LEAD Network); and
  • International stakeholders (for the Data Collection Strategy).
Results

The LEAD Network was the CAPAR initiative primarily intended to increase the level of knowledge of culturally competent policing. The LEAD Network was intended to function as a catalyst for new research focused on policies and practices for policing in a multicultural society. The formative evaluation of the program (undertaken in 2007) indicated that, to a large degree, the network had produced limited results, and the program was cancelled in 2008. Some relevant activities undertaken prior to the cancellation of the program included: a conference on diversity, which included nearly 500 participants; and four cultural competency workshops with police diversity officers. A LEAD Network website was a major deliverable of the project, and was to act as a new forum for police services to exchange information on policing in Aboriginal and ethno-cultural communities. However, the website was found to have had a very limited impact on police services.

CAPAR appears to have been more successful in its funded activities related to increasing knowledge of hate crimes. Indeed, the Nationally Standardized Data Collection Strategy on Hate-Motivated Crime was frequently identified by interviewed key informants as one of the key successes of CAPAR. Between 2005-06 and 2008-09, a total of 681 participants at 156 police services / detachments received training on hate crime reporting through the Data Collection Strategy, and the training was well-received by participants. This training enabled police services to collect nationally consistent data on hate-motivated crime, which was noted as one of the main benefits of the Strategy. Further, Statistics Canada, through the Data Collection Strategy funding, produced a series of information products (including annual reports, data tables, and other materials) that appear to have raised the level of understanding of hate crime among police services that accessed them. The evaluation noted that the outputs of the Strategy will continue to become more relevant and valuable to researchers and other stakeholders as more years of data become available and as the geographic coverage continues to improve.

The DoJ component of CAPAR also involved a range of different activities related to hate crimes’ victims and perpetrators. These included, among others:

  • The Research and Statistics Division of the department produced a report on hate crime victims, and another (draft) report on hate crimes was presented at two conferences in 2009-2010;
  • The department funded two projects intended to increase the understanding of effective interventions for perpetrators of hate crimes;
  • The African Canadian Legal Clinic developed an Anti-Black Hate Crimes Manual, and held a forum on anti-Black hate crime, which included 200 attendees;
  • Training materials related to the needs of hate crime victims were developed for victim service workers; and
  • A chapter on victims of hate crimes was added to a manual used by criminal justice professionals and victim service workers and, at the time of the evaluation, 3,400 copies of the manual had been distributed.

The department had also undertaken consultations and research activities to determine how it should move forward in addressing Internet-based hate crime. Activities were halted; however, as the Department was awaiting further clarity on this issue following a review of a related section of the Canadian Human Rights Act.

With respect to racial profiling, some work was undertaken through DoJ on this topic, including policy research and consultations within the department. The LEAD Network had also commissioned a study on racial profiling.

Finally, activities related to the problem of overrepresentation of ethnocultural groups in the Justice system, with a particular emphasis on the overrepresentation of Black Canadians and Aboriginal people, were undertaken by DoJ. Activities included:

  • The National Anti-Racism Council of Canada conducted four regional sessions with a total of 111 participants to highlight activities that help to address issues of overrepresentation of African Canadians and Aboriginal people;
  • A report outlining the research and underlying factors in the overrepresentation of racial minorities in the Canadian criminal justice system was prepared; and
  • Four forums were held in Ottawa, organized by DoJ’s Collaborative Working Group: Responding to Violence in Aboriginal Communities.

3.3.3. Value-added of horizontal approach for CAPAR

Finding: The horizontal approach provided very limited added value to either the CAPAR-funded initiatives or the non-funded initiatives.

Overall, interviewees had mixed views on whether designing CAPAR as a horizontal initiative resulted in any added value. Senior managers, Secretariat members and CAPAR-funded representatives were asked to rate (using a scale from 1 to 5, with 1 being no extent and 5 being great extent) the extent of added value resulting from the horizontality of the initiative. Half of the interviewees rated this as three (5 of 10); three rated it a four or five; and two rated it a one or two. Of those that provided comments to explain their rating, four of 10 suggested the horizontal design added value because it provided the opportunity to bring people together. However, some (5 of 10) suggested that generally it did not add any value and that the funded initiatives were too different for meaningful collaboration (2 of 10).

Overall, the evaluation demonstrated that there was very little value added to either the funded or non-funded initiatives as a result of CAPAR’s horizontal approach. As previously discussed, there was little cohesion between the funded initiatives in terms of target groups, activities and intended outcomes. The non-funded initiatives, although part of CAPAR’s original action plan, were not involved in subsequent activities, and it is not understood how they were expected to fit into the planned outcomes. Ultimately no communication, information-sharing or outreach was conducted with the non-funded initiatives and there is no information to determine their progress or outcomes.

The design of CAPAR also limited any meaningful coordination of performance measurement activities among the funded initiatives. Ultimately, while performance indicators were established and reported on, there was no apparent usage for the information. It did not appear in the Annual Reports to Parliament on the Implementation of the Canadian Multiculturalism Act and no senior body was in place that would have made use of the information. Also with respect to the funded initiatives, it is unlikely that their progress or performance was influenced by CAPAR and the same results likely would have been achieved in the absence of horizontality.

Despite their views on the actual horizontal management of CAPAR, all interviewees were supportive of the idea of using a horizontal approach to address the issue of racism and discrimination in the future (26 of 26). Specific benefits included supporting the exchange of best practices, providing the government with a comprehensive view of what it is doing to combat racism (and enabling it to identify gaps) and helping the government to tell a coherent story (including integrated reporting). The preference on the part of interviewees for a federal government response to the issue of racism being managed as a horizontal initiative can be attributed to the fact that interviewees see the issue as one that cuts across all departments (with 18 of 26 interviewees citing this as a reason).

Page details

Date modified: