Evaluation of the Welcoming Communities Initiative
4. Evaluation Findings
This section summarizes the major findings of the evaluation regarding program relevance, performance, impact, and design and delivery.
In assessing program relevance, the evaluation considered the need for such programs and the consistency of the WCI with CIC and Government of Canada priorities and federal roles and responsibilities. The following describes the findings and related evidence.
Key Findings: There is a need for programming that addresses barriers to newcomer integration, including racism and discrimination. Through supporting a range of anti-racism and multiculturalism efforts that foster more welcoming and inclusive communities, the WCI plays a role in addressing existing needs. The document review indicated that the WCI is aligned with CAPAR and consistent with Government of Canada and CIC objectives and priorities, as well as unique among anti-racism programs funded under CAPAR in its focus on newcomers.
There is a need for programming to address issues of racism and discrimination in Canada, given the increasing diversity of the population; the continued existence of racism and discrimination against newcomers and visible minorities; and the distribution of immigrants to rural areas and small cities, which have traditionally been comprised of fairly homogenous populations. In addition, there are social, economic, and political factors that can contribute to incidences of racism and discrimination and serve to delay the social and economic integration of newcomers.
The cultural make-up of Canada is becoming increasingly diverse. According to the 2006 Census, visible minorities accounted for 16.2% of the total population in Canada, up from 13.4% in 2001Footnote 22. Immigration is a key factor in increasing the visible minority population. Three-quarters (75%) of the immigrants who arrived between 2001 and 2006 belonged to a visible minority group whereas visible minority groups encompassed only 38% of immigrants who arrived before 1991Footnote 23. Between 2001 and 2006, Canada’s visible minority population increased by 27.2% which is five times faster than the 5.4% growth rate of total populationFootnote 24 (Figure 4-1).
Figure 4-1 : Increase in Visible Minority Populations
Despite Canada’s longstanding commitment to embrace diversity and combat discrimination and racism, recent surveys indicate that 74% of Canadians believe that racism still existsFootnote 25. The Ethnic Diversity Survey (2002) shows that one-fifth (20%) of visible minority respondents reported they had sometimes or often experienced discrimination and unfair treatment in the past five years because of their ethno-cultural characteristicsFootnote 26. Furthermore, visible minority immigrants are more likely than other immigrants to have lower incomesFootnote 27. The wage gap is particularly significant among immigrant visible minority malesFootnote 28. Findings from the literature review confirmed these challenges. For example:
- Racism and discrimination against newcomers in general and visible minorities in particular continues to exist. The proportion of visible minorities who felt they experienced discrimination was twice that of non-visible minoritiesFootnote 29. About 81% of those also believed that it was because of their race or ethnic origin. Furthermore, nearly 50% of Blacks and about 33% of South Asian and Chinese respondents reported discrimination or unfair treatmentFootnote 30.
- Other studies have also identified a variety of concerns, including:
- Visible minority immigrants are more likely than other immigrants to experience low incomeFootnote 31 .
- The wage gap for visible minorities increased over the last decade from 11% to 14.5% and this trend is expected to persist. This must be addressed given Canada’s continuing reliance on immigrants for net labour force growthFootnote 32.
- Other studies have also identified a variety of concerns, including:
- Strategies are needed to address possible racism and discrimination associated with the increased settlement of newcomers in non-traditional regions of Canada. As stated earlier, the majority of the immigrant population belongs to visible minority groups and various ethnic originsFootnote 33. These groups are immigrating across the country, including to rural areas and small cities which could be experiencing a shift from a long-standing homogenous nature to multicultural diversityFootnote 34. Thus, there is a particular need in these areas to create welcoming and inclusive communities in order to facilitate the healthy integration of immigrants. For example, research conducted in St. John’s shows that young newcomers can be at a higher risk of dropping out of school than other students due to experiencing racism and isolationFootnote 35.
- Counteractive social, economic, and political factors may increase racism and discrimination. SPOs mentioned that two major societal factors which have heightened the need for anti-discrimination activities are the economic downturn and the September 11th terrorist attacks. SPO senior representatives explained that the recent economic downturn magnified the existing barriers preventing newcomers from social and economic integration. Hate crime may also be an issue. According to the report, “Anti-Racism & Anti-Discrimination Toolkit”, projected hate crimes were estimated at 60,000 per year and are expected to rise. Almost two- thirds of hate-crime victims are radicalized group members. Also, after September 11th 2001, discrimination has become more overt and blatant. A review of literature shows that since September 11th, Muslims and Arabs have endured increased incidences of discrimination against members of their communities, both by individuals and the stateFootnote 36.
Programming that addresses barriers to participation, such as racism and discrimination, is needed to help support newcomer integration. The WCI aims to create connections between newcomers and communities; educate communities on racism, discrimination and multiculturalism; and increase awareness of the benefits of immigration.
The need for the WCI was attributed to the role that the Initiative plays in integration by helping newcomers connect with the community, as well as by reaching out to the communities and educating them about diversity and the benefits of immigration. According to stakeholders, the WCI is also needed to:
- Address existing discrimination practices and prejudice against visible minorities and immigrants;
- Increase the capacity of immigrant and non-immigrant service agencies to deliver culturally sensitive services and effectively respond to racism and discrimination;
- Increase public awareness about racism and discrimination, particularly in neighborhoods where the risk of isolation, racism and discrimination is higher;
- Create inclusive work environments and combat the barriers to newcomer employment and issues of racism;
- Help retain newcomers and increase their participation in rural areas; and
- Reach out to the community members and work with them in creating more accepting communities.
Findings from the interviews showed that all participant groups believed that there is a major need for the WCI (see Figure 4-2). Eleven of 12 CIC Directors and Managers and all five provincial representatives rated the need for the WCI (on a scale of 1 to 5 where 1 is no need at all and 5 is a major need) to be major.
Figure 4-2: Need for the WCI
CIC officers attributed a greater need, on average, for the WCI and ISAP B projectsFootnote 37 (which contribute to capacity building of SPOs) than SPO Managers/Directors (see Figure 4-3). However, this trend was the reverse for the ISAP and Host programs (which focus more on service delivery to newcomers).
Figure 4-3: Need for the various settlement programs
4.1.2. Consistency with GoC and CIC Priorities and Federal Roles and Responsibilities
A review of various documents on government policies and priorities indicated that the WCI is consistent with Government of Canada priorities and federal commitments and aligned with CAPAR and CIC objectives.
The objectives and intended outcomes of the WCI to build the capacity of the settlement sector, receiving communities and newcomers to take action to reduce racism and discrimination and foster strengthened participation of newcomers and more inclusive and welcoming communities in Canada are aligned with federal commitments both in Canada and internationally.
Equality and protection from discrimination are fundamental principles of Canadian society; they are embedded in Canada’s laws and articulated in its foundational documents, such as the Constitution, the Canadian Charter of Rights and Freedoms and the Canadian Human Rights Act. Canada is also a party to the International Convention on the Elimination of all Forms of Racial Discrimination, which requires “States Parties [to] condemn racial discrimination and undertake to pursue by all appropriate means and without delay a policy of eliminating racial discrimination in all its forms and promoting understanding among all races”…“and to ensure that all public authorities and public institutions, national and local, shall act in conformity with this obligationFootnote 38.”
The Canadian Multiculturalism Act sets out the Multiculturalism Policy for Canada, and CIC is responsible for the administration of this Act (since October 2008). According to the Act, it is the policy of the Government of Canada to “promote the full and equitable participation of individuals and communities of all origins in the continuing evolution and shaping of all aspects of Canadian society and assist them in the elimination of any barrier to that participation” and “that all federal institutions shall promote policies, programs and practices that enhance the understanding of and respect for the diversity of the members of Canadian societyFootnote 39.”
The WCI is funded under CAPAR. The Initiative is aligned with its objectives and priorities to eliminate racism, discrimination and barriers to participation, while promoting the inclusion of all Canadians.
CAPAR outlines the plan and priorities for the Government of Canada, inviting all sectors of society including governments, organizations, individuals, and ethno-racial and ethno-cultural communities to combat racism and racially-based discrimination. Anti-racism activities across the country focus on various aspects of racism and discrimination, such as racism in the workplace, discrimination against Aboriginal peoples, and organizational and structural racism. PCH, Justice Canada and Human Resources and Skills Development Canada (HRSDC) also received funding under CAPAR to develop and deliver anti-racism initiatives.
These initiatives have a different focus from the WCI. The WCI is uniquely positioned within CAPAR because it is the only initiative that focuses on newcomers and reducing the barriers that they face related to racism and discrimination. Table 4-1 demonstrates how WCI projects support CAPAR in a number of priority areasFootnote 40.
|Assist victims and groups who are vulnerable to racism and any forms of discrimination||Research shows that many newcomers belong to visible minority groups. Newcomers, particularly those who are visible minorities, are vulnerable to racism and discrimination. Youth can be particularly vulnerable, as well as refugees who have experienced trauma and violence and have no experience with urban western societies. Some WCI projects aim to empower newcomers to address issues of racism and discrimination.|
|Develop forward-looking approaches to promote diversity and combat racism||WCI projects are often community-based. Communities develop their own approaches and innovations to address their unique needs. The “two-way street” approach to integration, with its focus on receiving communities (and not just newcomers), in a way, is an innovation of the settlement program to facilitate the full and active participation of newcomers in Canadian society.|
|Strengthen the role of civil society||WCI provides funding to community organizations and agencies to develop and deliver strategies to address racism and discrimination which build the capacity of communities, newcomers and the settlement sector to take action on these issues.|
|Strengthen regional and
|Research and development of program tools and resources, and national projects have helped to encourage the collaboration of experts in the field and stakeholders across the regions and the country.|
|Educate children and youth on diversity and anti-racism||Through enhancing Host and SWIS program, WCI focuses on helping school children and youth to ease their transition into the Canadian school system. Events, forums, and other initiatives are also aimed at engaging students in discussions and increasing their understanding of multiculturalism and diversity.|
|Combat hate and bias||WCI projects have developed and delivered numerous workshops and presentations aimed at counteracting social and political factors that can lead to hate crime and stereotypes against newcomers.|
The review of CIC strategic outcomes and their contribution to the Government of Canada goals shows that WCI objectives and activities are relevant and well aligned with CIC strategic outcomes focused on the development of integration and citizenship programs that promote a diverse society and social inclusion (see Table 4-2).
|Government of Canada||CIC Objectives and Priorities||WCI|
|Social: Diverse society that promotes linguistic duality and social inclusion.||Integration Program – Expected results: Newcomers contribute to the economic, social and cultural development needs of Canada.||Promotes welcoming communities and diversity, builds capacity to address issues of racism and discrimination and aims to reduce barriers to newcomer integration and facilitate their full participation and inclusion in Canadian society.|
|Social: Diverse society that promotes linguistic duality and social inclusion.||Citizenship Program – Expected results: Full participation in Canadian society by citizens.||Promotes welcoming communities and diversity, builds capacity to address issues of racism and discrimination and aims to reduce barriers to newcomer integration and facilitate their full participation and inclusion in Canadian society.|
The majority (75%) of CIC Directors and Managers indicated that WCI is consistent with the strategic outcomes and priorities of CIC and those of the Government of Canada. They noted that WCI has an important role to play in newcomer integration by addressing the barriers they face, promoting more supportive and inclusive communities and ensuring social cohesion and the diversity of Canada.
SPO representatives who participated in the evaluation also agreed that the development and funding of WCI is an appropriate role for the Government of Canada in support of newcomer integration, participation, and adaptation to new communities.
Therefore, WCI projects are consistent with the objectives of the settlement programs, which aim to link newcomers to their communities, help them establish social and professional networks, and reduce barriers to their economic, social and cultural participation, so that they feel welcome in their communities. WCI has also become more salient to the mandate of CIC with the move of the Multiculturalism portfolio from PCH to CIC. “Canada’s ability to leverage the benefits of diversity depends on its success in ensuring that Canadians are engaged and have the opportunity to participate in the economic, social, political and cultural aspects of Canadian society. Multiculturalism fosters increased intercultural understanding, and supports the goal of Canadians living in a society where they are treated fairly and equitably, regardless of their backgroundFootnote 41.”
In reviewing the performance of WCI, the evaluation considered the reach and success of the projects in delivering relevant activities and outputs. The findings also provide a description of factors that can contribute to the success of WCI projects as well as those that can constrain their success.
Key Findings: While it is evident that the individual WCI projects reached a broad range of groups and organizations and delivered a broad range of activities, there is insufficient evidence to properly assess the overall performance of the WCI as a program, due to a lack of comparable data on program performance. Individual projects are, however, perceived as successful in developing and delivering their intended products and outputs. Some of the factors that contribute to this success are effective partnerships and community involvement, media coverage, flexibility of the design, and involvement of experts in the field.
4.2.1. WCI Project Reach
The projects were able to reach a variety of stakeholder groups including newcomers, SPOs, other organizations and Canadians. In particular, many WCI projects were designed to reach youth and school children through enhancements to Host and SWIS or through other activities that focused on engaging youth in discussions on anti-racism and diversity.
WCI projects expanded the outreach of traditional settlement programs beyond just the immigrant or settlement sector organizations. Development and delivery of WCI projects involved a broad range of populations including newcomers to Canada, local communities, mainstream service providers (social services, health, housing, police department etc.), settlement organizations, volunteers, youth, school boards, police services and businesses.
At least 17 WCI projects in 2007/08 and 18 in 2008/09 delivered activities directly to target groups (e.g., through youth forums, events, training, workshops and public education on anti-racism and diversity). In addition, settlement services provided to youth in schools and host activities were enhanced to include anti-racism component and engage youth in issues of integration and diversity. Table 4-3 provides a summary of the available information on the target groups reached for individual WCI projects in 2007/08 and 2008/09Footnote 42.
|Projects (Organizations Delivering)||Individuals/Organizations Reached|
|The Global Youth Forum (YMCA Windsor-Essex)||500-600 students attended|
|Diversity Cup Basketball Tournament (Peel Regional Police)||300 people and 15 social service providers attended|
|Pathways to Transformation (Catholic Immigration Centre)||87 Host staff representing 23 Ontario Host programs participated|
|Settlement and Integration Service Organization||700 staff members trained through the Cross-Cultural Training in Support of Police Services|
|Immigrant Family Recreation Program (Calgary Catholic Immigration Society) (4 Seasons)||800 newcomers involved in the 4 Seasons|
|Public Awareness Outreach project (Central Alberta Refugee Effort)||750 clients in schools, colleges, agencies and businesses involved|
|Settlement Support Workers in Schools (Saskatoon Open Door Society)||730 immigrant children and their families served|
|Rural Community Awareness – Alberta||2,000 community groups, schools and individuals reached through diversity awareness workshops in rural Alberta|
|Studying Teacher’s Attitudes: Racism, Immigration and Multiculturalism (Association for New Canadians)||125-150 teachers surveyed|
|Sharing our Cultures||3,000 Canadians and 100 immigrant students|
|Newcomers’ Resource Center (Compare Cultures / City of Saint John)||100 stakeholders engaged in consultations|
|International Day of Racial Elimination (Multicultural Association of Fredericton)||1,940 individuals involved in festival, art challenge and school tour activities|
|Diversity Kiosks (Carrefour d’Immigration Rural)||800 people participated|
|Multicultural Education presentations and workshops (PEI Association of Newcomers to Canada)||590 people participated|
|Halifax Immigrant Learning Centre||200 immigrant serving organization staff trained|
|Metropolitan Immigrant Settlement Association||65 settlement workers involved in workshops on dealing with racism|
|Changing the Canvas project (Canadian Labour Congress)||1,000 individuals involved in labour education course|
|Anti-racism and Diversity Workshops (Host Program Network)||150 people involved|
|Diversity workshops (JVS Toronto)||24 newcomers involved|
|Youth Training and Action project||Over 775 community members reached|
|Anti-Racism and Human Rights workshops||Over 200 individuals participating|
|A Rural Community Awareness project||800 clients received|
|Settlement support services in schools in Saskatoon||7,200 individuals reached|
|Settlement support services in schools in Calgary||Over 2,000 clients received settlement support|
|Peace Ambassadors Initiative in Alberta||Over 700 youth participated|
|Diversity and Organizational Change presentations in Newfoundland||Over 1,000 individuals participated|
|Sharing our Cultures||Over 1,200 students involved|
|Youth conference for March 21st in Halifax||70 youth involved|
|Anti-Racism Multicultural Festival in Fredericton||1,000 residents involved|
|The Multicultural Festival in Saint-Leonard, New Brunswick||450 participants|
|Presentations by a community and cultural sensitivity agent in PEI||240 students|
|Through the Lens training workshops (Halifax Immigrant Learning Center)||Over 160 tutors, instructors, service providers, volunteers and staff trained|
|In Anti-racism workshops in Nova Scotia||Approximately 35 community leaders, 1000 students and 85 youth involved|
|Equity Initiative – Training (NARCC)||Approximately 50 participants|
|DiverCity on Boards (Maytree Foundation)||350 visible minority or immigrant community members recruited as civic leaders|
The data presented in Table 4-3 demonstrates that reaching out to youth, particularly in schools, is the major focus of many WCI projects and activities. Youth was the main group identified in the CAPAR as a target group for the initiatives supported by the Action Plan. In addition to projects that provided settlement support to students in schools, two workshop projects (‘Sharing our Culture’ in NL and ‘Anti-Racism Workshops in Nova Scotia’) each reached over 1,000 students in 2008/09. Other WCI projects, such as festivals in Fredericton and Saint-Leonard, involved significant community participation. Also, numerous workshops and presentations were delivered to mainstream organizations, as well as to the general public.
According to the available data on projects providing services directly to target groups, over 32,000 individuals used products or participated in activities provided by the WCI projects in 2007/08 and 2008/09. Students and youth (including both newcomers and Canadian born), and workshop participants including settlement and social service providers, employers, employees, stakeholders, and tutors represent the major groups receiving services.
4.2.2. WCI Project Activities and Outputs
WCI projects have been successful at delivering activities that align with the planned activities and services for this Initiative. WCI projects also produced outputs outlined in the Logic Model.
Our research findings indicate that WCI developed and delivered a wide range of activities and generated products that support program delivery. The review of the CAPAR RMAF/RBAF highlighted the overall direction and areas of contribution of WCI to CAPAR Horizontal Initiative. As stated in the CAPAR RMAF, WCI projects were to contribute to the areas of:
- Training, Information and Awareness Development – Development and delivery of training for settlement staff, volunteers, youth and wider audiences on the benefits of immigration, as well as to deliver workshops, presentation and seminars to raise awareness about issues related to racism and discrimination.
- Policy, Programs and Services Development – Citizenship and Immigration Canada through WCI activities will develop programs and services to increase the ability of newcomers to successfully integrate, and provide better tools and outreach strategies to target populations in the workplace, educational institutions and ethno-cultural, ethno-racial and Aboriginal communities.
- Financial Support and Partnership Development – By developing partnerships with existing settlement programs, WCI is expected to increase their capacity (Host, SWIS) to provide services to more newcomers. Developing partnerships with educational institutions, business, and other community organizations is expected to increase the outreach of the programs and increase cross-cultural understanding and the reduction of racial stereotyping.
Table 4-4 summarizes the link between WCI project themes and the intended area of contribution as outlined in CAPAR.
|CAPAR||WCI Project Themes||Example of Activities/Products||Number of Projects|
|Training, Information and Awareness Development||Raising Awareness and Education||
|Newcomer Integration and Participation||
|Anti-Racism and Diversity Training||
|Policy and Program Development||Tool and Resource Development / Research and Policy Development||
|Partnership Development and Support||Community/Settlement Services Expansion/Enhancement||
The three most common project activities reported by WCI SPOs were conducting workshops and training (92%), outreach and awareness (75%), and developing partnerships (75%). The least reported project activities were service bridging (e.g., support of non-settlement organizations) (17%), and the coordination of existing services (17%).
|Project Activities||Reported by WCI SPO Respondents (n=12)|
|Workshops and training||92%|
|Outreach and awareness of antiracism and discrimination||75%|
|Developing partnerships, networking events||75%|
|Disseminating tools; training materials; best practices||67%|
|Developing tools, templates and training materials||42%|
|Developing special support (e.g., Health, Mental health, Child support)||25%|
|Promotion of services||25%|
|Service bridging (e.g., support of non-settlement organizations)||17%|
|Coordination of existing services||17%|
In support of program delivery, WCI projects are expected to produce a range of outputs such as promotion and communication materials, information sessions, conferences and presentations, reports, curricula, toolkits and other resources. For the full list of expected outputs in program delivery, refer to the Logic Model in Appendix A:. Our research findings indicate that curricula, tools and resources were the products most commonly reported by those involved in delivering the projects and by those using the outputs (reported by 75% of the SPO representatives interviewed and utilized by 42% of the WCI users surveyed). Promotion and communication materials were produced by 50% of the SPO representatives interviewed and used by 42% of those who completed the user/participant survey (see Table 4-6). About 45% of project users said that they were involved, in some way, in the development of the products that they subsequently used.
|Type of Product||Generated
(% of WCI SPO)
(% of WCI Users)
|Curricula, tools and resources||75%||42%|
|Conferences, presentations, and networking events||67%||13%|
|Training modules and workshops||58%||21%|
|Promotional and communications material||50%||42%|
|Guides and resource materials/DVDs/videos, etc.||42%||50%|
|Research reports and studies||25%||21%|
4.2.3. WCI Project Accomplishment and Success
Stakeholders generally perceived individual WCI projects as successful in development and delivery of planned activities and products, and achieving their intended objectives.
When asked to rate the success of the WCI in achieving its objectives on a scale of 1 to 5, where 1 is not successful at all and 5 is very successful, WCI SPOs provided the highest rating of 4.5, CIC Officers provided a rating of 4.1, and CIC Managers and Directors rated the program success at 3.9 (see Table 4-2). Stakeholders generally perceived that WCI has made considerable progress to date, recognizing that WCI is still in an early stage of development. A few CIC representatives said that they could only comment on the success of the individual projects with which they are familiar, noting that the overall success of the Initiative is not well understood due to the inability of existing systems to compile and report data from all projects funded by the Initiative.
Figure 4-4: Overall Success of the WCI
Stakeholders noted the success of individual projects in raising community awareness of racism and discrimination, promoting the concept of the “two-way street” for integration, developing tools and resources on regional and national level to support service providers working with newcomers, creating training opportunities, developing partnerships, and building community capacity to increase access to services and respond to issues such as hate crime, racial profiling, discrimination, prejudice and stereotypes.
One indication of success is the formal awards and recognition that some projects have received.
WCI SPOs identified various measures through which the success of their projects can be assessed such as the number of successfully completed/delivered activities, number of individuals using WCI products or participating in the activities, external recognition of WCI projects, and community responsiveness and interest in the WCI projects and activities. Several WCI projects specifically identified awards and recognition that they had received for their projects, including:
- 2007 Canadian Mental Health School Award for reducing impact of racism with youth: The Peace Ambassadors project, conducted by Northern Alberta Alliance on Race Relations, provided diverse youth with significant opportunities for training about racism, discrimination, stereotyping, conflict resolution, inclusion, and the barriers and challenges that newcomers face in Canada. The project used innovative methods such as drama and art for creating dialogue on difficult topics like racism and discrimination. The project was particularly successful in establishing a small group of peer educators who then assisted in the recruitment and training of other youth. The success of the project has been a result of the “buy-in” from the local youth community and partners in the school system and community.
- 2008 Minister’s Award of Community Excellence. Central Alberta Economic Partnership (CAEP) worked with Innisfail's Welcoming Community Committee to develop a supplement toolbox to the Attracting and Retaining Immigrants Toolbox of Ideas developed by the National Working Group on Small Centres Strategies. The Innisfail Toolbox for Small Centres helped CAEP communities to create more welcoming communities through the implementation of best practices outlined by Innisfail.
- 2009 “Exemplary Resource” chosen by the Federal Family of Community Collaborators. Developed by the Immigrant Learning Centre (HILC), Through the Lens is a tool for instructors and settlement staff to use in and out of the classroom to raise awareness and open up a dialogue on anti-racism and discrimination while teaching language. The tool was very well received throughout Nova Scotia and it was chosen as an exemplary resource by the Federal Family of Community Collaborators in February 2009Footnote 44. The Executive Director of HILC also presented Through the Lens at the Collaborative Community Initiatives Speaker Series (CCISS), a monthly learning opportunity for federal employees to network, exchange ideas, and to explore new ideas on community issue and approaches.
Key factors contributing to the success of the individual WCI projects include effective partnerships and community involvement, media coverage, flexibility of the design, and involvement of experts in the field. The users of the WCI project outputs expressed their satisfaction with the products and information, which they found up-to-date, comprehensive, engaging and accessible.
Stakeholders, particularly WCI SPOs, identified the key factors that contributed to the success of their projects and the Initiative overall. These factors include:
- Effective partnerships and active involvement of key stakeholders. Participants indicated that the successful WCI projects are conceptualized and delivered with the input and involvement of all parties. Active participation of community members to identify needs, further promote project activities and raise the project profile is crucial for program success.
- Attracting media coverage to increase promotion and participation. Media coverage can contribute to success in various ways such as attracting new participants, encouraging current participants, increasing awareness of the issues and services, getting community attention, and creating opportunities for more partnerships.
- Flexibility of the WCI to create room for creativity and innovation. The flexibility of WCI allows for projects to be more responsive to the changing needs. For example, the Safe Harbour Project was able to adapt their training material to rural communities to reflect issues relevant to a rural context.
- Good project planning and expertise of staff. Organizing events and community activities requires a high level of planning as well as experienced and dedicated staff who are well connected in their communities. For example, YMCA staff experience and connections to schools and to youth made it easier to plan a youth forum and recruit participants. A few provincial representatives noted that successful development of programs addressing sensitive issues such as racism and discrimination requires the expertise of individuals who have a deep understanding of the subject matter and have experience working in the field.
According to the survey of project users, WCI projects were particularly successful in: increasing community and settlement organization capacity to use technology and media; providing up-to-date information on events, policies and program changes; organizing activities focused on social networking, outreach, promotion, and human rights issues; and creating opportunities for learning, networking, partnerships, sharing of best practices and exchange of ideas. WCI project users were very satisfied with the products they used, providing an average rating of 4.8, on a scale of 1 to 5, where 5 is very satisfied. They noted that the information they received was very useful and comprehensive, materials were engaging, and project staff were helpful and accessible.
Although different WCI projects face unique challenges, some common factors that may constrain the success of WCI projects include the lack of ongoing and multiple-year funding, limited marketing and promotional activities, and difficulty attracting volunteers and community organizations.
Stakeholders identified specific factors that could constrain the success of particular projects including:
- Limited access to on-going funding, particularly for on-going multi-year projects for promotion or future developments. In some cases, a large portion of funding was spent on developing the projects (tools and resources) or delivering the services to clients and little was left to conduct effective promotional and marketing activities or provide logistics and planning for future developments.
- Marketing and promotional capabilities. Limited capacity of service providers to conduct the necessary marketing and promotional activities and to reach out to target audience affects the continuation of the projects and dissemination of products. In the case studies, participants noted that allocating time and resources towards project promotion or dissemination of WCI products was challenging. WCI SPOs most frequently reported that their products were promoted or made available to users online (4 of 12) or through presentations to the community (2 of 12), presentations/booths at non-settlement agency events (2 of 12), presentations/booths at settlement agency events (2 of 12), specific training events related to the products (2 of 12), mailing them out to organizations and their membership (2 of 12), or making hard copies available at their office (1 of 12).
- Difficulties in attracting volunteers and community organizations. Recruitment and retention of youth volunteers has been an ongoing challenge because of the nature of the target group (1 of 12). Ensuring sustainable funding helps in developing on-going long-term partnerships and innovative strategies to reach youth and secure their participation (1 of 12). Some projects also faced difficulties in developing partnerships with community organizations and involving them in their projects (3 of 12).
4.3. Achievement of outcomes
One of the objectives of this evaluation was to examine the early outcomes of WCI. The following findings represent the early impacts of WCI. Though early, they can open a discussion for better defining the outcomes and target groups for WCI projects. The identified impacts can also shed light on more specific performance indicators that should be tracked and measured periodically in order to facilitate the future evaluation of WCI.
Key Findings: The findings indicate that individual WCI projects can have positive impacts on both the understanding and the capacity of newcomers, communities, and settlement organizations to better deal with issues related to racism and discrimination. However, it is still premature to relate these findings to the targeted outcomes of the Initiative as a whole and draw conclusions, due to the nature of the evidence and limitations of the study (discussed in Section III on Methodology).
In reviewing the findings, the reader should keep in mind that:
- The WCI is still relatively new. The Initiative had only been in operation for three years during the reporting period for the evaluation (2004/05 to 2008/09). Some intended WCI outcomes involve behavioural changes on the part of newcomers and communities (e.g., newcomers’ ability to deal with racism and discrimination; settlement sector and communities take action to reduce racism and discrimination). Such outcomes take a longer time to appear and be measureable.
- Attributing attitudinal and behavioural changes to a single program is difficult. This is an inherent problem in evaluating anti-racism programsFootnote 45. Even under controlled circumstances, it is difficult to attribute behavioural and attitudinal changes to a single program. As described in the Evaluation Assessment of CAPAR, to evaluate these programs, there is a need for direct relationships between the initiatives and a set of measureable variables that should be implemented at the early stages of projects and then monitored frequently. The impact evaluation of such programs should be completed through the evaluation of individual projects using a before-and-after design (i.e., measuring changes before, during, and/or after the intervention) or use of control groups (i.e., collection of data on changes in participants and comparing these results to another group who did not participate in the intervention). This type of research is very expensive and time-consuming, and may raise ethical concerns related to limiting access to government programs.
- There were challenges during the development of WCI. The WCI RMAF noted challenges, including “a major reorganization of the lead branch responsible for the project, the absence of funding during the first year, and the lack of strategic direction and a clearly defined set of objectives and performance indicatorsFootnote 46.” This slowed the pace at which WCI was implemented.
- The diversity of projects and activities results in a wide range of immediate and intermediate outcomes. Although flexibility of the WCI was mentioned as strength of the Initiative, the lack of consistent and well-defined outcomes and associated measureable indicators for the WCI impedes any attempt to meaningfully map and aggregate the program impacts.
- Lack of adequate administrative data and limited response to the user survey. Collecting data that accurately conveys the impact of the initiative was reported as a main challenge in CIC’s 2006-2007 and 2007-2008 reports to CAPAR. In addition, as earlier stated as a limitation of this study, it is difficult to generalize the results of data collected from WCI users and participants to all WCI project users due to the low number of respondents and the great diversity in the projects.
The data were analysed to assess program impacts related to newcomers, receiving communities, the settlement sector and the existing settlement programs. However, it was found that many WCI projects simultaneously target more than one group. These findings were identified mainly through an exploratory analysis of the collected qualitative data.
4.3.1. WCI Impacts on Newcomers
WCI projects have had a positive impact on the capacity of newcomers, particularly youth, to deal with issues related to racism and discrimination and better understand multiculturalism. In addition, WCI projects increased the capacity of newcomers to better integrate in their communities.
Workshops, seminars and events were used to reach out to newcomers in order to provide information and engage them in discussion about racism and discrimination. Young newcomers were engaged directly in multicultural and anti-racism activities through festivals, tournaments and forums, or by providing support in schools. They were also engaged more indirectly though activities involving teachers and parents. Of the 56 WCI projects reviewed for the evaluation, 30 had newcomers as at least one of their target audiences. Of these, eight had a focus on community/settlement services expansion/enhancement and eight on integration and participation. Fifteen projects were found to have a youth focus.
The expected program result for newcomers was to increase their understanding of issues related to racism and discrimination, and to develop resources and tools that would help them do soFootnote 47. According to the WCI SPOs who were surveyed, their projects helped newcomers, particularly youth, to better understand issues related to racism and discrimination, as well as how to recognize discriminatory actions and implement strategies to address them. For example, sessions in ESL classrooms on prejudice and discrimination helped newcomers learn the language to discuss these issues and their personal experiences, and to explore various ways to respond appropriately to incidents of racism. In other projects, students were trained to recognize and deal with racism, as well as to help others facing racism. In another WCI project, newcomer youth were trained to become ambassadors for anti-racism amongst their peers.
More generally, when asked to rate (on a scale of 1 to 5) the extent to which their projects expanded/contributed to the ability of newcomers to better deal with issues related to racism and discrimination, project users/participants attributed a higher rating, on average, than WCI SPO representatives (Figure 4-5).
Figure 4-5: Rating WCI Impact on Newcomers Ability to Deal with Racism and Discrimination
Project users/participants who completed the survey for the evaluation pointed to activities, such as multicultural fairs, that helped newcomers understand Canada’s social fabric and its diversity. They indicated that these events provided participants with opportunities to share their experiences, learn from others, and feel connected while celebrating cultural diversity.
Eight of the 14 projects selected for case study reported that their projects had an impact on newcomers, including youth. The activities and reported impacts of these projects are summarized in Appendix C:. Representatives from the Peel Police project were particularly outspoken about the ways that their program impacted newcomers. They indicated that the overall communication between Peel Police and newcomer communities has significantly improved since the beginning of the WCI project. As a result, the media sergeant received calls on a daily basis from newcomers who were more willing to approach the police and report issues ranging from traffic, domestic violence, family advice, sexual abuse, drugs and alcohol.
The following case studies are examples of WCI projects that impacted newcomers:
- One-day Event: Anti-racism day for Youth: This project was an anti-racism day for youth aimed at giving youth participants shared opportunities to discuss and reflect on barriers that youth face in regards to racism. In particular, this project had an impact on newcomer youth (80%) and mainstream Canadian youth (20%) between the ages of 13 and 18. In total, sixty-two youth attended. As a result of this event, mainstream Canadian youth in attendance became more aware or racism and were motivated by this event to be allies. Youth made plans for creating welcoming communities and became more aware of discriminatory behaviour and practices, while meeting new young people they could interact with in other contexts.
- Welcome Here – Phase II: Delivered by the Canadian Association of Family Resource Programs (FRP), a national, charitable organization with over 350 member organizations across Canada, the Welcome Here (I & II) project was designed to increase participation of new immigrant parents in community based programs. Some of the other objectives of this project included: sharing knowledge through resources and training; creating new partnerships among settlement agencies and family resource programs in communities across Canada; and, expanding the range of resources available to immigrant parents. As a result of this project, newcomers had greater access to resources in their own language and new immigrant parents had increased their participation in community based programs.
- Anti-Racism and Human Rights Outreach Project: This project provided educational workshops, consultations, and information about relevant issues to newcomers, businesses, human service providers, property managers and landlords, and the community at large. As a result of this project, participants were forced to think about diversity and racism, leaving the workshops with a greater awareness of the realities of racism. In addition, participants left feeling empowered and more comfortable with their abilities to deal with diversity.
4.3.2. WCI Impacts on Receiving Communities
WCI projects have had a positive impact on receiving communities, enabling them to become more aware of issues related to racism and discrimination. The findings also suggest that WCI projects have contributed to the capacity of receiving communities to be welcoming to newcomers.
Many WCI projects work with non-settlement organizations and other community members. Examples of such members include business owners, municipalities, and Canadians employed in mainstream organizations such as banks, family services, social services, and schools. These organizations and people were offered workshops, presentations, and resources to better understand the experience of immigrants and their contribution to Canadian society as well as to build skills to address racism and discrimination in their work and their communities.
An immediate outcome of WCI related to receiving communities was to increase their understanding of issues related to racism and discrimination as well as the availability of resources and tools in these communities to better deal with these issuesFootnote 48. Of the 56 projects reviewed, 39 had receiving communities as at least one of their targets. Of these, 12 focused on raising awareness and education, eight on integration and participation, four on anti-racism and diversity training, and three on resource and tool development.
When asked to rate (on a scale of one to five) the extent to which WCI projects expanded/contributed to the ability of receiving communities to better understand and address issues related to racism and discrimination, project users/participants attributed a higher rating, on average, than WCI SPOs (see Figure 4-6). Conversely, WCI SPOs attributed a higher rating than project users/participants with respect to the extent to which WCI projects expanded/contributed to the understanding of newcomer needs, problems and issues by non-settlement organizations.
Figure 4-6: Rating WCI Impact on Receiving Communities and Non-Settlement Organizations
When asked about activities that contributed to these impacts, WCI SPOs most frequently reported on the delivery of presentations and workshops, involving different organizations and the general public, which had focused on anti-racism and discrimination. Users of WCI projects also referred to WCI multicultural events that were well attended by the representatives of the community organizations and community members and promoted cross-cultural understanding in their community. Some projects provided training and educational sessions to community members who work with newcomers, such as teachers, volunteers, ESL tutors and library staff, businesses owners and other workers in community organizations. Other projects brought together immigrant and Canadian youth, encouraging them to work together and learn from each other. Representatives of art and theatre organizations in one community learned about racism and discrimination and how to incorporate some elements of multiculturalism in their work in order to raise awareness of these issues in their community and make their communities more welcome to diversity.
Nine of 14 case study projects reported generating an impact on receiving communities (see Appendix C:). One example of such a project is the Safe Harbor Program, which encouraged changes in behaviours within a group of business owners that publicly identified their businesses as a supporter of diversity and a safe place for victims of discrimination. Various businesses in that community were inspired to pursue further education and training in the area of cultural competency and to develop inclusive and supportive workplaces for newcomers.
The following case studies are examples of WCI projects that impacted receiving communities:
- Rural Community Awareness Program: This project brought volunteers from around the world (El Salvador, Mexico, African Countries etc…) to share stories, experiences, and culture to give a perspective on their journey to becoming a settled newcomer in the community. Specifically, this project targeted students in schools ranging from pre – kindergarten to college. The workshops had a humbling effect on the participants, helping to dispel many pre-conceived notions that participants may have had about how newcomers are treated and settle in Canada. Participants became more aware and more empathetic towards the plight of newcomers.
- Safe Harbour: Respect for All: This project identifies businesses and organizations as “Safe Harbours” for newcomers, signalling to the community that they understand and welcome diversity and are willing to serve as temporary sanctuaries for people who experience discrimination or harassment and briefly need a safe place to go. These locations not only model a proactive stand against racism and hate but help create a welcoming community where diversity is respected and celebrated. As a result of this project, various businesses and NGOs increased their awareness of newcomers’ settlement needs and gained a better understanding of the importance of diversity. Furthermore, this project acted as a catalyst for other ‘diversity positive’ activities in businesses or organizations (e.g. some businesses have advanced their diversity policies/procedures, sought further diversity training, produced signage in different languages, or displayed multi-faith calendars).
4.3.3. WCI Impacts on the Settlement Sector
WCI projects have had a positive impact on the capacity of the settlement sector to address issues related to racism and discrimination through the development of resources and tools, training and research to support program and policy development.
A variety of tools and resources were produced as part of WCI projects and disseminated to settlement organizations to support them in their service development and delivery. Some projects were involved in researching organizational attitudes towards newcomers, issues that immigrants face with respect to housing and health, and gaps in knowledge among service delivery staff. The representatives of these projects indicated that their findings were useful in the development of settlement policies and programs. WCI SPOs also frequently reported that the resulting tools and resources were used to help settlement providers learn about:
- Issues and experiences of newcomers and visible minorities related to racism and discrimination;
- Recognizing prejudice and discrimination in service delivery (e.g., ESL); and
- Talking comfortably about these sensitive topics with their clients.
An immediate outcome of WCI related to the settlement sector was to increase the sector understanding of issues related to racism and discrimination as well as the availability of resources and tools to better deal with these issuesFootnote 49. Of the 56 projects reviewed, 14 had the settlement sector as at least one of their targets. Of these, five focused on developing tools and resources, four on anti-racism and diversity training and three on research and policy development.
WCI SPOs and users/participants were asked to rate the extent to which WCI projects expanded/contributed to the availability of tools and guidelines to build the capacity of service providers to manage and deliver services to newcomers, as well as the availability of information and support in the development of settlement policies or the delivery of settlement programs. These findings are presented in Figure 4-7. Of note, WCI SPOs attributed a higher rating, on average, to the contribution of these projects to the management and delivery of services to newcomers than to policy development or program delivery (see Figure 4-7).
They were also asked the extent to which WCI projects expanded/contributed to the ability of settlement organizations to better understand and address issues related to racism and discrimination. Of note, WCI users/participants attributed a higher rating, on average, than WCI SPOs.
Figure 4-7: Rating WCI Impact on the Settlement Sector
Five of the 14 case studies projects reported an impact on the capacity of settlement sector to better address issues related to racism and discrimination. Two examples of WCI projects that impacted settlement sector capacity are:
- Toolbox Development Project – The Toolbox is a practical tool for smaller centres to use as they build a strategy to address the issues surrounding the attraction and retention of immigrants. The Toolbox covers the breadth of issues and information necessary for diverse smaller centres to successfully implement their strategy. This project targets community representatives within immigrant service providers, non-settlement service providers, schools, health systems, and employers. As a result project, the settlement sector and other service providing partners had a tangible tool that contributed to their efforts to build capacity and engage the community. Some users have used the tool to build upon existing initiatives, further improving the end product.
- Karibuni – The Karibuni project offers workshops and delivers education to the settlement sector, schools and workplaces to help participants explore the experiences, challenges and barriers facing newcomers to Canada. They help identify ways to build equitable and inclusive spaces for immigrants in their new communities. Organizations attended the workshops for professional development purposes; getting a better understanding of the immigrant population. After a workshop, the participants feel more prepared and educated on how to address some of the barriers related to racism and discrimination and implement the strategies they have learned.
4.3.4. WCI Impacts on Settlement Programs
WCI projects have supported the expansion or enhancement of settlement and other community services.
WCI is designed to have some positive impact on settlement programs, particularly Host and SWISFootnote 50. Twelve of the 56 WCI projects reviewed contributed to the expansion or enhancement of settlement or community services. Of these, 10 were an expansion or enhancement of Host or school-based settlement (SWIS) services, while 2 were related to other community services, contributing to the capacity of non-settlement organizations to provide appropriate services to newcomers.
According to the review of WCI projects, WCI funding supported school-based settlement (SWIS) service delivery in Alberta, Saskatchewan and Ontario, and supported Host service delivery in the PEI and Ontario. WCI SPOs identified the following areas in which WCI projects particularly supported Host and school-based settlement (SWIS) services:
- Added specific components to the settlement services to cover issues of racism directly and educate students about cultural diversity and laws in Canada;
- Partnership with the schools, libraries, community centers;
- Used the facilities of other organizations to expand service delivery of Host and SWIS; and
- Promoted the services of these programs.
4.4. Design and delivery
In reviewing the program design and delivery, the evaluation considered stakeholder perceptions, opinions and suggestions about the flexibility of the WCI design, the overall focus and strategic approach of the initiative, monitoring and reporting, communication and information sharing, and funding. The results of the document review were also employed to complete the findings. The following section presents the key findings for each topic, followed by relevant details and evidence.
Key Findings: Flexibility in design has allowed the WCI to respond to the needs of a wide variety of communities across the country. This flexibility, however, has resulted in a diversity of projects, with a broad array of activities, target groups and outcomes, making it difficult to measure and report on outcomes for the Initiative as a whole. A more focused and strategic approach to the Initiative would help to address these issues, and is timely given the move to the modernized approach to settlement programming.
The flexibility of the WCI design provides for greater responsiveness to community needs, allowing communities across the country to tailor projects to meet their specific objectives. However, the great diversity in projects has led to a wide range of outcomes and indicators that cannot be easily mapped and measured to assess program performance.
Both WCI SPOs and CIC (staff and Directors/Managers) pointed to the flexibility and adaptability of the WCI as a success factor of the program. Research indicates that issues related to racism and discrimination, promoting multiculturalism, or developing welcoming communities vary widely across rural communities as well as between rural and urban communitiesFootnote 51. This flexibility has allowed service providers to tailor the WCI projects and products to meet the objectives and specific needs of the communities targeted. The broad eligibility criteria have enabled the development of projects that vary widely in terms of target groups, activities, outputs, and intended outcomes.
The review found 56 different WCI projects, and showed that the scope of activities and target groups supported by the Initiative is extensive. Six broad themes were used to categorize the main objective and focus of activities for each project identified. However, it was also noted that many WCI projects included activities that cut across multiple themes and target audiences, and that this diversity has led to a wide range of outcomes and indicators that cannot be easily mapped and measured to assess program impacts.
The diversity of WCI projects funded made it difficult to develop the logic model during the planning for the evaluation. The lack of focus in the funded activities is thus reflected in the broadly defined outcomes and indicators in the evaluation framework that was used to guide the study. Similarly, the review and analysis that was conducted to develop the typology of WCI projects was also challenged by this lack of focus.
When asked if they agree that the current delivery structure for the settlement programsFootnote 52 is flexible enough to be able to respond to local needs, CIC program officers all agreed (3 somewhat agreed and 4 strongly agreed). However, results were more mixed for senior representatives of SPOs associated with WCI. Two agreed (1 somewhat agreed and 1 strongly agreed), two neither agreed nor disagreed, and one somewhat disagreed. CIC and provincial representatives highlighted the importance of flexibility in developing local partnerships to focus on local issues and needs as well as facilitate ongoing communication with communities.
4.4.2. Focus and Strategic Direction
The focus of WCI has evolved over the reporting period. A total of 56 projects were reviewed as part of the evaluation, with multiple target groups and activities spanning six broad themes. In light of this expansive scope, the overall focus and expected outcomes of WCI need to be revisited to be more strategic and aligned with the modernized settlement approach.
As discussed earlier, the focus of WCI has evolved over the reporting period. WCI funding has been used to support both the expansion of existing settlement programs (Host and SWIS), as well as develop anti-racism activities. Consistent with the WCI’s accountability as a CAPAR-funded initiative, the scope, objectives, and logic model for WCI were aligned with the objectives and the larger goals of CAPAR during the reporting period. As such, the focus on the development of anti-racism activities has grown over the reporting period.
When asked about WCI design and delivery, representatives of SPOs supported the overall design of the Initiative, but added that the future development of projects should be more focused and clear in terms of their strategic outcomes and how they support integration and settlement of newcomers. Most senior representatives of SPOs associated with WCI agreed (4 somewhat agreed and 1 strongly agreed) that the settlement programsFootnote 53 are well-designed to meet the needs of newcomers.
They also agreed that the objectives of the settlement programs,Footnote 54 the roles and responsibilities of the service providers, and accountabilities for the Initiative are well-defined (3 somewhat agreed and 3 strongly agreed). However, most SPO representatives were unsure in what ways WCI contributes to SWIS, Host and ISAP, and (apart from their project) what specific activities are funded through the Initiative. Regional CIC representatives noted that the lines between WCI projects and other settlement programs have blurred over time, overlapping particularly with ISAP B activities.
However, some CIC representatives expressed concern that the great diversity in projects makes it difficult to measure the progress made against the objectives and intended outcomes. In addition, there was concern that the diverse approach and the small size of the Initiative diminish the overall impact. To address this, it was suggested that clear strategic priorities need to be developed to better guide the selection of projects.
These findings are consistent with information reported in the program profile of the 2006 WCI RMAF which underlined that “the Initiative lacked strategic direction and a clearly defined set of objectives and performance indicators”, as well as in the 2007/08 annual report to CAPAR, which highlighted “a demonstrated need among partners and stakeholders for further guidance on the intended outcomes of the WCI.”
The need to revisit the strategic approach of WCI and its outcomes was amplified with the 2008 announcement regarding the modernization of settlement services. The modernized approach, which is intended to improve settlement and longer-term integration outcomes, establishes one comprehensive settlement program that works towards common goals and outcomes for newcomers. In this approach, WCI is central to the Community Connections stream. The intermediate outcomes related to the Community Connections stream include: 17. Clients are connected to the broader community and social networks and 18. Program participants are aware of newcomers’ needs and contributions and are engaged in newcomer settlement. The long-term outcomes associated with this stream include: 21. Canadians provide a welcoming community to facilitate the full participation of newcomers into Canadian society and 22. Newcomers contribute to the economic, social and cultural development needs of CanadaFootnote 55.
Subsequent policy work related to WCI has focused on defining and developing this stream of settlement programming. In 2008, CIC held a series of workshops in regional centres across the country on the role of social engagement in integration. One of the objectives of these discussions was “to clarify and clearly articulate foundational principles that should guide policy and programming approaches to social engagementFootnote 56.” During these consultations, some of the discussions focused on defining the concept of a community. More recently, research was commissioned to understand key characteristics of a welcoming community. Thus, WCI has been in a transition phase during the latter part of the reporting period.
CIC Managers and Directors predicted that, under the modernized approach, WCI should be able to retain its flexibility but improve its focus, expectations, outcomes and evaluability. WCI SPOs suggested that CIC should hold community consultations in order to gain input and feedback from different agencies regarding future strategies for WCI.
4.4.3. Performance Measurement and Reporting
Performance measurement and reporting has been a consistent challenge for WCI. The diverse nature of the projects, particularly in terms of expected outcomes, and the absence of standardized performance indicators and accompanying data make it difficult to assemble an accurate, aggregate picture of WCI performance and impacts.
WCI has a requirement and structure in place for monitoring and reporting on WCI performance. All WCI-funded projects report their results to CIC for departmental and CAPAR accountability and reporting purposes. Then, CIC NHQ prepares an annual report based on the CAPAR reporting template. Table 4-7 presents the three layers of WCI reporting.
|WCI SPOs to CIC Regional Office||
|CIC Regional Office to NHQ||
|CIC NHQ to CAPAR||
As stated in 2006/07 and 2007/08 CIC reports to CAPAR, performance measurement and reporting has been a major challenge for WCI. CIC NHQ attempted to institute a robust performance measurement strategy for WCI, including a questionnaire to be completed by CIC regions and a template participant questionnaire to collect participant feedback. However, this strategy was revised in response to regional concerns that it was onerous. As a result, a shorter questionnaire was administered to CIC regions to report on 2006/07 WCI projects, outputs and outcomes, but yielded inconsistent information.
A different approach was taken to report on the 2007/08 and 2008/09 funding years. CIC-NHQ requested copies of Contribution Agreements and SPO reports in order to generate information on WCI projects. However, according to the 2008/09 annual report to CAPAR, “the performance information and level of detail received through these documents varie[d] between projects”, not allowing for an adequate national reporting strategy.
Monitoring, performance measurement and reporting for WCI have been difficult due to the number of factors, including:
- The flexibility of the WCI funding approach and diversity of the projects subsequently funded. The broad array of activities, target groups and outcomes has hindered the development of standard performance indicators for WCI. Reporting on WCI activities and progress has been largely narrative and inconsistent in content.
- Difficulties in aligning CIC departmental and CAPAR reporting requirements. When first implemented, most WCI-funded activities were recorded via the regular reporting structures for other CIC-funded settlement programs.
- Challenges with tracking WCI funds and activities. Separate financial coding for WCI was not established until 2007; however, it was not used consistently during the reporting period. Moreover, WCI is not explicitly captured in CAMS. This makes it difficult to distinguish WCI projects from other activities supported by the settlement programs (coded as either ISAP or Host).
As a result, there is not sufficient data to adequately assess the impacts of WCI as a whole.
Views were mixed among CIC program officers with respect to the approach to monitoring, reporting and performance measurement of the WCI. One strongly disagreed, while three neither agreed nor disagreed and one somewhat agreed that this approach is appropriate. They noted that, although the reports are collected continuously, the current reporting strategy is largely based on outputs rather than outcomes with little information reported on the impacts and outcomes of the projects.
In addition, half of the CIC program officers somewhat agreed that the WCI is supported by adequate tools, resources and the mechanisms needed to deliver the Initiative effectively, while one strongly disagreed and one neither agreed nor disagreed. CIC representatives explained that while the resources, such as the RMAF, were helpful in specifying objectives, outcomes and outputs for WCI, adequate tools and resources for data collection and performance monitoring have not been provided. They suggested that the reporting requirements should incorporate more quantitative data and project-specific outputs. For example, all projects that deliver services directly to clients should track and report quantitative data on clients served and services provided.
Views were more mixed among senior representatives of SPOs associated with WCI. Two agreed that the tools and resources to support the delivery of settlement servicesFootnote 57 are effective (one somewhat agreed and one strongly agreed), while two neither agreed nor disagreed and one somewhat disagreed. WCI SPO representatives noted that reporting requirements are time-consuming and the guidelines are not clear. Data collected as part of WCI reporting is not used effectively to measure the performance of these projects, or to provide SPOs with feedback on project success. The absence of long-term tracking strategy for projects, once they are completed, contributes to further difficulties in performance measurement.
During the interviews, surveys and case studies, participants highlighted some factors that may limit the actual or perceived progress made by the Initiative, including:
- The overlap between WCI and other settlement programs makes it difficult to isolate its impacts. The evaluation findings of ISAP show that almost one-third of SPOs delivering traditional ISAP services reported activities similar to those delivered under WCI such as group sessions, conversation circles, workshops and individual counseling on anti-racism and discrimination. One-third of representatives involved with ISAP B projects also reported activities similar to WCI and about one-half of Host SPOs reported that volunteers received training related to cultural sensitivity and anti-racism. SPOs expressed uncertainty whether these activities were delivered as part of their traditional settlement services or supplemented by WCI projects.
- The lack of common data collection and comparable data across the projects and regions makes it difficult to conduct a comparative analysis and measure program success.
- The lack of clear direction on expectations and outcomes makes reporting onerous and time-consuming for individual projects. Senior SPO representatives noted that a lack of a clear perspective on the overall effectiveness of the Initiative and best practices makes it difficult to plan and improve existing tools and activities.
- Over the course of this evaluation, it was evident that the lack of data at the Initiative level limited key informants in providing a clear opinion on various aspects of WCI. It was also difficult for participants to refer to any analysis and data set in order to draw a comprehensive picture of the program or speak about best practices and benchmarking of projects.
4.4.4. Communication and Information-Sharing
The level of communication and information-sharing within CIC and between CIC and SPOs is limited.
CIC program officers and senior representatives of SPOs associated with WCI were asked whether they agreed or disagreed with the statement that there is an appropriate level of communication and information-sharing between CIC and service providersFootnote 58. Half of CIC representatives disagreed with this statement (one somewhat disagreed and two strongly disagreed), while two somewhat agreed. They noted that there are not strong mechanisms, or avenues in place to share information, best practices and program success. The communication and information sharing about WCI projects is mostly done through contribution agreements and one-time final reports that provide little follow up on best practices and lessons learned and no opportunity for discussions with service providers about the planning for future actions.
Senior representatives of SPOs associated with WCI provided a more positive view. Two-thirds agreed with the statement (two somewhat agreed and two strongly agreed). They explained that CIC had successfully communicated the availability of the funding as well as the roles, responsibilities and mandate for the WCI. However, they also noted that no further communication, feedback or advice was provided regarding the progress, performance and future plans for the Initiative. Several examples were provided where similar workshops were delivered by different organizations in the same community within a short period of time.
WCI SPOs also provided a few suggestions to improve communication and information-sharing, including:
- Conduct community consultations in order to gain input and feedback from different agencies.
- Organize information sessions with organizations to apprise them of CIC's programs.
- Encourage more CIC staff to participate in the project activities such as workshops and conferences.
- Develop materials such as a booklet or on-line resource to showcase all of the projects funded under WCI, inspire communities and experts to participate, and give the WCI SPOs a sense of accomplishment.
4.4.5. Level of Funding
The level of funding is generally perceived to be appropriate for current activities. However, some SPOs suggested that more funding should have been available to support the management of projects, promotional activities, and the dissemination of the project outputs.
CIC Managers and Directors said the level of funding was appropriate for the current activities, but expressed concern whether long-term funding would be available to support ongoing activities. However, two representatives strongly disagreed that existing funding is adequate, adding that not enough resources and expertise are available to support the management of the WCI projects at the regional level.
When asked about the adequacy of the funding available to support the delivery of the settlement programsFootnote 59, two-thirds of senior representatives of SPOs associated with WCI agreed (two somewhat agreed and two strongly agreed), while one-third somewhat disagreed. WCI SPO representatives argued that although funding has been available, the funding schedule (when the CAs are signed) and lack of clear long-term commitments for funding makes it difficult for them to start their project activities on time and to plan for the future.
For example, it is challenging to plan for school programs to be delivered when the school year is different than the fiscal year schedule. Several representatives emphasized the importance of allocating additional funding to increase awareness and disseminate project outputs, particularly for projects focused on developing tools and resources. They also suggested that money should be provided to SPOs earlier in the fiscal year so that they can commence on time with the WCI activities and avoid shuffling budgets from one program to another.
Report a problem or mistake on this page
- Date modified: