Evaluation of the Settlement Workers in Schools (SWIS) Initiative

Research and Evaluation Branch
Immigration, Refugees and Citizenship Canada
Project reference number: E1-2020

February 2022

Executive summary

Background

This report presents the findings of the evaluation of Immigration, Refugees and Citizenship Canada’s (IRCC) Settlement Workers in Schools (SWIS) initiative. The evaluation was conducted in fulfillment of requirements under the Treasury Board’s Policy on Results and covered fiscal years 2017/2018 to 2020/2021. The evaluation represents the department’s first formal evidentiary examination of SWIS outcomes.

The evaluation’s primary focus was to assess the design, implementation and effectiveness of SWIS, including how SWIS is delivered across regions (e.g., activity types, delivery models). The evaluation also included a Gender-Based Analysis Plus (GBA+) lens, as well as considerations of COVID-19 impacts on SWIS design and delivery.

Summary of conclusions

The evaluation highlighted a clear and demonstrable need for SWIS. SWIS provides newcomer youth, children and their parents/guardians with school-based settlement services that are essential for integrating into Canadian society, and the Canadian education system. Moreover, the Government of Canada has an evident role in providing SWIS, despite education being a provincial/territorial responsibility, as the Federal Government is responsible for newcomer integration.

SWIS makes use of unique delivery models which allow service providers to adapt their programming for regional, provincial/territorial and local priorities. However, the evaluation found this flexibility creates challenges for reporting on initiative outcomes, comparing different service providers, and understanding what SWIS interventions work best for whom, and under what conditions.

Despite data limitations, the evaluation found clients perceive SWIS services to be useful and responsive to their needs, for instance by contributing to increased knowledge, involvement and performance in the education system. SWIS is also useful for referring clients to other settlement services needed on their integration journeys, and for enhancing cultural understanding on the part of school staff.

While the evaluation’s findings generally showed SWIS contributes to newcomer settlement and integration, there are ongoing gaps in the department’s capacity to report on the initiative’s delivery and success. Fine tuning to SWIS’ design and management would enhance the department’s ability to monitor and report on SWIS outcomes. To this end, the evaluation put forward three recommendations.

Summary of recommendations

Current policy guidance does not specify required or core features of SWIS, nor does guidance contain an exhaustive list of SWIS interventions. While broad policy guidance is praised for its flexibility, it results in inconsistent activity offerings across Canada and a lack of common understanding of which activities constitute SWIS.

The evaluation identified extensive variability between Service Provider Organizations (SPOs) in how Needs and Assets Assessments and Referrals, Information and Orientation and Community Connections activities are being delivered, including the comprehensiveness of needs assessments and the frequency of SWIS interventions. Newly developed policies that better balance flexibility with standardization would help contribute to a higher level of consistency across Canada in how services are delivered, as well as help identify SWIS activities more clearly, supporting stronger data collection and sharing of best practices.

Recommendation 1: IRCC should confirm and implement a common definition of SWIS with core services/activities supported by clear policy/guidelines.

Current SWIS data collection procedures struggle to distinguish SWIS services from other settlement services, and efforts to systematically identify contribution agreements with SWIS components have not been fully implemented. Executing a reliable strategy to report on SWIS interventions and SWIS-serving organizations is crucial for reporting on and monitoring SWIS success.

The evaluation found a high level of variability in how different service providers report on SWIS interventions, even when conducting similar activities, and service providers are not always reporting on the same client types, including temporary residents. Moreover, as school staff are a key client group, IRCC should ensure reporting for activities delivered to this group.

Enhancing data collection will ensure IRCC is well-positioned to conduct stronger assessments of SWIS performance, in support of evidence-based policy decisions, improving SWIS design, and delivering more effective services.

Recommendation 2: IRCC should:

  1. Develop and implement a strategy to clearly capture the core SWIS activities and services and SPOs delivering them, identified as a result of Recommendation 1, in Immigration Contribution Agreement Reporting Environment (iCARE) and Grants and Contributions System (GCS); and
  2. Implement and disseminate updated policies and guidance for SWIS-related data entry procedures.

The evaluation identified duplication between SWIS activities and other services in IRCC’s settlement suite, as well as services offered in the community and by schools. While duplication is not inherently bad, there are challenges in understanding the full extent of duplication and whether it is addressing a need.

As IRCC does not require funding recipients to establish agreements with the school boards they serve, it is difficult to determine how the services offered by schools compare with the services offered by IRCC, creating a risk of overlapping roles and responsibilities. Moreover, IRCC does not mandate information sharing with schoolboards (or SWIS SPOs) with respect to clients served, which presents challenges in attributing successful education and settlement outcomes to participation in SWIS activities.

Another challenge facing SWIS is provision of services to ineligible clients. Under current policy guidance, no clients are denied services, regardless of their immigration status as a result of the in-kind contributions of schools. Presently, reporting on in-kind contributions are linked to contribution agreements as a whole rather than SWIS specifically, making it hard to assess value for money. Combined with reporting guidance, there are challenges in assessing the extent to which serving ineligible clients is impacting SWIS worker workload.

Consistent reporting on SWIS clients and activities would improve IRCC's ability to assess and attribute client outcomes to SWIS, and understand the trade-offs of in-kind contributions and serving ineligible clients.

Recommendation 3: IRCC should:

  1. Explore SWIS policy changes to ensure more standardized information sharing with IRCC on SWIS clients and activities; and
  2. Review and clarify the department’s position, policies and procedures around providing SWIS services to TRs to ensure a consistent approach across service provider organizations.

Page details

Date modified: