2023 Settlement Outcomes Report: Part 2 – Digital Transformation

Download: Part 2: Digital Transformation (PDF, 1.9 MB)

On this page

• Introduction
• Responsiveness of online settlement services
• Effectiveness of online services
• Conclusion
• Data sources
• End notes

Introduction

In recent years, online settlement services have become an increasingly important mode of service delivery. Prior to the COVID-19 pandemic, the vast majority of settlement services were offered in-person with the exception of pre-arrival services. In March 2020, IRCC instructed service provider organizations (SPOs) to focus on providing non-critical services remotely whenever possible. As a result, SPOs transitioned programs and services traditionally offered in-person to either remote or hybrid formats. By autumn 2020,^{Footnote 1} 97% of domestic direct service providers reported offering at least some services online.^{Footnote 2} Since then, some service providers have continued offering their services remotely, while others have either returned to in-person services or adopted hybrid approaches.

To better understand the opportunities and limitations arising from the use of remote services, IRCC embarked on a Digital Case Study (DCS) research project which explored the contexts in which Settlement Program services could be offered online, which newcomer clients could be best-served through this channel, and whether online services can be offered in a responsive, effective, and efficient way. The project applied a mixed-methods approach by analyzing existing quantitative and qualitative data as well as generating new data. The project primarily focused on domestic and pre-arrival Information and Orientation (I&O) services and Employment-related Services (ErS), but also examined other service types.^{Footnote 3} The research compared in-person services (i.e. services offered face to face in a physical location) with remote services (i.e. an umbrella term that captures any service not being offered in-person). Remote services include online services which are a high-tech form of remote service delivery, as well as via telephone, email, or text services which are considered a low-tech form delivery. Hybrid forms of services were also examined, not as a distinct format but rather as a combination of service formats.

Remote Services is an umbrella term that captures both high-tech and low-tech forms of service.

Responsiveness of online settlement services

During the study period between fiscal years 2020-21 and 2021-22, clients’ increased use of remote service formats was related in part to their increased availability, as well as the restrictions on accessing in-person services during the pandemic.

Remote service delivery was the top format for domestic clients at the height of the pandemic. SPOs returned to offering in-person services as physical distancing requirements

Remote services continued to be the dominant form of service delivery for pre-arrival clients in the years following the of the start of the pandemic.

As online services become a more regular service delivery channel of Settlement Program interventions, it is important to understand the extent to which newcomers are able to use and benefit from these services. Newcomers may face barriers acquiring digital devices, getting onto the internet, or simply navigating a new format of service. For any of these reasons, or as a matter of preference, clients may not use online services. One the other hand, online services may be preferred by others. This section explores how responsive IRCC’s online settlement services can be to these realities.

Client access

Although technological advancements make online services more readily and broadly available, access to technology, digital literacy, language proficiency, and a lack of reliable broadband are often cited as major barriers for newcomers.^{Footnote 4} As a result, online services can be perceived as introducing more challenges to newcomer clients. However, the research also highlights the many positive impacts of technology on information dissemination, employment opportunities, skill and capacity building, and language training.^{Footnote 5} In order for newcomers to effectively use IRCC-funded online settlement services, it’s important to understand if clients are able to acquire a digital device (e.g. laptop, mobile phone, tablet) and use the internet.

The vast majority of survey respondents owned a digital device to access settlement services.

More clients reported owning a mobile phone than any other device.

Multiple lines of evidence indicate that the majority of newcomers have the ability to use a digital device and access the internet. According to IRCC’s Remote Service Delivery Survey (which surveyed service providers at the onset of the pandemic), most clients were able to access online services. Likewise, among respondents to the Digital Case Study client survey, 97% of the 18,777 respondents reported having access to their own device to use settlement services. Notably, more clients reported owning a mobile phone than any other device.^{Footnote 6} Similarly, a study of IOM refugees who used IRCC-funded pre-arrival services found that 80% owned a mobile phone and 70% always or often had internet access. Today, many newcomers either own a smartphone or have access to one (e.g. through a family member or neighbour). This also means that they possess some degree of knowledge regarding how to use a device and access the internet. However, the degree of knowledge can drastically differ depending on personal, social, and economic factors. For example, in the Digital Case Study Client Survey only 60% of those aged 65+ said they were comfortable using the internet, compared to 90% of those aged 35-64 and 94% of those aged 18-34. Similarly, the type of online activity makes a difference. When asked about recent online activities in the past month, the most common activity was sending or receiving an email (low-tech) and the least common was taking an online class (high-tech). Ultimately, while most newcomers use digital devices and the internet, accessibility of online services will vary based on skillsets, experiences, and demographic factors. This is where client supports may be needed to assist some newcomers.

Fewer clients were using support services when they access remote services, especially CC.

Client supports

IRCC traditionally provides several types of support services: child care, translation, interpretation, transportation, short-term counselling (previously known as crisis counselling), and provisions for disabilities. Support services are intended to enable clients to access the settlement services they need to settle and integrate into their communities. In the years of the pandemic, support services accounted for approximately 9% of annual Program spending.^{Footnote 7}

Between 2020-21 and 2021-22, clients who accessed services remotely used fewer overall support services compared to those who accessed services in-person. This is especially true for domestic clients of Employment-related Services and Community Connections (CC) who accessed significantly fewer childcare and transportation supports than their in-person counterparts.

The Digital Case Study Client Survey and client focus groups provide some relevant insight to better understand these findings. For example, survey respondents noted that the main benefits of remote services included not needing to commute to a service location, and not needing to seek childcare. Digital Case Study client focus group participants also noted the benefits of on-demand remote services for flexible scheduling that was helpful for working newcomers and those with caring responsibilities. Participants in the service provider focus groups also indicated that they could serve more clients from a wider area once transportation was removed as a barrier.

Interpretation and short-term counselling services were the only supports that were accessed at higher rates remotely than in-person.

While clients used fewer overall supports in remote settings, the picture looks slightly different when examining the different types of supports. For example, more clients accessed short-term counselling remotely than in-person – the vast majority of which were female clients. In fact, regardless of service format, more female clients used support services compared to other genders. Consequently, changes to the delivery format of support services may impact female newcomers differently.

It’s clear that support services are still needed in remote settings.^{Footnote 8} This is especially true as some newcomers have challenges accessing supports online. In fact, among the 6% of Digital Case Study Client Survey respondents who indicated they had problems using remote support services, the majority of them relied on a family member, friend, or service provider to help them (e.g. for interpretation). This indicates that support services, and the service providers who offer them, remain an important vehicle for some clients to access settlement services. Overall, the shift to remote delivery has meant some newcomers continue to use traditional support services, while others may no longer need them (e.g. working clients, ErS clients, long-distance clients).

Beyond the traditional support services that IRCC typically provides, a new type of client support emerged as a necessity during the pandemic. Digital skills training are areas that were repeatedly referenced by SPOs and clients across multiple lines of evidence. According to SPOs, some clients did not have the necessary knowledge or skills to independently use online settlement services.^{Footnote 9} In response, SPOs provided various forms of digital literacy and skills training to clients as a separate service or as an introductory lesson to an existing service. While the evidence suggests that the majority of settlement clients do not require this type of support, the need for digital literacy and skills training appears to be concentrated among certain client demographics. Newcomers who fell into one or more of the following demographic groups were identified as needing additional supports to use online services: resettled refugees, seniors, and clients with lower education/literacy levels. These groups, more than others, had challenges understanding how to use online services.

The top issue with remote services varied by survey respondents’ demographics. For resettled refugees, clients with secondary school or less education, and seniors, the #1 issue related to digital literacy/skills.

The abovementioned issues can be mitigated by providing clients with supports to access services like digital skills training, or by delivering services in-person. Trade-offs exist with both options and may even be context-specific. For example, a study of IOM refugees using IRCC-funded pre-arrival services found that digital technologies provide a range of benefits, but critical human elements which technology cannot replace are needed to support refugee clients.

Ultimately, newcomers have varied experiences, skills, assets, and needs. Even within specific demographic categories like resettled refugees, many differences exist. The evidence provides an indication of which clients could be in need of additional supports; however it is not prescriptive. Every newcomer’s circumstances, needs, and preferences will differ.

 

Client preferences

Beyond client access and capabilities, newcomers have agency to choose the format of service that best meets their needs. Prior to the pandemic, the vast majority of settlement services were offered only in-person (with the exception of pre-arrival) but physical distancing measures completely changed this trend – with remote options suddenly becoming the necessary format. Now, SPOs have begun offering multiple formats for different services. In turn, clients have experienced a range of options for how services can be delivered.

To better understand client preferences, newcomer respondents to the Digital Case Study Client Survey were asked how they wanted to access different settlement services.^{Footnote 10}

The top services that respondents preferred to access in-person were social activities with the community (a CC service) and employer networking (an ErS service).

Many clients appear to want to access Employment-related Services in-person (e.g. employer networking, resume & interview services, employment counseling); however these preferences vary across immigration category. Clients from economic categories generally prefer online formats more than clients from non-economic categories. The exception is employer networking where all clients have similar preferences to access these services in-person.

Findings from the Digital Case Study client focus groups contextualize some of these preferences. Participants noted that services which emphasize socialization and body language were best-suited for in-person delivery.

These are services where interpersonal skills and social cues are important such as interview practice, mentorship, and developing friendships in the community.

In contrast, focus group participants noted the very practical benefits of receiving Information and Orientation services online. This included factors such as convenience, flexibility, efficiency, as well as ease of access. For example, clients who lived significant distances from service locations found it easier to access services.

Focus group participants explain why newcomers prefer to access information online:

Convenient and flexible

• Online services were easier to accommodate in clients’ schedules (e.g. working clients, pre-arrival clients preparing for departure to Canada)
• On-demand services (e.g. pre-recorded sessions) were convenient for busy clients
• Clients with health concerns (e.g. immunocompromised) were able to participate remotely

Time and cost efficient

• Less time or money spent on travelling to the service location
• Rural/remote access services more easily when commuting barriers are removed
• Some clients felt they could attend sessions without worrying about the cost of childcare

Easier to access information

• Information available publicly, freely, accessible at any time (online) has made it easier for clients to learn about Canada
• Online information can be easier to navigate at clients’ own pace
• Recorded sessions allow clients to access information anytime – not just at a session

Source: IRCC (2022) DCS Client Focus Groups

Above all, newcomer clients prioritized receiving the right service, regardless of the format in which it was provided. In the Digital Case Study Client Survey, only 10% of respondents said they would look for another organization if a service wasn’t offered in their desired format. Being able to access the service in a way that meets their individual needs is what clients prioritized the most, and that can vary across service formats.

Effectiveness of online services

To assess the effectiveness of online services, it’s important to understand how IRCC measures the performance of the Settlement Program. The Settlement Program logic model identifies the different outcomes that settlement services aim to achieve. Since Information and Orientation as well as Employment-related Services were the primary focus of this research, the immediate outcomes of interest were: knowledge of life in Canada and knowledge of working in Canada.

In order to assess client outcomes by format of service, administrative data from iCARE was combined with responses to several questions in the Newcomer Outcomes Survey to create an average index score across several topics. Regression analysis was also conducted to determine the impact of service format on the outcomes of interest, while controlling for relevant demographic characteristics of clients. Notably this analysis was only done for domestic settlement services.^{Footnote 11}

Knowledge of life in Canada

Information and Orientation services are designed to increase newcomers’ knowledge of life in Canada. During the study period, clients accessed I&O services both in-person and remotely. In-person formats returned as the more common form of service delivery as physical distancing requirements eased. To assess and compare the effectiveness of these formats (including combinations of formats – i.e. hybrid), an index of information topics was created using the Newcomer Outcomes Survey. Therefore “knowledge of life in Canada” represents an index of newcomers’ knowledge (scored out of 100) on the following topics: housing, healthcare, education, money/finance, and law/justice.

Descriptive statistics show that clients who used online I&O services had a higher average index score for knowledge of life in Canada than those who used other formats. After controlling for relevant demographic and socioeconomic characteristics, regression analysis also confirmed these results.^{Footnote 12} In fact, clients who used online I&O services had higher scores even compared to those who used various hybrid formats of service delivery.

Clients who accessed I&O online had stronger knowledge results compared to other formats.

Source: iCARE-NOS data

There are some notable findings within demographic groups. For example, working-aged clients (aged 35-64) who used online services had higher knowledge of life index scores than youth and seniors. This supports an earlier finding from the Digital Case Study Client Survey and focus groups that noted working newcomers found online services helpful because it complemented their schedules. Notably, economic immigrants who used I&O services online had the highest average knowledge of life index scores compared to any other immigrant category, even after controlling for other demographic and socioeconomic characteristics.

Knowledge of working in Canada

Clients also have more access to remote Employment-related Services than ever before. In fact, remote services were the most common form of ErS delivery for domestic clients, with low-tech remote services becoming more popular after physical distancing requirements eased. In order to assess the effectiveness of online ErS, an index of employment-related topics was created using the Newcomer Outcomes Survey. Therefore “knowledge of working in Canada” represents an index of knowledge (scored out of 100) on the following topics: how to apply for jobs, establishing networks, getting education assessed, starting a business and soft skills.

Descriptive statistics show that clients who used online services to access ErS had higher average index score for knowledge of working in Canada compared to those who used other formats. After controlling for relevant demographic and socioeconomic characteristics, regression analyses also confirmed these results for ErS.^{Footnote 13} Similar to the I&O findings, clients who used online ErS services had higher knowledge scores even compared to those who used hybrid formats of service delivery.

Clients who accessed ErS online had stronger knowledge results compared to other formats.

Source: iCARE-NOS data

Therefore, both ErS and I&O newcomer clients have stronger knowledge scores when accessing services online compared to other formats. However, some limitations exist with this analysis. To begin, this analysis examines the relationship between knowledge scores and service format. Other variables can still affect knowledge scores despite having controlled for relevant socioeconomic and demographic factors. For example, digital literacy is one external variable that could potentially explain the relationship between knowledge scores and format of service (e.g. those with stronger digital skills may also have more knowledge of life in Canada despite the format of IRCC-funded settlement services). While immigration status and education level can be used as a proxy for digital literacy, they are far from perfect substitutes. Therefore other variables not investigated in this analysis should be considered in making sense of these findings. With that said, these results still provide very promising indication that online delivery can be a more effective form at of service for I&O and ErS.

Efficiency of online settlement services

To assess the efficiency of online settlement services, it’s important to understand if they are more or less resource intensive compared to other formats (e.g. compared to the popular in-person format that existed prior to the pandemic). To make this assessment, this analysis explored if any savings were realized during the shift to remote service delivery, as well as examining other program delivery considerations that may affect future resourcing needs.

The cost of shifting online

Because IRCC does not deliver settlement services directly, service providers are better-placed to report on resourcing needs associated with a change in program delivery. Therefore, the following high-level assessment relied heavily on the insights of service providers through focus groups, as well as through the data they submitted via program reporting mechanisms. Overall, service providers reported that online services could entail future savings, for example, reducing costs in real-estate/rent associated with physical service locations. However, SPOs also noted that the shift to remote service delivery created upfront costs related to acquiring new technology, making changes to services, and hiring new staff (e.g. IT Support). These costs were realized by SPOs as they transitioned to remote service delivery. Additionally, the ongoing cost of delivering multiple service formats and/or hybrid formats simultaneously also had an impact on resources. In other words, continuing to offer both in-person and remote options can be redundant and costly. As a result, service providers reported no net savings during the study period, which also entailed new costs associated with the pandemic. As online and hybrid forms of service delivery become increasingly popular, it will be important for cost estimates and budget allocations to be revisited accordingly.

Service providers saw no net savings in the shift to remote service delivery.

Program delivery considerations

Beyond the factors noted above (e.g. software, hardware, infrastructure) and earlier in this section (e.g. digital literacy and skills supports) other program delivery considerations exist for online services that will have resource considerations.

Privacy and security: This was an area referenced repeatedly by both service providers and clients. For clients, these challenges related to sharing personal information, ensuring online safety, and knowing how to trust an online service provider. SPOs also noted some challenges associated with delivering remote services which included establishing new security protocols and managing client data remotely. The Settlement Sector & Technology Task Group also stressed the importance of establishing a sector-wide privacy, security, and confidentiality compliance framework related to digital service delivery.

Digital standards: As online services or tools are delivered to newcomer clients, they should consult the principles outlined in the Government of Canada’s Digital Standards. These standards emphasize critical steps such as designing services with users, iterating often, building in accessibility from the start, and ensuring good data stewardship.

Continuous improvement: As new ways of delivering services continue to expand, so too will service providers’ need to test new approaches and build capacity. This will involve researching, piloting, iterating, as well training personnel. Similarly, the Settlement Sector & Technology Task Group emphasized the need for consistent training of staff and sharing resources and learnings across the sector to build collective capacity.

Overall, the discussion around resource considerations and the efficiency of online settlement services is ongoing. Despite the sudden shift to remote service delivery, the sector is in a transitionary stage that is not necessarily reflective of the past or the future. Instead, the focus continues to be learning what works well, focusing on what needs to be studied, and identifying where more investments are needed to arrive at a delivery model that works best for clients.

Conclusion

What largely began as an operational response to a crisis, has now laid the groundwork for a digital transformation of settlement services. Remote service delivery, and particularly online service delivery, has shown some promising results for the benefit of newcomers. The evidence suggests that online delivery of Information and Orientation services benefits newcomers (e.g. comparatively more convenient, accessible and effective than in-person formats). In fact, newcomers prefer to access information about life in Canada online more than any other format. Similarly, Employment-related Services also had promising results from a service delivery and effectiveness perspective, however newcomers still prefer to access some of interpersonal aspects of these services in-person (e.g. employer networking).

However, while the research suggest that most newcomers will be able to access online services (i.e. through a mobile phone), some may face significant barriers. Specifically, newcomers who fall into one or more of the following groups may face challenges understanding how to use online services: resettled refugees, seniors, and clients with lower education levels. This is where digital skills supports could benefit newcomers. Through various initiatives (e.g. IRCC-funded pilots, Settlement Sector & Technology Task Group identified projects) the sector is already exploring different ways to deliver these types of supports.

Service providers invested a lot of effort and resources into adapting their services to meet the needs of newcomers. The quick transition to remote service delivery resulted in significant upfront costs in areas such as software, hardware, IT infrastructure, and personnel. While online services have potential for future savings (e.g. in real estate and program efficiencies), this will need to be an ongoing assessment. Finally, as services continue to be delivered online, service providers will need to consider other important aspects of operating in a digital world including the importance of privacy and security.

Ultimately, the digital transformation of settlement services is an area ripe with opportunity that could be seized for the benefit of newcomers, while also ensuring that no one is left behind.

Data sources

This following contains information about primary data sources used in the report – i.e. data collected directly by IRCC. Many other sources of information were used in the development of this report. Secondary data sources (i.e. data collected or analysis done by other parties) are referenced directly in the report.

IRCC Permanent Resident Database

The IRCC Permanent Resident Database is based on the Global Case Management System (GCMS). GCMS is IRCC’s single, integrated and worldwide system used internally to process applications for citizenship and immigration services.

This report uses GCMS information about newcomers to Canada:

• who were admitted as permanent residents between 2013 and 2020, or
• who used settlement services in 2020-21 and 2021-22.

The data were extracted between December 2022 and June 2023.

iCARE

The Immigration Contribution Agreement Reporting Environment (iCARE) is a data entry system that collects key characteristics of services used by clients of the Settlement and Resettlement Programs, including Pre-Arrival. The data is entered by Service Provider Organizations (SPOs) and they are required to report monthly as per their Contribution Agreement with IRCC. iCARE has been used to collect Settlement Program data since 2013 and Resettlement Program data since 2014.

The data were extracted between December 2022 and June 2023.

Grants and Contributions System (GCS)

The Grants and Contributions System (GCS) is an online tool that allows Service Provider Organizations (SPOs) to submit applications for funding, as well as manage Grants or Contribution Agreements.

This report uses GCS information about Contribution Agreement funding and reporting for 2020-21 and 2021-22.

SAP

SAP is the Department’s financial management system used to support the preparation of financial statements and other reporting.

This report uses SAP information about settlement program expenditures in 2019-20, 2020-21 and 2021-22.

The data were extracted March 2023.

Newcomer Outcomes Survey (NOS)

The Newcomer Outcomes Survey (NOS) is an annual survey of newcomers to Canada that collects settlement outcomes information from both clients and non-clients of IRCC’s Settlement Program. Each year, the survey is sent to all newcomers who became permanent residents in specific years. Two years of survey data are combined to provide a response set from newcomers across eight admission years.

This report uses NOS responses to the 2020 and 2021 survey waves about the settlement experiences of newcomers who became permanent residents between 2013 and 2020.

The data were extracted March 2022.

ARPIO / APRCP

SPOs delivering direct settlement services provide an Annual Report on Project Implementation & Outcomes (ARPIO) and Local Immigration Partnerships/Réseaux d’immigration francophone provide an Annual Project Report on Community Partnerships (APRCP). The reports provide an equitable opportunity to hear from SPOs across the country on similar issues. Note that the ARPIO was formerly called the APPR – Annual Project Performance Report.

This report uses ARPIO and APRCP reports about Contribution Agreements funded for 2017-18, 2018-19, 2020-21 and 2021-22.

The data were extracted after the end of the reporting cycles.

2020 Remote Service Delivery Survey

IRCC conducted a Remote Service Delivery Survey of Service Provider Organizations (SPOs) in November 2020 to better understand what the delivery of direct settlement services looked like in the COVID-19 context for SPOs receiving IRCC funding.

This report uses survey information about the impact of the transition from in-person to remote service delivery.

The data were extracted January 2021.

2022 Digital Case Study Client Survey

IRCC conducted an online survey of client experiences using online settlement services in August 2022, to better understand which settlement services were working well, and what needed improvement.

This report uses survey information about client experiences using settlement services prior to August 2022.

The data were extracted April 2023.

2022 Digital Case Study Client Focus Groups

IRCC conducted focus groups with clients who had accessed settlement services, to better understand what types of settlement services should be offered online, for which clients, and how to ensure that online services work well for clients.

This report uses focus group information about newcomer client views and experiences regarding online settlement services.

The focus groups were held May 2022.

2022 Digital Case Study Service Provider Focus Groups

IRCC conducted focus groups with staff from Service Provider Organizations (SPOs), to better understand in what contexts Settlement Program services should be offered online, for which clients, and how online services can be offered in a responsive, effective, and efficient way.

This report uses focus group information about views and experiences on the delivery of online settlement services.

The focus groups were held May 2022.

2022 RNIP Newcomer Survey

IRCC conducted an online survey of newcomers’ experiences participating in the Rural and Northern Immigration Pilot in October 2022, to better understand how the pilot is working, and what needs improvement.

This report uses survey information about RNIP newcomers’ use of services and supports, experiences of discrimination, sense of belonging, and desire to stay in the RNIP communities where they settled.

The data were extracted February 2023.

2022 RNIP Employer Survey

IRCC conducted an online survey of employers participating in the Rural and Northern Immigration Pilot in August 2022, to better understand how the pilot is working for employers, and what needs improvement.

This report uses survey information about the settlement supports provided by RNIP employers to support their candidates.

The data were extracted December 2022.

2022 RNIP Client Focus Groups

IRCC conducted focus groups with newcomers who had responded to the RNIP Newcomer Survey, and expressed a willingness to participate in focus group discussion, to contextualize survey responses and obtain further insights into RNIP newcomers’ experiences using settlement supports.

This report uses focus group information about experiences with volunteer matching, use of settlement services, and experiences within the community.

The focus groups were held February 2023.

2022 IRCC Survey: Newcomers’ experiences of discrimination in their city or town

IRCC conducted an online survey of newcomers in February 2022 to better understand experiences of discrimination in their city or town. The results of the survey will help IRCC inform its services and awareness campaigns going forward, and to meet the needs of newcomers to Canada.

This report uses survey information about experiences of discrimination in newcomers’ city or town, including the type of discrimination experienced.

The data were extracted April 2023.