Evaluation of the Opportunities Fund for Persons with Disabilities – Phase I

Key findings

Introduction

Employment and Social Development Canada's Skills and Employment Branch and Program Operations Branch thank the Evaluation Division for conducting Phase I of the evaluation of the Opportunities Fund for Persons with Disabilities. The contributions of key informants, who participated in interviews during the course of the evaluation, including external stakeholders and program participants, are also acknowledged.

The Opportunities Fund provides participants with the opportunity to acquire or develop employment readiness skills and/or to obtain employment experience or assistance in becoming self-employed. The Program's participant success rates have improved steadily from fiscal year 2012 to 2013 to fiscal year 2016 to 2017. Opportunities Fund projects also support activities that improve employers' awareness of the contributions that persons with disabilities bring to the workplace. Findings indicate that the Program continues to be relevant and meet the needs of persons with disabilities, including those who may face multiple barriers. The evaluation nonetheless points to improvements that can be made.

Employment and Social Development Canada is already addressing some of the findings identified in this Phase I report. For example, the Opportunities Fund's 2018 Call for Proposals placed a greater emphasis on working with employers to better integrate their needs into project design and prepare workplaces for the inclusion of persons with disabilities.

Recommendations and follow-up actions

The evaluation outlined four recommendations, all of which the Program area supports:

Recommendation 1:

Continue enhancing employers' awareness of the valuable contribution that persons with disabilities can make and of the various supports available to facilitate their hiring and workplace integration.

Management response:

The Program's 2018 Call for Proposals included additional emphasis on employer-focused activities. Program funding can now be used for an expanded range of activities such as coaching, training and human resources support to help employers develop inclusive workplaces. These activities can be combined with participant-focused activities or provided as stand-alone supports to employers with a demonstrated commitment to hiring persons with disabilities.

In addition, $18 million in new funding was announced in June 2018 to fund additional activities and enhance Opportunities Fund programming to improve employer knowledge, policies and practices to recruit, employ and retain persons with disabilities in the workplace. The development of in-house recruitment strategies in conjunction with the development of matching services will serve to close this gap.

Under a renewed performance measurement framework, employer engagement activities undertaken by service providers will be tracked. This is expected to improve the ability to measure employer engagement activities and outcomes.

Recommendation 2:

Explore the possibility of extending the availability of supports and follow-ups with participants.

Management response:

Employment and Social Development Canada agrees with this recommendation. Following the 2018 Call for Proposals, regional agreements will last for up to 3 years rather than for 1 year, which is expected to allow for increased follow-ups with participants. Employment and Social Development Canada will also examine the availability of supports in the context of other federal programs and initiatives.

Recommendation 3:

Further clarify the concept of social innovation and its application within the context of the Program and establish specific criteria for assessment and monitoring purposes.

Management response:

Employment and Social Development Canada agrees with this recommendation. While the 2018 Call for Proposals maintained the same criteria for social innovation (that is, pay for performance, social enterprise, and social partnership) as the 2014 Call for Proposals, the departmental definition^{Footnote 18} has evolved considerably over the last few years. Employment and Social Development Canada will explore the definition in the context of the Opportunities Fund to decide how it should be redefined and implemented for future funding investments.

Recommendation 4:

Better link the program's logic model to the reforms and new policy directions of the Program, and continue improving indicators used for performance measurement purposes.

Management response:

Employment and Social Development Canada supports this recommendation. Work has already been initiated to both revise the Opportunities Fund's logic model as well as strengthen the performance measurement strategy for both participant-based and employer-based projects.

In advance of the launch of the 2018 Call for Proposals, a multi-branch Performance Measurement Working Group convened to develop indicators to reflect the new policy directions, in particular for the new "Working with Employers" stream. Work will continue through this group to further revise the Opportunities Fund's logic model to reflect the 2014 reforms and the new policy directions introduced with the 2018 Call for Proposal.

As new projects begin, service providers will be required to report in greater detail on engagement activities with employers. Work is also underway to explore how to improve the measurement of "enhanced employability" and innovation projects, and how this aligns with the measures and performance indicators developed under the new Provincial and Territorial Workforce Development Agreements. 

1.1 Program background

Due to limitations inherent to their condition, and to barriers and obstacles they may encounter, many persons with disabilities have to drop out of the labour market intermittently or permanently, while others can only work on a part time basis or following a fixed schedule.^{Footnote 19} Such challenges not only limit their ability to obtain and maintain employment, but also to work enough hours to qualify for Employment Insurance benefits during periods of unemployment.

In 2012, around 10% of Canadians aged 15 to 64 (2,338,200 people) had a disability.^{Footnote 20} Approximately 45% of these individuals (1,057,100 people) were employed. Among those who were not employed, about 411,600 reported having the potential to work.^{Footnote 21} Of these potential workers, around 25% (103,300 people) had never worked or had not worked in the previous five years, and may have potentially benefited from the Program.^{Footnote 22}

Launched in 1997, the Opportunities Fund was designed to fill gaps in federal support for persons with disabilities who were not eligible for Employment Insurance or training under Employment Insurance Act Part II. To ensure that these individuals can access the assistance needed when trying to integrate or re-integrate into the labour market, the Program supports a range of projects. Most of these projects are designed and delivered by organizations known as Community Coordinators (also referred to as funding or contribution recipients). These organizations are external to the Department and act as intermediaries between participants and employers.

Projects designed and delivered by Community Coordinators must include Wage Subsidies and/or Self-Employment activities, which contribute to the mandatory employment experience component, as defined by the Program's Operational Directives. More specifically:^{Footnote 23}

• Wage Subsidy activities provide a financial incentive to encourage employers to hire persons with disabilities whom they would not normally hire, in the form of a share of the wages paid to these participants. Such activities must be used to place participants in positions that are likely to lead to permanent jobs with the host employers, and to assist participants whose likelihood of finding employment will be markedly improved through the creation of employer-employee relationships;
• Self-Employment activities provide technical and consultative expertise to participants who have developed a business concept or are interested in self-employment. These activities must be used to provide participants with a variety of services that will improve their chances of success in developing and implementing their business plans.^{Footnote 24}

In addition, projects can include interventions that fall into these 2 categories:^{Footnote 25}

1. Enhanced Employment Assistance Services support a mixture of pre-employment activities tailored to meet the needs of persons with disabilities and facilitate their integration into employment. They may include counseling and job placement services, access to labour market information, assistance in determining appropriate skills training, and more.
2. Skills for Employment interventions enable participants to undertake short-duration training and develop skills, from basic to advanced. Such skills must relate to a specific job opportunity in a stable or expanding sector, and where there is a reasonable expectation of employment.^{Footnote 26}

The Program also funds Employer Awareness activities, which are intended to raise the profile of persons with disabilities within the employer community and to highlight the capabilities and skills of workers with disabilities. These activities can be delivered through stand-alone agreements or with other program activities.

Projects can be regional or national in scope. Regional projects are managed by Service Canada regional offices, and national projects by Employment and Social Development Canada's National Grants and Contributions Delivery Centre. Although there are more regional projects than national projects, the latter receive, on average, a larger amount of funding.^{Footnote 27} This is mainly due to the fact that national projects are delivered in 3 or more provinces or territories across Canada and are usually of longer duration.^{Footnote 28}

1.2 Program reforms

A number of changes have been made to the design of the Program since the last evaluation was carried out in 2014. These changes, outlined in Budgets 2012 and 2013, include:

• providing more demand-driven training solutions to persons with disabilities
• making the Program more responsive to labour market needs
• tnvolving employers and community organizations in the design and delivery of projects
• connecting more Canadians with disabilities to work experience in small and medium-sized businesses
• further ensuring employers are aware of the contribution persons with disabilities can make

These reforms have resulted in the following operational changes,^{Footnote 29} beginning in fiscal year 2015 to 2016:

• mandatory employment experience is now included in all projects except for stand-alone Employer Awareness projects. Accordingly, Enhanced Employment Assistance Services and Skills for Employment interventions can no longer be delivered unless they are followed by, or combined with, Wage Subsidies or Self-Employment activities;
• through the reformed Opportunities Fund, participants can only acquire subsidized employment experience in positions that are part of the host employer's normal operations and are likely to lead to a permanent job.^{Footnote 30} In all cases, the host employer has to cover part of the wages paid to participants.^{Footnote 31} These changes are consistent with observations brought to the attention of the Standing Committee on Human Resources, Skills and Social Development and the Status of Persons with Disabilities.^{Footnote 32} The Program has also established that at least 50% of the duration of all project activities must relate to the employment experience component by 2017 to 2018^{Footnote 33};
• under the reformed Program, Skills for Employment interventions (that is, short-duration training programs) can only be supported if they help participants develop skills "related to a specific employment opportunity in a stable or expanding sector, where there is a reasonable expectation of employment."^{Footnote 34} Before April 2015, this criterion was only considered "optimal", not mandatory.^{Footnote 35} Further, while Skills for Employment interventions could last up to 52 weeks or 78 weeks under specific circumstances,^{Footnote 36} their maximum duration is now 6 months or 26 weeks. Consequently, participants have a shorter timeframe to prepare for the labour market through the Program.

The following enhancements were also made to the Program:^{Footnote 37}

• in order to help more youth with disabilities gain employment experience, the Program now allows employers to hire students with disabilities through Wage Subsidy agreements on a part-time basis (for up to 15 hours per week) during the school year.^{Footnote 38} The Program also established the objective that youth represent a quarter of all participants.^{Footnote 39} These enhancements are aligned with observations made by the 2013 Panel on Labour Market Opportunities for persons with disabilities, which underscored the need to provide opportunities for early work experience to youth with disabilities;^{Footnote 40}
• to strengthen performance measurement and assess longer-term outcomes, the Program now measures – for multi-year projects only – the number of clients employed or self-employed, and the number of clients returned to school, 6 months and 12 months post-intervention, within the timeframe of the agreement.^{Footnote 41} Prior to the reforms, these results were assessed, at the latest, 24 weeks following the end of the last intervention. A new participant information form is also used to collect additional information on employers hiring persons with disabilities through Wage Subsidy agreements;
• under the reformed Program, applicants who include an alternative social innovation model as a framework for project activities do not have to leverage funding from sources other than the Department for their project to be approved. In the context of the Opportunities Fund, "social innovation" is encouraged to maximize the impact of federal spending, and can take one of the following forms: social partnerships, pay-for-performance and social enterprise.^{Footnote 42}

Finally, all projects except stand-alone Employer Awareness projects now have to include at least 8 participants. Consequently, employers can only design and deliver stand-alone Wage Subsidy projects if they hire at least 8 eligible participants simultaneously. In comparison, prior to the reforms, Wage Subsidy stand-alone projects could include only one participant.

1.3 Budget initiatives

In Budget 2014, the following investments were announced:

• CommunityWorks (now Worktopia), a national network of vocational training programs for persons with autism spectrum disorders administered by the Sinneave Family Foundation and Autism Speaks Canada, will receive $11.4 million over 4 years (2014 to 2018).
• The Ready, Willing and Able initiative, a partnership between the Canadian Association for Community Living and the Canadian Autism Spectrum Disorders Alliance, will receive $15 million over 3 years (2014 to 2017). This project aims to help employers develop their capacity and confidence in hiring people with autism spectrum disorders or developmental disabilities.

Neither of these 2 projects, both specifically designed to respond to the needs of individuals with autism spectrum disorders, intellectual or developmental disabilities, makes use of targeted wage subsidies.

CommunityWorks/Worktopia enhances the employment readiness of individuals with autism spectrum disorders, through interventions such as pre-employment programming, one-to-one peer mentoring and the facilitation of the acquisition of volunteer experience. This project serves teens and young adults aged 15 to 29.

The Ready, Willing and Able initiative promotes understanding and awareness of the benefits of hiring persons with intellectual disabilities or autism spectrum disorders, among employers and the general public. This is done in part through activities similar to those delivered through Employer Awareness projects. The project also reaches out to specific employers, providing information and encouraging them to make concrete commitments to hiring persons with disabilities. As well, participants of the Ready, Willing and Able initiative may receive employment related supports, including: job coaching; short-term training, educational books, manuals and reference guides; job site assessments, worksite accessibility accommodations and alterations; adaptive technology; attendant care and sign language interpretation; and work related transportation.

Both budget initiatives, sometimes referred to as pilot projects, were implemented in parallel to the Program's reforms. The evaluation examined the extent to which these projects were implemented as designed. Related findings are discussed in Section 3.4.

3.1 Program relevance

The Program responds to the needs of persons with disabilities, and continues to fill a gap in employment services for those who are not eligible for Employment Insurance benefits or training support under Employment Insurance Act Part II (or cannot access such services). Evidence also indicates that there is an ongoing need to enhance employer awareness of the potential contribution that persons with disabilities can make, and confirms that financial incentives may be needed to encourage employers to hire more persons with disabilities. 

The objectives of the Opportunities Fund for Persons with Disabilities continue to be relevant and reflect the needs of persons with disabilities

Evidence from the review of project documents and publications on the labour market outcomes for persons with disabilities confirms the need to address barriers and challenges that prevent persons with disabilities from obtaining and keeping employment or self-employment.

• Important gaps can be observed between employment outcomes for persons with disabilities and outcomes for persons without disabilities: For example, according to the 2011 National Household Survey and the 2012 Canadian Survey on Disability,^{Footnote 43} in 2011, 49% of Canadians with disabilities aged 25 to 64 were employed, compared with 79% of their counterparts without a disability. The unemployment rate was 11% and the participation rate was 55% for Canadians with disabilities aged 25 to 64, compared to 6% and 84% for their counterparts without a disability.^{Footnote 44}
• Unequal labour market outcomes for persons with disabilities represent a large opportunity cost that is likely to increase in the context of an aging population: According to the latest estimates available (at the time this report was prepared), about 411,600 Canadians with disabilities aged 15 to 64 are not employed, despite having the potential to work.^{Footnote 45} This issue is likely to become increasingly concerning with the aging of the population, a trend that already emphasizes the need to fully utilize the country's talent pool.^{Footnote 46} Further, the portion of the labour force represented by older workers is expected to increase. Since the prevalence of disability increases with age (see Figure 1 below), the proportion of Canadians in the labour force with a disability will also tend to increase. As well, disability is likely to affect an increasing number of women.

Text description of Figure 1

Age group	Male	Female
15 to 24	4%	4%
25 to 44	6%	7%
45 to 64	15%	17%

Source: 2012 Canadian Survey on Disability (most recent data available when this report was prepared)

• While youth are less likely to have a disability, those who have one are more likely to see their condition negatively impact their employment outcomes. The prevalence of disability is lower among youth, but those who have one are more likely to have a mental or psychological disability, which makes them more likely to face disability related employment barriers or challenges. In fact, according to the 2011 National Household Survey and the 2012 Canadian Survey on Disability, persons with mental or psychological disabilities have a significantly lower probability of being employed than individuals who have other types of disabilities, even when taking severity and other factors into account.^{Footnote 47}
• The majority of Canadians with severe or very severe disabilities are working-age adults. According to the 2012 Canadian Survey on Disability, nearly 16% of seniors aged 65 or more had a severe or very severe disability, representing 685,400 individuals. In comparison, around 5% of Canadians aged 15 to 64 had a severe or very severe disability, which represented 1,147,000 individuals, including 814,000 individuals aged 45 to 64. As shown in the graph below, the severity of disability has a significant impact on the likelihood of being employed.^{Footnote 48}

Text description of Figure 2

Gender	Mild	Moderate	Severe	Very Severe
Female	65%	53%	39%	27%
Male	65%	53%	45%	26%

Source: 2012 Canadian Survey on Disability (most recent data available when this report was prepared)

Findings indicate that there is a continued need to enhance employer awareness of the contributions that persons with disabilities can make

Evidence from key informant interviews and project documents underscored the continued need to educate employers to treat persons with disabilities fairly, and not to underestimate their potential and ability to perform in the workplace.

In addition, evidence from the document review showed that Program activities may help employers diversify their workforce and/or address specific labour shortages. For example, a number of projects designed and delivered by industry associations were identified in the document review. These projects were aimed at responding to the labour needs of these associations' business partners through a combination of Wage Subsidy and Employer Awareness activities, or through stand-alone Employer Awareness projects.

Financial incentives may be required for employers to hire participants^{Footnote 50}

A number of employers interviewed for the evaluation indicated that the persons with disabilities hired through Wage Subsidy agreements surpassed expectations and were hired on a regular basis when wage subsidies ended. These employers stated that they likely would not have hired the workers in question if they had not been first exposed to the Program.

Additional evidence from the document review and from key informant interviews indicates that employers expect to receive wage subsidies from agencies working with persons with disabilities. Key interviewees also noted that employers tend to be more responsive to Employer Awareness activities when they are with a Wage Subsidy. It should be noted, however, that evidence also shows that financial incentives (that is, wage subsidies and support for workplace accommodations) do not always guarantee that participants will secure employment through the Program, or that those who succeed in obtaining employment will be hired on a regular basis by their host employer.

In this regard, it was also reported that employers were sometimes concerned by the amount of paperwork that would be required, or by the fact that receiving wage subsidies would not be authorized by their employees' union, head offices and/or human resources policies. During key informant interviews, it was also noted that employers may prefer to hire participants directly, without requesting a subsidy, as they saw it as an opportunity to contribute to a cause or to their community. It was not possible to determine the extent to which this happened, due to data limitations.

The Opportunities Fund continues to fill gaps left by other employment programs for persons with disabilities

An array of provincial/territorial and federal programs exist to enhance the labour market participation of persons with disabilities. Some of these programs can overlap, complement or substitute each other, but gaps persist.^{Footnote 51}

This is especially true for job seekers with disabilities who do not qualify for Employment Insurance benefits or other supports funded through Employment Insurance Act Part II, and/or who are located in rural or remote areas where the supports and services delivered through the Opportunities Fund may be the only ones available for persons with disabilities. It should be noted that the Opportunities Fund is the only employment-focused program for persons with disabilities at the federal level. Therefore, it plays a key role in supporting initiatives undertaken in this area at the national level.

Key informant interviewees also noted that persons with disabilities are more likely to be employed on a part-time basis, which makes them potentially vulnerable when downsizing occurs in an organization. This can negatively affect the ability of these individuals to qualify for Employment Insurance benefits. While the number of hours worked by participants employed is not captured by the Program, the survey of participants in single year projects done by the evaluation revealed that most respondents (64%) who were working at the time of the survey were working on a part-time basis (less than 30 hours in a typical week).^{Footnote 52}

Finally, as pointed out by a few key informants, unlike for some other employment programs for persons with disabilities, a formal medical assessment is not needed to be eligible for Opportunities Fund services and supports.^{Footnote 53} This allows potential participants to avoid a process that can be lengthy, burdensome, and even costly.

The Program continues to align with priorities, roles and responsibilities of Employment and Social Development Canada and the federal government

The Program aligns with the Government of Canada's policy directions and priorities. This is reflected in the continued funding of the Program, the recent tabling of new federal accessibility legislation^{Footnote 54} and the Government's broader engagement towards the elimination of barriers to employment among underrepresented groups on the labour market, including persons with disabilities.

More specifically, the Program aligns with the Government of Canada's efforts to “eliminate systemic barriers and deliver equality of opportunity to all Canadians living with disabilities,”^{Footnote 55} and to “improve work opportunities and employment outcomes for persons with disabilities,”^{Footnote 56} as outlined in Budgets 2016 and 2017.

The objectives of the Opportunities Fund also reflect Employment and Social Development Canada's mission of building “a stronger and more inclusive Canada, to support Canadians in helping them live productive and rewarding lives and improving Canadians' quality of life.”^{Footnote 57}

The Program also aligns with a number of the Department's responsibilities, such as:

• developing policies that ensure that all Canadians can use their talents, skills and resources to participate in learning, work and their community;
• fostering inclusive growth by providing opportunity and assistance to Canadians with distinct needs, such as Indigenous people, people with disabilities, homeless people and recent immigrants.^{Footnote 58}

Finally, Employment and Social Development Canada is well positioned to deliver the Program given its leading role in delivering programs and services that affect Canadians throughout their lives. These include supports to unemployed workers who are not eligible for Employment Insurance benefits and/or who are traditionally underrepresented in the labour force.  

There is potential to increase the reach of the Program

The Opportunities Fund served approximately 4,300 clients per year, on average, from fiscal year 2012 to 2013 to fiscal year 2016 to 2017. In comparison, as of 2012 (most recent estimates available at the time this report was prepared), approximately 103,300 Canadians aged 15 to 64 had a disability and, despite having the potential to work, had never worked or had not worked in the previous five years.^{Footnote 59} Potentially, they could have benefitted from the Program.^{Footnote 60}

It was also noted during key informant interviews that there could be better awareness and understanding, among employers, of the Program in general and how it works, and more broadly about the resources available to help integrate persons with disabilities into the workplace.^{Footnote 61} This indicates that there is opportunity for greater awareness and visibility of the Program among service providers and potential clients. In this regard, it should be noted that both the 2008 Summative Evaluation and the 2014 Summative Evaluation of the Opportunities Fund concluded that there was no systematic approach to promote the Program to clients and service providers.^{Footnote 62}

Finally, persons with disabilities may decide to remain outside the labour market despite being aware of the availability of employment services and supports such as those delivered through the Opportunities Fund. This may be due to various factors that were identified in existing studies, and were confirmed by the current evaluation. These include:

• the expectation that employment income will not be high enough to compensate for the loss incurred in terms of social assistance benefits, including related supports such as a drug plan or assisted housing;
• the inability to commute to and from work;
• the lack of access to affordable childcare;
• family responsibilities;
• the perception that jobs are not available.^{Footnote 63}

3.2 Program performance

Evidence indicates that progress is being made towards the achievement of the Program's direct outcomes. From fiscal year 2012 to 2013 to fiscal year 2016 to 2017, the percentage of participants who were employed, self-employed or returned to school (up to 12 months after their last intervention) increased progressively from 42% to 55%. Ongoing challenges or barriers faced by persons with disabilities continue to prevent approximately half of the participants from becoming employed or self-employed through the Opportunities Fund. Finally, limited information is collected on employers' experience and satisfaction with the Program, and the progress made through Employer Awareness activities.

Various benefits for persons with disabilities can be derived from participation in the Program

Evaluation evidence confirms that, through the Program, progress is being made towards enabling participants to access labour market information, to develop skills needed to search for and find employment, to acquire employment experience and workplace accommodations or to take part in Self-Employment activities. Evidence also indicates that the Program can have the following benefits for participants and/or their family:

• the development of social skills, workplace communication and interview skills, organizational, administrative and problem solving skills, as well as time, anxiety and stress management skills;
• the improvement of overall well-being, attitudes, behaviors and motivation;
• an increase of self-esteem, pride, self-confidence and sense of self-worth;
• an increase of self-awareness and understanding of personal strengths, weaknesses and professional interests;
• an increase of networking opportunities;
• a decrease of social isolation and feelings of stigmatization;
• the acquisition or development of technical skills in areas like customer service, general maintenance and logistics, landscaping and bookkeeping.

The Program's overall success rate increased progressively over the 5 previous years

Table 1 shows that the proportion of participants served who were either employed, self-employed or returned to school after participating in the Program, improved progressively between 2012 and 2017, reaching 55% in fiscal year 2016 to 2017, up from 42% in fiscal year 2012 to 2013.^{Footnote 64} On average, during the same period, the Program served approximately 4,316 persons with disabilities each year. Of these, around 41% (1,791) were employed or self-employed and 6% (253) had returned to school.

These improvements could be due to numerous factors that can influence a job seeker's likelihood of finding employment, such as the evolution of economic conditions and of the types of skills that are in demand on the labour market.

As well, changes in reporting requirements have occurred along with the reform of the Program.^{Footnote 65} Prior to fiscal year 2014 to 2015, participants' results were assessed at the latest 24 weeks following the end of their last intervention. Since reforms were implemented, the Program updates these results:

• at 6 months and 12 months post-intervention, within the timeframe of the agreement, for multi-year projects; and
• for single year projects, up to 12 weeks after participants' last intervention, and within 60 days of the project end date.

A survey conducted as part of this evaluation to collect more recent results for participants in single year projects (at 6 and 12 months post intervention) showed that approximately 42% were employed or self-employed following participation. This figure is very close to the one indicated in Table 1 (for fiscal year 2015 to 2016). Survey results indicate that female participants were more likely to work at the time of the survey (47%) compared to male participants (38%).^{Footnote 66}

Source of data: Performance Measurement, Analysis and Reporting, Common System for Grants and Contributions - Client Module and Participant Information Module, June 22, 2017.

Note: These results do not include those of participants of the CommunityWorks/Worktopia and Ready, Willing and Able budget initiatives.

In addition, survey results suggest that the percentage of participants returning to school increases with time, subsequent to their participation in the Program, and that Table 1 may underestimate this result.^{Footnote 67} When asked to indicate the level of education they were enrolled in (information that is not collected by the Program through participant information forms), survey respondents who had returned to school provided the responses presented in Table 2 below.

Source of data: TNS Canadian Facts Ltd. (2017), Survey of Participants for the Evaluation of the Opportunities Fund for Persons with Disabilities – Phase I, prepared for Employment and Social Development Canada.

Based on the figures in this table, males were more likely than females to be enrolled at the high school or equivalent level (46% versus 16%, respectively). Females were more likely than males to be enrolled in a community college (47% versus 23%, respectively). It should be noted that only 54 respondents reported being enrolled in school, which constitutes a small sample. Therefore, this comparative analysis should be interpreted with caution.

The Program's target population is mainly composed of individuals who are initially far removed from the labour market and can face a variety of employment related challenges

Evidence indicates that participants' journey through the Program does not follow a standard path. This is consistent with the fact that persons with disabilities form a particularly diverse segment of the population. Some may have multiple disabilities that greatly limit all their daily activities, while others have only one disability that has less of an impact on their activities. Persons with disabilities also have various levels of educational attainments, which can significantly affect employment outcomes.^{Footnote 68}

Further, the Program fills gaps in areas where the services offered to persons with disabilities are limited or nonexistent, and is delivered in different regions across Canada, where economic and labour market conditions can vary considerably. Finally, the Opportunities Fund may act as a last resort solution for individuals with disabilities who are not eligible for Employment Insurance benefits or employment programs delivered through Employment Insurance Act Part II.^{Footnote 69} This means that most Program participants may have never worked in Canada, may have not worked sufficient hours to qualify for the Employment Insurance program, or may have withdrawn from the labour force for five years or more.

Therefore, the Program's clientele conceivably includes individuals that have greater challenges to overcome and greater labour market vulnerabilities, as opposed to both persons without disabilities and persons with disabilities who have stronger ties to the labour market. If needed, the latter may qualify for Employment Insurance benefits and be eligible for other employment supports and services delivered at the federal, provincial or territorial level.

Many participants are successful while others may continue to face barriers or challenges in trying to secure or maintain employment 

About half of participants have successful outcomes, which include finding employment or pursuing further education after taking part in the Program. At the same time, about half of participants do not reach that objective.^{Footnote 70} Such outcomes may be due to different factors. The most commonly identified by the evaluation were physical or mental health related issues that emerged during the course of the Program, such as injuries, illnesses or relapses requiring short-term hospitalization, social anxiety disorders, episodic depression and addiction issue. Lack of motivation or commitment among participants was also mentioned on one or more occasions in project documents.^{Footnote 71}

Further, the document review revealed that participants may have a limited range of jobs for which they are qualified – a situation that can be attributable to adverse economic conditions at the local or regional level, and also to participants' limited formal education, abilities, skills, professional interests or employment history. Participants may not be able to perform functions that are physically demanding and this may further limit their employment opportunities.^{Footnote 72}

Finally, participants may not obtain or maintain employment through the Program for reasons similar to those that can initially discourage persons with disabilities who are not employed from accessing employment services and supports. These reasons include lack of affordable childcare, difficulties commuting to and from work (adapted transportation is unavailable or too costly), and the inability to obtain an employment income that exceeds available social assistance benefits.^{Footnote 73}

Overall, these findings are consistent with evidence from the 2012 Canadian Survey on Disability, from previous evaluations of the Program and from the evaluation of the Canadian Disability Savings Program.^{Footnote 74} Findings were also confirmed by results of the survey conducted in the context of this evaluation. Approximately 29% of survey respondents had never worked after participating in the Program.^{Footnote 75} When asked to identify the reasons why they had not been working:

• 24% cited illness or disability
• 13% noted that they believed no work was available
• 7% mentioned having other personal or family responsibilities^{Footnote 76}
• 7% indicated that it was difficult to find a job or that they could not find a job
• 6 % mentioned that it was a personal choice or that they lacked motivation

Participants may secure jobs with employers they were not initially involved with through Wage Subsidy activities

Among respondents who had taken part in Wage Subsidy activities, about 39% were working at the time of the survey conducted for this evaluation. Of these respondents, approximately 36% (or 14% of all respondents who took part in Wage Subsidy activities) continued to work for the same employer, 6 to 18 months after the subsidy ended. The other 64% (or 25% of all respondents who took part in Wage Subsidy activities), were working with other employers than those who hired them through Wage Subsidy agreements.^{Footnote 77} It should be noted that data collected by the Program through participant information forms does not indicate whether participants employed are working with the employer they were involved with through Wage Subsidy activities or with another employer. 

During key informant interviews, it was noted that participants of Wage Subsidy activities may not always want to occupy the jobs initially obtained through the Program, if they find better opportunities. Similar input was obtained directly from employers interviewed as key informants, some of which also reported that participants had to be dismissed for aggressive behavior or stopped showing up at work and could not be reached, while others simply decided to leave as they felt they could not work at the level required, or could not handle the stress of the workplace and the resulting anxiety.^{Footnote 78}

Additional evidence is required to fully assess the Program's effect on the medium- to long- term labour market outcomes of participants

The majority of key informants who addressed this issue indicated that the Program had a positive effect on the employment opportunities that are available to persons with disabilities.^{Footnote 79} However, the performance information and evidence gathered through the document review are not robust enough to indicate that Program interventions resulted in increased employment opportunities for persons with disabilities or led to more persons with disabilities to be integrated into the labour market.^{Footnote 80}

Additional evidence is also needed to assess the extent to which changes resulting from the reform of the Program^{Footnote 81} have contributed to improve participants' employment outcomes on the medium and long term. Such evidence will be developed and examined during Phase II.^{Footnote 82}

Further, it should be noted that survey respondents who worked for pay at some point following participation provided mixed responses when asked how likely they would have been to obtain employment without participating in the Program. For instance, 28% of these respondents suggested that they would have likely found employment without participation, while about a quarter (22%) noted that they would have been unlikely to find work without participating in the Program.^{Footnote 83}

These findings indicate that various factors other than Program interventions can influence participants' chances of success in the labour market. Phase II of the evaluation will examine this aspect more thoroughly.

Important investments are being made to enhance employers' awareness of the valuable contribution that can be made by persons with disabilities

Three stand-alone national Employer Awareness projects were funded through the Program for the fiscal year 2015 to 2016. These projects received combined contributions of approximately $3 million, part of which was spent over several years.^{Footnote 84} In addition, Employer Awareness activities are delivered through other projects, generally in combination with Wage Subsidy activities, and through the Ready, Willing and Able initiative.^{Footnote 85}

Data collected with regards to Employer Awareness activities is limited and generally comprised of indicators such as the number of activities delivered and/or the number of employers reached through the process. While relevant, such information is not sufficient to assess the impacts of these activities, not only in terms of awareness and understanding, but also with regards to employers' human resources policies and practices. For example, the impact of employer awareness activities could be measured by increases in the number of persons with disabilities who were hired by employers after receiving awareness material or being reached through other channels.

Further, when asked how their experience with the Program affected their perception of persons with disabilities, employers interviewed as key informants provided a range of responses. Some found persons with disabilities to be as competent as those without disabilities,^{Footnote 86} while other employers indicated that they needed more experience with the Program to make a determination. Finally, some employers reported that they would consider renewing their involvement in the Program, but preferably with participants that do not have specific types of disabilities.

A number of measures may help to foster better results for participants and improve the Program's performance

Approximately 29% of survey respondents^{Footnote 87} did not find an employment after participating in the Program and were asked what additional assistance, accommodations or supports would have helped them to find employment. Of these respondents:^{Footnote 88}

• close to half (45%) identified the need to develop a system to connect participants with employers willing to hire persons with disabilities, through job postings for persons with disabilities and introductions to employers. This echoes the findings of Employment and Social Development Canada's latest Evaluation of Learning and Labour Market Information;^{Footnote 89}
• approximately a third (35%) identified a need for more training and longer interventions, which was also mentioned by a few key informants. On this matter, a few key informants also specified that funding agreements should be longer (for example, 3 years) and the processing period shorter;
• about a fifth (18%) identified a need for help and support with job search (for example, from an advisor or employment recruiter). While these supports can be delivered through Enhanced Employment Assistance Services, they may not always be accessible to participants, considering that funding recipients have limited resources;
• approximately 12% expressed that more funding is needed, especially for workplace accommodations, which is consistent with input from a few key informants who noted that, funding recipients sometimes have to partner with other organizations who have access to additional resources or funding in order to ensure that the needs of each participant are fully met.

The document review, a line of evidence used in this evaluation, could not determine the extent to which support for workplace accommodations were provided to Program participants as needed. Input from key informant interviews revealed that the availability of such supports may not be well known among employers.

It was also mentioned during interviews with Service Canada representatives that coaching and other forms of psychosocial supports could help participants to stay employed or remain motivated when they do not succeed in finding employment. In particular, this could be useful for those who have mental or psychological health related disabilities. It should be noted that it was not always clear among service representatives whether such supports can be provided via the Program.

Finally, a few informants among project representatives, Service Canada regional staff and Employment and Social Development Canada National Head Quarter staff, identified options to explore. These would help to strengthen the efforts made in the area of employer awareness and/or to make securing job placements easier and quicker for participants. Their suggestions included:

• a broader national employer awareness strategy, inclusive of all disability types and delivered by the Federal Government in consultation with key stakeholders
• workshops and training on disability for specific types of employers or job sites

3.3 Implementation of reforms

More participants are acquiring employment experience in small and medium-sized businesses through the reformed Program. Also, the proportion of youth included among participants is increasing. The evaluation found no evidence that more demand-driven training activities are being provided nor that the reformed Program is more responsive to labour market needs. Further, findings indicate that employers are generally not involved in the design of projects; they mainly contribute to their delivery by covering part of participants' wages and providing on-the-job training and supervision. More evidence is needed to determine to what extent the use of social innovation has contributed to the optimization of federal spending dedicated to the Program.

Progress is being made towards all participants acquiring employment experience or taking part in Self-Employment activities through the Program

As shown in Table 3, Wage Subsidy activities represented a higher proportion of all activities undertaken in the fiscal year 2016 to 2017 (37%), than in the fiscal year 2015 to 2016, during which they accounted for 25 % of all interventions undertaken. When adding Self-Employment activities, which can also account for the mandatory employment experience component of participants' action plans, these percentages reach 29% for the fiscal year 2015 to 2016 and 41% for the fiscal year 2016 to 2017.

Following the implementation of Program reforms, the number of participants receiving Enhanced Employment Assistance Services remained higher than the total number of Wage Subsidy and Self-Employment activities undertaken.^{Footnote 90} Given data limitations, it was not possible to determine if specific variables explain why participants may receive Enhanced Employment Assistance Services without subsequently acquiring employment experience through Wage Subsidy activities or undertaking Self-Employment activities.^{Footnote 91}

Finally, information from key informant interviews with project representatives and Service Canada regional staff indicate that participants may find employment without requiring the help of a wage subsidy. In this type of situation, they would still be included among the group of participants considered to have succeeded in the Program, both as participants with enhanced employability and as participants employed or self-employed.^{Footnote 92} The evaluation could not determine how often this occurred, due to limitations in data available.^{Footnote 93}

Source of data: Performance Measurement, Analysis and Reporting, June 22, 2017.

Note: carry-over interventions are not included (new starts only).

Note: These results do not include those of participants of the CommunityWorks/Worktopia and Ready, Willing and Able budget initiatives.

It should be noted that, before the Program was reformed, participants could also acquire subsidized employment experience through interventions referred to as Work Experience activities. Unlike positions occupied by participants of Wage Subsidy activities, those occupied by participants of Work Experience activities had to be incremental to, and not part of, the employer's normal operations. Also, there was not the same expectation that the employer would keep the participant as an employee once the subsidy would have ended.

Finally, while participants were paid through both Wage Subsidy and Work Experience activities, wages paid to participants of Work Experience activities were entirely subsidized by Employment and Social Development Canada, while those of participants of Wage Subsidy activities are subsidized only in part. Therefore, all employers hiring individuals with disabilities through the Program now have to contribute financially to the wages paid to these individuals.

More participants are gaining employment experience in small and medium-sized enterprises

The evaluation could not determine the exact proportion of Wage Subsidy activities that took place in small and medium-sized businesses, due to data limitations. However, the vast majority of key informants who addressed the question indicated that employers from this category account for the largest part of employers across Canada. Therefore, it can be inferred that more persons with disabilities are acquiring work experience in small and medium-sized enterprises as the number of participants of Wage Subsidy activities increases.

The proportion of youth among Program participants increased after the implementation of reforms

Program administrative data reviewed indicates that the share of youth aged 15 to 30 among all participants is significantly higher than 25%, and has increased progressively between the fiscal years 2014 to 2015 and 2016 to 2017, starting at 43% and reaching approximately 52%, respectively.^{Footnote 96}

According to the regional Service Canada key informants who addressed the issue of greater youth participation, this increase mainly stems from the fact that the goal to include more youth among cohorts of participants has been communicated to funding recipients.

Finally, part of this increase can be attributed to the fact that the participants in the CommunityWorks/Worktopia project,^{Footnote 97} one of the two budget initiatives implemented in parallel to the Program's reform, were all youth aged 15 to 30.^{Footnote 98}

Findings do not indicate that the reforms implemented have facilitated the provision of demand-driven training solutions 

Under the reformed Program, Skills for Employment interventions (that is, short-duration training programs) can only last for a maximum of 6 months or 26 weeks. The majority of the regional Service Canada key informants who were interviewed stated that this period is not long enough to allow for participants to consolidate the skills they acquired or to develop specialized skills that are in demand on the labour market. The reduction of the maximum duration of Skills for Employment interventions was implemented within the context of the reforms. The underlying rationale behind such change, and its alignment with the objective to provide more demand-driven training solutions to participants, could not be formally identified.

Overall, within the sample used for the purposes of the document review, a few projects were designed to provide participants with training that aligns with specific needs of the labour market, and one of these projects made use of Skills for Employment interventions. This particular project enabled a group of unemployed youth with disabilities living in First Nations communities to complete the Heavy Equipment Operator training program. These individuals were subsequently hired under a Wage Subsidy agreement by their band councils to perform duties that would have otherwise been contracted out to individuals living outside their communities.

In the other projects examined, the content of activities delivered through Enhanced Employment Assistance Services was adapted to prepare participants to occupy jobs in specific sectors where there was demand for labour at the regional level (for example, wood manufacturing, gardening and catering industries); however, this could also be done before the Program was reformed, and the evaluation found no evidence that this happened more commonly under the reformed Opportunities Fund. Further, project documents indicate that not all of the participants obtained jobs in one of these sectors, for different reasons including skills mismatches and a lack of interest on the part of some participants.

When asked where their project has helped to respond to labour demand in specific industries, the majority of project representatives interviewed mentioned the retail and hospitality sectors.^{Footnote 99} While a few key informants from this group noted that steps have been undertaken to open new opportunities for participants in the IT, banking, green energy and energy efficiency sectors, none of them confirmed whether these initiatives have led to concrete employment opportunities for participants.

Evidence indicates that some employers are involved in the design of projects and that this aspect of the Program could be strengthened

Among the documents reviewed, the evaluation identified a few projects that were designed and delivered by organizations representing employers (industry associations) and with the objective of addressing specific shortages of workers and/or to help employers from a specific sector diversify their workforces through targeted stand-alone Employer Awareness projects (as well as through a combination of Employer Awareness and Wage Subsidy activities). During one of these projects, the funding recipient also tested a “disabilities in the workplace self-assessment tool” designed for a specific occupation, and found three employers to ‘champion' this initiative.”^{Footnote 100}

The other examples of employer involvement in specific project activities or initiatives identified by the evaluation include:

• the creation of a network of "ambassadors" representing locally based small and medium-sized enterprises that have demonstrated an interest in hiring qualified persons with disabilities
• an "employer recognition" campaign designed to demonstrate the value of the employment relationships established through the Program
• discussions on employers' expectations with a panel of local employers that took place in the context of enhanced employment assistance services^{Footnote 101}

Under the reformed Program, the vast majority of projects that comprise Wage Subsidy activities are not designed and delivered by employers looking to address specific needs in labour, but by Community Coordinators, who act as intermediaries between several participants and employers. All projects except stand-alone Employer Awareness projects now have to include at least eight participants,^{Footnote 102} a criterion that was described as being very difficult or even impossible to meet for employers, especially for small and medium enterprises.^{Footnote 103}

At the same time, it was noted that, from a program delivery perspective, allowing employers to design and deliver stand-alone Wage Subsidy projects with less than eight participants would not be a cost-effective option, considering the administrative burden and overhead costs incurred for applicants and Employment and Social Development Canada, whose regional service-delivery staff has diminished over the years.  

Partnerships between funding recipients and employers are usually created or formalized after projects have been approved for funding

According to the Program's operational directives, applicants should provide supporting documents from project partners with their application for funding, such as a list of potential employers for wage subsidy participants. The majority of applications for funding reviewed in the context of this evaluation did not identify actual or potential employer or business partners, and none of the employers interviewed as key informants reported having been involved in the design of a project.

These findings are consistent with Employment and Social Development Canada and Service Canada key informants who indicated that, although applicants may have informal discussions with potential employers prior to the submission of their application for funding, these first steps do not necessarily lead to the creation of formal partnerships.

Evaluation findings indicate that employment relationships created through OF Wage Subsidy activities are generally fostered by initiatives of funding recipients and by job seeking efforts made by participants, in collaboration with project staff, when funding for projects has already been approved by the Department. In turn, employers' contributions towards the delivery of projects usually consist of covering part of participants' wages and providing them with on-the-job training and supervision.^{Footnote 104}

Efforts continue to be made to ensure the needs of employers are met during project delivery

Interviews conducted with project representatives and findings from the review of project documents indicate that, during the delivery of projects, funding recipients may conduct follow-up activities such as worksite visits to ensure that participants receive the support and training needed, to discuss issues that may arise and to provide additional job coaching if needed. This was confirmed by the majority of employers interviewed. The latter indicated that funding recipients were helpful during project delivery as they helped participants acquire/develop skills for the workplace and/or have made efforts to ensure a good fit between participants and the jobs in which they were placed.

One employer interviewed stressed that the selection of participants for Wage Subsidy activities had been entirely made by the recipient organization, while another reported having the opportunity to meet and decide (or not) to hire potential participants (during an "open house" event organized by a funding recipient).^{Footnote 105}

Most employers interviewed for this evaluation mentioned that participants were adequately prepared before starting work. They indicated that the preparation level could vary among participants and/or that “a bit more work” could be done to ensure participants knew what employers are looking for and what is expected from them in the workplace.^{Footnote 106}

Overall, only one of the employers interviewed reported that the individuals hired through Wage Subsidy activities were unable to perform very basic tasks. This key informant also noted that if he/she would have been consulted beforehand, these participants' needs in terms of pre-employment education and training would have been identified, which in turn would have increased their chances of staying employed after participation.^{Footnote 107} 

The majority of regional Service Canada key informants as well as a few Employment and Social Development Canada key informants mentioned that the Program was more flexible before the 2015 reforms and used to put a greater focus on developing participants' pre-employment, transferable and “soft” skills.^{Footnote 108} A few of these informants specified that the interventions falling under the “Work Experience” category^{Footnote 109} were a good way to prepare certain participants to integrate the labour market, notably as they could help better assess their needs in terms of skills development and workplace accommodations, which can be difficult to do in the limited timeframe allowed under the reformed Program.

Phase II of the evaluation will help determine if, and to what extent, participation in Wage Subsidy activities leads to better labour market outcomes in comparison with participation in Work Experience activities, which are no longer eligible for funding under the Opportunities Fund. On this matter, a key informant from Employment and Social Development Canada also noted that, while Work Experience activities could lead to employers “taking advantage” of persons with disabilities, the new focus on Wage Subsidy interventions may encourage funding recipients to select participants that were initially more “job ready.”^{Footnote 110}

More evidence is needed to determine the extent to which social innovation has helped maximizing the impact of program spending

The Opportunities Fund encourages the use of social innovation (that is, social enterprise, social partnerships or pay-for-performance) in order to maximize the impact of federal spending.^{Footnote 111}

The document review identified a few projects where the recipient organization used a social innovation model. In the majority of cases, it consisted of a social enterprise that obtains revenues from the provision of services that are related to the workplace integration of persons with disabilities. Such services are provided on a fee-for-service basis to employers. They may consist of workplace accommodations assessments and support with the development and implementation of inclusive human resources policies and practices.

In another project, the recipient organization made use of a pay-for-performance model where a fraction of departmental funding is conditional on the achievement of predetermined results. More specifically, the funding agreement specifies that an amount of $108,000, which represents about 5% of the total project costs, will be paid to the funding recipient at a rate of $1,500 per successfully self-employed participant.

The evaluation could not determine the extent to which these projects' social innovation component has helped maximizing the impact of Opportunities Fund spending. These components were either already in place when projects have started or were at too early a stage of implementation and had not produced concrete results when the evaluation was conducted.^{Footnote 112} It should also be noted that the value of the resources generated by recipient organizations through their social enterprise could not be determined, due to limitations in available data.

Overall, these findings were confirmed by the few regional Service Canada and Employment and Social Development Canada key informants who discussed the subject. Some of these informants further noted that the concept of social innovation was not defined in a clear and concise manner in the Program's call for proposals.^{Footnote 113} This resulted in administrative challenges both at the project assessment and monitoring stages, and required the Program to develop a rating scale for social innovation.^{Footnote 114}

Finally, one of these informants also mentioned that the use of social innovation within the context of the Program was not promoted and often happened incidentally.

3.4 The Ready, Willing and Able and CommunityWorks/Worktopia initiatives

The Ready, Willing and Able initiative faced issues of collaboration between partners and difficulties with the recruitment of participants. In the case of the CommunityWorks/Worktopia project, many initial participants did not complete the project. In order to achieve their objectives, changes were made in the delivery approach of both projects.

When the evaluation work was conducted, only preliminary results were available for the Ready, Willing and Able and CommunityWorks/Worktopia initiatives. Challenges faced in the implementation of these 2 projects were identified through the review of the project documents and interviews with key informants. As well, 2 evaluation reports (a formative evaluation report and a progress report) of the Ready, Willing and Able initiative were prepared by the University of British Columbia, on behalf of the recipient organization. Some of the findings discussed in this section were drawn from these reports.

Ready, Willing and Able

Ready, Willing and Able is a national strategy delivered by the Canadian Association for Community Living, in partnership with the Canadian Autism Spectrum Disorders Alliance. This initiative aims to increase the labour force participation of people with intellectual disabilities or autism spectrum disorders. To support this goal, it generates increased demand to hire people with developmental disabilities, and works with employment service providers to ensure that effective job matching tools are in place.

More specifically, this project delivers interventions similar to Employer Awareness activities. The project also reaches out to specific employers, providing information and encouraging them to make concrete commitments to hiring persons with disabilities. As well, a range of employment supports can be provided to participants; examples include job coaching, short-term training, job site assessments, worksite accessibility accommodations and alterations, adaptive technology, tools and equipment, attendant care and sign language interpretation.

Unlike other Opportunities Fund projects, detailed information on each participant of the Ready, Willing and Able initiative is not collected by the Program. This is normally done through participant information forms, which are also used to verify participants' eligibility for the Program. Instead, results for participants of the Ready, Willing and Able initiative are self-declared by the recipient organization, in aggregate form (for example, total number of participants served who found employment), and are not validated by the Program.

The Ready, Willing and Able initiative had to address supply-side issues

There were situations where jobs were created in collaboration with employers, yet could not be filled due to supply-side issues. The causes of this situation were not clearly identified, but were perceived by key informants interviewed by the University of British Columbia (during focus groups), as being more related to structural issues rather than to an absence of potential workers. Such issues included a lack of collaboration from third party employment support agencies.^{Footnote 115}

A progress report prepared by the University of British Columbia indicates that the introduction of autism outreach coordinators has helped to make progress in the area of participant recruitment. These coordinators' role is to identify job seekers with autism spectrum disorders who do not have a co-occurring intellectual disability.^{Footnote 116}

According to this report, these individuals are “unlikely to qualify for traditional services offered to people with intellectual disabilities or co-occurring intellectual disabilities and autism spectrum disorders,” while generally having “employment skills and assets that are beyond those required for jobs traditionally accessed through employment services for people with intellectual disabilities”. The report also states that this situation results “in a disconnect between opportunities and ability which can lead to a lack of opportunity to meet their full employment potential.”^{Footnote 117}

Progress was realized in addressing challenges encountered in the delivery of the project

A formative evaluation of the Ready, Willing and Able initiative found that there were tensions between national and provincial/territorial project partners, and third party employment agencies. Such tensions were said to have created a “competition-like environment” as well as a “disincentive to collaborate.”^{Footnote 118}

According to a progress report prepared by the University of British Columbia, significant improvements have been reported in this regard, in particular between the national Ready, Willing and Able team and provincial/territorial project partners. At the same time, according to the report, collaboration issues remained, especially between Ready, Willing and Able staff and locally supported (third party) employment agencies.

Such challenges range from an absence of service providers in some regions to what was described by key informants as "philosophical differences" in the Ready, Willing and Able approach and that of traditional service providers.^{Footnote 119} More specifically, the first focuses on supporting people in getting jobs, while the latter focuses on training and work readiness.

Finally, it was also reported during focus groups that participants have expressed concerns with the fact that the roles and responsibilities of the wide range of partner organizations involved in the Ready, Willing and Able initiative were not clearly defined.^{Footnote 120}

The majority of Ready, Willing and Able participants find employment in sectors that are traditionally associated with supported employment

According to the progress report prepared by the University of British Columbia, notable increases in job opportunities for participants occurred in sectors where people with intellectual disabilities haven't historically worked. At the same time, the engagement of “non-traditional” sectors (and “non-traditional roles” within traditional sectors) was said to remain a “complicated long game” that requires a “significant investment of time with a relatively limited number of outcomes.”^{Footnote 121} Such challenges were further impacted by job-seeker availability issues in provinces or territories across Canada.^{Footnote 122}

The document review indicated that an emphasis was placed on national-scale employers for the work placement of Ready, Willing and Able participants. According to a few project representatives interviewed as key informants, in general, larger employers tend to be more receptive to hiring persons with disabilities, as they have more resources. Therefore, they may already have human resources policies that help better integrate these individuals into the workplace. A few other project representatives noted that small and medium enterprises tend to have more flexible and simple hiring processes. This makes them easier to work with for the purposes of Wage Subsidy activities, which are not undertaken by Ready, Willing and Able participants.

3.5 Program costs

The average costs per client served through the Opportunities Fund for Persons with Disabilities were comparable to the average costs per client of other federal labour market programs for disadvantaged groups

A limited cost analysis of the Program was undertaken for Phase I. As shown in Table 4, average cost per client served through the Opportunities Fund for Persons with Disabilities compared favorably with average costs per client served through other federal labour market programs for disadvantaged groups.

For example, from fiscal year 2012 to 2013 to fiscal year 2016 to 2017, the average cost per client served under the Opportunities Fund was $6,961. This is lower than both the average cost per client served under the Aboriginal Skills and Employment Training Strategy (ASETS) ($7,389), and the average cost per client served under the Skills Link stream of the Youth Employment Strategy ($8,149).^{Footnote 127}

Data sources: Chief Financial Officer Branch; Performance Measurement, Analysis and Reporting, Employment and Social Development Canada, July 2017.

As shown in Table 5, on average, from fiscal year 2012 to 2013 to fiscal year 2016 to 2017, the cost per Opportunities Fund client with successful outcome (employed, self-employed or returned to school) was $14,687. This is slightly higher than the average cost per Aboriginal Skills and Employment Training Strategy client employed or returned to school ($12,886), and higher than the average cost per Skills Link client employed or returned to school ($11,719).

Table 5: Costs per successful client, fiscal years 2012 to 2013 till 2016 to 2017

Source of data: Chief Financial Officer Branch; Performance Measurement, Analysis and Reporting, Employment and Social Development Canada, July 2017.

It should be noted that the amounts presented in Table 4 and Table 5 only include departmental contributions towards projects' costs. However, these costs do not include contributions made towards the Ready, Willing and Able and the CommunityWorks/Worktopia initiatives. Due to differences in their design, these two projects cannot be compared to the other Opportunities Fund projects.

Finally, it was not possible to determine whether operations and management costs of the Opportunities Fund (detailed in Table 6 below), were in line with the Department's standards for grants and contributions programs.

Table 6: Operations and management costs of the Opportunities Fund

Source of data: Chief Financial Officer Branch; Performance Measurement, Analysis and Reporting, Employment and Social Development Canada, July 2017.

Note: data was only available for fiscal years 2015 to 2016 and 2016 to 2017.

3.6 Performance measurement strategy and quality of administrative data

More information could be collected on employers' involvement with the program, and on the progress made in the area of employer awareness. Progress made through Self-Employment activities, as well as the impacts interventions have on participants' skills and employability, require a fuller assessment by the Program. Administrative data were found to be of good quality, with minimal missing data.

Administrative data were found to be of good quality, while gaps were identified in relation to up-to-date participant contact information and information on employers in general

The Summative Evaluation of the Opportunities Fund for Persons with Disabilities that was conducted in 2014 noted a lack of consistency in the reporting of project numbers in the Common System for Grants and Contributions. The evaluation noted that pre-April 1, 2015 data still contained missing or questionable end dates and duplicate intervention records. Such issues were not found in the data collected after April 2015.

An area that could be improved upon is the participants' contact information fields. These were found to contain out-of-date, inaccurate or missing information. In particular, the administration of the survey of participants revealed that a high proportion of mailing addresses, telephone numbers and e-mail addresses were invalid, and that approximately half of participants did not have an email address.^{Footnote 128}

Similarly, employer contact information and information on their experiences with the Program could be collected in a more systematic way. Almost all employers hiring participants through Wage Subsidy activities under the reformed Program do so through a Community Coordinator, and do not enter directly into funding agreements with the Department. Therefore, employers' contact information is now very rarely collected directly by the Program.^{Footnote 129} Having access to such information would facilitate the collection of data used for evaluation and performance measurement purposes, including feedback from employers who hired program participants through wage subsidy agreements or have received funding for workplace accommodations.

Changes in post-program labour market outcomes for participants' could be more accurately reflected in the Program's performance data

According to the Program's operational directives, for single year projects, participants' results can be updated up to 12 weeks after the end of their last intervention. As well, updates cannot be done later than 60 days following the project end date. Consequently, the situation of a single year project participant who becomes employed or self-employed only after the end of the reporting period (for example, four months after completing the Program) would not be accurately reflected in the Program's performance data.

Further, the Program's operational directives indicate that participants' results can only be updated if the initial result is ‘not employed' or ‘unknown.' Therefore, as soon as a participant is deemed employed, self-employed or returned to school, the Program does not update this participant's labour market situation, even if changes happen within the reporting period.

Finally, the review of regional and national project documents indicated that participant results may vary substantially from one project to another. It was also found that the results of participants in small scale single-year projects were often equal or better than those of some multi-year projects. The causes behind this trend could not be determined.

Further evidence is needed to assess the progress made through Self-Employment activities

The results showing the number of participants employed or self-employed at the end of an intervention are reported into a single category, and cannot be disaggregated. Further, the documents reviewed for projects comprising Self-Employment activities do not indicate whether or not participants have started to implement their business plans, or if those who did eventually proved successful.

Program activities' impact on participants' skills and employability needs to be better assessed

For performance measurement purposes, participants are considered to be more employable as soon as they have completed one program activity or did not complete their first program activity because they obtained employment.^{Footnote 130} Therefore, as soon as participants have received Enhanced Employment Assistance Services, they are recorded as participants with “enhanced employability,” whether or not they subsequently take part in Wage Subsidy activities, or become employed or self-employed.^{Footnote 131}

A recipient organization noted that persons with disabilities may experience many unique situations that cannot necessarily be reflected in the categories used for performance measurement purposes under the Program. This organization pointed to a need for a more comprehensive database that effectively captures participants' transitions between interventions and path through the Program.^{Footnote 132}

Key elements of the Program's reforms have not been reflected in the Program's performance measurement strategy

In order to strengthen performance measurement and to reflect the Program's reforms, the Department implemented a new participant information form in fiscal year 2016 to 2017. This new form notably contains a field that captures the small and medium-sized enterprise status of organizations hiring participants through Wage Subsidy agreements.

However, it is still not possible to determine whether or not participants were retained by their host employer once the subsidy ended. Such an indicator would help gauge the extent to which employers' needs are met, and the extent to which Wage Subsidy activities help persons with disabilities seize “meaningful mainstream employment opportunities” and “obtain permanent jobs.”^{Footnote 133}

Overall, employment relationships created through the Program are discussed on an ad hoc basis. This is mainly done in the success stories and lessons learned sections of activity reports prepared by funding recipients. Finally, the new performance strategy identifies “the number of employers who indicate increased awareness” as an indicator of success for Employer Awareness activities. Presently, performance data are not collected in support of this indicator.

The results for participants of the Ready, Willing and Able project appear to be strong and monitoring those results more closely may be warranted

When the evaluation work was conducted, the latest activity report produced by the Canadian Association for Community Living indicated that a total of 1,447 employment outcomes, including 301 employment opportunities at less than 15 hours per week, have been generated through the Ready, Willing and Able initiative. This activity report also indicated that approximately 95% of the individuals who were hired through the initiative were retained by their employer.^{Footnote 134}

These results are reported at an aggregate level by the recipient organization and are not validated by the Program. In comparison, the Department collects individual information on each participant through participant information forms, for all other projects except stand-alone Employer Awareness projects, which do not include participants. This process notably allows to confirm eligibility and constitutes a key source of performance related data.