Evaluation of the Future Skills program - 2018 to 2023

Ongoing need and demand for the program

• Canada’s skills development ecosystem continues to face ongoing challenges in the area of job-related training and labour market information.
• The labour market is evolving, and adjustments are needed to make sure Canadians, and the Canadian economy can thrive in a rapidly changing environment. There is an ongoing need for information and evidence about solutions for re-skilling or up-skilling to enable workers to adapt to changes in the labour market.
• Underrepresented and underserved groups in the labour market continue to face more barriers. The Program’s focus on these groups remains in line with government priorities.
• There is an ongoing effort to continue to avoid duplication and complement existing programs and initiatives in the skills and training ecosystem.

Program outcomes

• The COVID-19 pandemic presented challenges in the early years of the Program and resulted in delays to achieve the Program's outcomes.
• The Program has been effective in engaging a pan-Canadian network of diverse stakeholders and experts across the skills development ecosystem.
• The Program is setting the stage for innovation in the skills development ecosystem. But the extent to which it is experimenting innovative approaches is unclear and may be too early to assess.
• Early results show that there was knowledge developed for the skills development ecosystem. But not enough evidence is available to assess the usefulness of the knowledge produced.
• The Program has undertaken a variety of knowledge mobilization activities to help inform and influence skills-related policies and programs, the impacts of these efforts on policies and programs are unclear.
• Most funded projects focus on underrepresented, disadvantaged and equity-deserving groups. But it is too early to assess the impacts on these groups.

Governance structure

The design of the Future Skills Program was based on 3 pillars with distinct roles and responsibilities to support the objectives. The first and central pillar is the Future Skills Centre (the Centre). The Centre was established in February 2019 via a single Contribution Agreement between ESDC and the Toronto Metropolitan University (then Ryerson University). A consortium was created between the Toronto Metropolitan University, the Conference Board of Canada, and Blueprint-ADE.

The second pillar is the Future Skills Council, an advisory body composed of subject-matter experts in skills development. The Future Skills Office (within Employment and Social Development Canada) is the third pillar.

• The third-party delivery model is seen as effective for delivering on the objectives of the program. But roles and responsibilities of the department and the Future Skills Centre aren't always clear.
• The Future Skills Council has fulfilled its mandate.
• The consortium model governing the Future Skills Centre is a non-traditional approach to achieving the objectives of a federal program. But, there are challenges related to lines of accountability and lack of clarity in responsibilities with this type of partnership.

Performance measurement

• Data available from the Future Skills Centre meets the reporting requirements of the Contribution Agreement.
• But, the information isn't sufficient for ESDC to monitor and report on performance over time and measure impacts, particularly on underreprensented groups.

Recommendations

1. Continue to improve and strengthen data collection and reporting on the Program’s performance to support better decision-making.
2. Strengthen knowledge synthesis and mobilization activities for evidence generated by the Future Skills Centre.
3. Take steps to clarify the role of ESDC and increase its visibility to support efforts to mobilize knowledge and influence skills-related policies and programming.

Overall management response

Management accepts the recommendations outlined in the Evaluation of the Future Skills Program. We will use the evaluation findings to inform continuous improvement of the program design and delivery. The Program aims to support Canada’s skills policies and programs to adapt and meet the evolving needs of job seekers, workers and employers within a changing world of work. It has a unique delivery model through the creation of the Future Skills Centre, an independent innovation and applied-research centre. The Centre identifies emerging skills trends, tests and implements new approaches to skills assessment and development, and disseminates evidence widely to inform future programming.

Future Skills’ objectives align with departmental priorities to strengthen the training systems in Canada to build human capital; to address the changing way that people work; and to foster equity, diversity and inclusion. The program also supports other key government priorities such as sustainable jobs, addressing labour shortages, and digital economy.

Evaluation findings show that there is an ongoing need for a program that is future oriented, evidence-based, with a focus on innovation, experimentation and inclusion in the skills and training development ecosystem. Through its delivery model, the Program has been effective in engaging a pan-Canadian network of multi-sectoral stakeholders and experts across the skills development ecosystem enabling the Program to complement existing programs and initiatives and avoid duplication. While the program has many ongoing projects, the full impact has yet to be realized as the bulk of the projects are due to complete in fiscal year 2023 to 2024.

Significant efforts were already in progress during the time of the evaluation aligned with the findings of the report, including improving performance management practices, strengthening knowledge synthesis, and increasing Program visibility.

Recommendation #1

Continue to improve and strengthen data collection and reporting on the Program’s performance to support better decision-making.

As the Program continues to mature and an increasing number of funded projects are reporting on results, it will be important to improve upon the performance measurement strategy. The strategy would benefit from focusing more on collecting consistent and meaningful outcome data and effectively utilizing the performance data to support the management and the accountability of the Program.

The Program should explore opportunities to collaborate with the Centre to better align and streamline data collection efforts around key performance indicators. The Program could also explore the Centre’s Research, Evaluation and Knowledge Mobilization’s “Learning and Evaluation Database” to find potential opportunities to support more timely, targeted, and advanced analysis of project information to support performance measurement needs.

Also, the Program could give further consideration to how it uses the performance information collected to better support decision-making. Indeed, it isn't clear how and to what extent the collected performance information from the Centre is currently used.

Management response #1

Management agrees with this recommendation. The Department had already been working on improving and strengthening performance measurement practices. Over the past year, ESDC undertook an in-depth analysis of existing data collection and reporting practices in relation to the requirements outlined in the Performance Information Profile and the contribution agreement. The review identified areas for improvement, and steps will be taken to strengthen the Program’s performance measurement strategy going forward. The Program will also explore the possibility for leveraging the Centre’s “Learning and Evaluation Database” for meeting identified reporting gaps and improving overall performance measurement of the Program.

ESDC will continue to monitor the Centre’s results with respect to quarterly data reporting requirements as outlined in the contribution agreement. As well, ESDC will continue to review reporting requirements and expectations through the contribution agreement and Performance Information Profile to collect relevant data to support continuous Program improvement.

Recommendation #2

Strengthen knowledge synthesis and mobilization activities for evidence generated by the Future Skills Centre.

A key program objective of the Future Skills Program is to support transformative labour market policy and program innovation to ensure that Canada’s labour market and training systems remain future-fit. To that end, the Program aims to:

•  find emerging in-demand skills now and into the future;
• support experimentation by prototyping, testing and rigorously measuring innovative approaches;
• publicly disseminate evidence on what works; and
• foster the enabling conditions for replication and scaling.

The Program has demonstrated significant progress in identifying skills and supporting experimentation. It has also effectively established relations with provincial, territorial, and federal partners through the work of the Centre and the Office. With the majority of projects funded by the program ending in fiscal year 2023 to 2024, the Program is encouraged to place greater emphasis on producing a more comprehensive analysis and reporting on lessons learned on what is working and not working to foster the enabling conditions to meet job seekers, workers and employers evolving needs. The Program should also aspire to more cohesive knowledge synthesis and mobilization to maximize insights from evidence generated by the Centre to support replication and scaling of promising practices.

Management response #2

Management agrees with the recommendation. The Department has begun working on improving the monitoring of the knowledge synthesis and mobilization activities generated by the Future Skills Centre. For example, ESDC will identify opportunities to support the Centre’s dissemination efforts through the Forum of Labour Market Ministers. The outputs from the Future Skills Centre, including but not limited to knowledge synthesis and mobilization strategy and products will be part of ongoing monitoring activities. The Department will assess whether related outputs provide quality policy and program design insights generated through knowledge synthesis activities such as but not limited to reports, webinars, and presentations.

Recommendation #3

Take steps to clarify the role of ESDC and increase its visibility to support efforts to mobilize knowledge and influence labour market policies and programming.

In the implementation of the Future Skills initiative, internal and external engagement and communications activities were mainly carried out by the Future Skills Council and Future Skills Centre. This has resulted in a lack of awareness and clarity about ESDC’s role to fulfill the Program’s objectives with respect to supporting transformative labour market policy and program innovation. As the Program continues to mature and more knowledge is produced, the role and visibility could affect its efforts to influence government policies and programs.

Furthermore, working with the Future Skills Centre to make sure roles and responsibilities are clearly defined can help ensure better support and complementarity of eachothers’ work and minimizes inefficiencies associated with potential duplication.

Management response #3

In the initial implementation of the initiative, the Department prioritized visibility of the Future Skills Council and Future Skills Centre to reinforce the importance of cross-sectoral collaboration to identify emerging workforce and skills trends and to develop solutions to real world challenges. With the launch of the Program, ESDC established bilateral relations with federal partners, such as Public Safety (on the National Cyber Security Strategy), and with the Department of Fisheries and Oceans (on the Blue Economy). ESDC also established an interdepartmental Jobs, Skills and Training Table to foster dialogue on whole-of-government priorities. This is besides ongoing bilateral (with provincial and territorial governments) and multilateral (through the Forum of Labour Market Ministers) engagement to ease collaboration on areas of mutual interest.

Management agrees with the recommendation. The Department will build on strong working relations developed with federal, provincial and territorial governments to augment efforts to support the adoption of evidence generated by the Centre into future policy and program design. ESDC will develop a plan to leverage knowledge generated to strengthen collaboration with different orders of government and other policy leads. The Department will reinforce clarity of roles and responsibilities between ESDC and the Centre. This will mitigate risks of duplication and maximize ESDC’s contribution to support transformative labour market policy and program innovation to ensure that Canada’s labour market and training systems remain future-fit.

2.1 Program objectives

The Program aims to provide access to in-demand quality training that addresses the changing nature of work, especially for underrepresented groups. It does so by achieving collaboration, innovation and transformation in Canada through research and investments with provinces and territories, as well as not-for-profit organizations. Based on evidence generated from the Program, individuals and organizations involved in skills-related policies and programs will adapt and keep pace with the changing labour market to meet job seekers’, workers’ and employers’ evolving needs. In particular, the Program aims to achieve the following outcomes:

• identify emerging in-demand skills now and into the future
• support experimentation by prototyping, testing and rigorously measuring innovative approaches
• publicly disseminate evidence on what works, and
• foster the enabling conditions for replication and scaling.

2.2 Program resources

As announced in Budget 2017 and 2018, the Program’s total budget amounts to $375M for the period from 2018 to 2019 to 2023 to 2024 (Table 4).

Program funding gradually scaled up in the first two fiscal years and reached $75 million per year as of fiscal year 2020 to 2021. The following tables give actual and planned resources information for the first 6 fiscal years of the program from 2018 to 2019 to 2023 to 2024.

The evaluation covers only the 5-year period from 2018 to 2019 to 2022 to 2023.

• Source: Chief Finance Office Branch.

• Source: GCInfobase.

2.3 The 3 pillars of Future Skills

The design of the Future Skills Program is based on 3 pillars with distinct roles and responsibilities to support its objectives. The first pillar is Employment and Social Development Canada, where responsibilities are shared between the Skills and Employment Branch and Program Operations Branch. The Future Skills Council is the second pillar, while the Future Skills Centre is the third and central pillar.

3.1 Evaluation coverage

This evaluation covers the 5-year period from April 2018 to March 2023. It was completed in compliance with the Financial Administration Act and the Policy on Results.

This is the first evaluation of the Program since its inception. Given early delays in the Program’s implementation and, above all the COVID-19 pandemic, the focus of the evaluation is more formative than summative in nature: it is based mainly on the delivery of the Program and the roles of partners. The program impacts could not be assessed as they have yet to be achieved and reported on. It takes time to test new approaches, observe and measure their impacts and disseminate the results. With only a limited number of projects completed recently, early evidence of program impacts could not be examined.

3.2 Objectives, scope, and methodology of the Evaluation

The evaluation sought to assess the extent to which the Program has been able to progress towards its objectives via the following 4 questions:

1. What is the relevance of the Program within Canada’s skills and training ecosystem, and how does it align with other models and complement similar initiatives in provinces and territories?
2. To what extent has the Centre and the projects it has funded contributed towards realizing its key objectives to improve knowledge, test innovative approaches and widely disseminate evidence?
3. How effective is the program design for developing skills development approaches that are more inclusive and responsive to underrepresented and disadvantaged groups?
4. Are there any lessons learned from the governance and its impact on the Program? In terms of (a) How effective is the consortium approach for program delivery by the Centre? And (b) To what extent did activities performed by the Future Skills Council influence the Program during its initial implementation stage?

The evaluation plan was designed in consultation of the Evaluation Working Group to complement the evidence gathered in the internal Future Skills Review (hereafter called the “two-year review”) conducted between 2020 and 2021. Besides, the evaluation used 3 lines of evidence :

• The Literature review included Canadian academic journal articles and the grey literature published since 2015 including provincial/territorial governments reports and data available from Statistics Canada. Further, information about similar programs in other countries were also explored. Key search terms included skills, skills development, employment training, future skills, labour market, workforce development, and re-skilling and up-skilling. A total of 59 sources were included as part of this review.
• 29 Key informant interviews were conducted with program stakeholders divided into two groups:
  • Internal stakeholders (n=14) included representatives of the Centre (n=9), the Office (n=4), and the Future Skills Council (n=1).
  • External stakeholders (n=15) included representatives of provincial and territorial governments, and Indigenous organizations (n=7), as well as representatives of regional stakeholder organizations, and other federal government departments and agencies (n=8).
  • The following scale was used to show the significance of the interview findings. The percentages correspond to the relative weight of responses from key informants who held similar views:
    • “All/almost all” – findings reflect the views and opinions of 90% or more of the key informants in the group.
    • “Most” – findings reflect the views and opinions of at least 75% but less than 90% of key informants in the group.
    • “Majority” – findings reflect the views and opinions of at least 51% but less than 75% of key informants in the group.
    • “Some” – findings reflect the views and opinions of at least 25% but less than 50% of key informants in the group.
    • “A few” – findings reflect the views and opinions of at least two, but less than 25% of key informants in the group.
• The Documents and Administrative Data review examined information produced since the Program’s start in 2018 such as:
  • Examples of knowledge mobilization activities carried out by Future Skills Office.
  • Future Skills Council meeting records of discussions and Final report.
  • Future Skills Centre documents (Strategic plan, Business plan, Contribution agreement, Project evaluation reports, quarterly activity reports, etc.)
  • Future Skills partners’ documents (Developmental Evaluation Summary, Blueprint Evaluation reports, Blueprint annual report)
  • Administrative data and documents such as Program’s Performance Information Profile, Treasury Board submissions, Departmental Plans and Departmental Results Reports.

3.3 Limitations to the evaluation

Some limiting factors affected the breadth and quality of evidence available to inform this evaluation.

First, the focus of the evaluation had to be realigned on delivery and outputs of the Program, rather than on its outcomes (either immediate or intermediate). Given early delays in the Program’s implementation and, above all the COVID-19 pandemic, more time is needed to have enough completed projects to inform impacts. As of January 1st, 2023, 14% of all initiated projects were complete (33 out of 230 projects). Most projects are still in progress and little evidence is available on their outcomes. This applies particularly to projects meant to implement replication and scaling-up of successful skills development innovative approaches.

The original evaluation approach was modified to increase efficiency and avoid duplication. So, the survey of stakeholders was cancelled with the expectation to use the results of the survey conducted by the Program, as part of the two-year review. But the very low response rate to this survey (1.37% or 126 responses) prevented its use to inform the evaluation given the lack of reliability of the data collected. This effectively limited the number of lines of evidence to inform this evaluation to 3.

The fact that the information provided by the Future Skills Centre to ESDC, over the years, isn't consistent limited the documents and administrative data review. Also, there is no standard measurement of projects funded by the Future Skills Centre, so comparison and analysis are difficult. Tables presented in the evaluation report include the most recent information provided by the Future Skills Centre. The Centre changed its information reporting format for quarterly activity reports in year 5, so overtime trends and tracking are not possible and limits the analytical possibility.

Finding 1. The labour market is evolving and adjustments are needed to ensure Canadians and the Canadian economy can thrive in a rapidly changing environment. There is an ongoing need for information and evidence about solutions for re-skilling or up-skilling to enable workers to adapt to changes in the labour market.

The literature reviewed indicates that the economy in Canada and abroad is changing rapidly, which has a profound impact on the nature of work and jobs of the future. Some major factors influence the labour market, including technological change, globalization, demographic shifts, climate change, as well as the COVID-19 pandemic and its aftermath. These factors have also affected the skills and qualifications needed to enter and grow in the workforce.

• Technological Change: Advances in Artificial Intelligence, robotics, and other technologies are pushing the frontier of automation, and threaten to displace workers in a range of jobs and tasks. Rapid technological advancement increases the pace of skill obsolescence. These advancements and new business models are affecting job seekers’, workers’ and employers’ ability to adapt and keep up with the pace of change.
• Globalization: For the OECD (2019), there is an expectation that technological advance combined with globalization could result in a larger number of “workers being displaced from their jobs because of skills obsolescence”. But “substantial contraction of employment is unlikely” mainly because these factors not only destroy jobs but also create and transform them. This reinforces the need for supporting workers (adversely affected by global trade) as they adapt to changing economic realities. This in turn enhances resilience of the labour market to “effectively prepare for a range of potential futures”.
• Climate Change: Climate change is having significant adverse effects on Canada and the world. These effects are most heavily felt by marginalized groups, contributing to increases in pre-existing social inequalities. Jobs and skills needed to transition to a more climate-friendly economy require re-skilling and up-skilling, especially among those in jobs related to natural resources extraction.
• Demographic Shifts: Canada’s labour market challenges are projected to compound as the population continues to age. The Canadian labour force among the working-age population is now the oldest it has ever been, with more than 20% of the working-age group being between the ages of 55 and 64 according to Statistics Canada. Canada’s increasingly aging population results in tighter labour markets and relatively higher levels of job vacancies. According to the OECD (2019), age is a significant predictor of continuous skill development. Especially when it overlaps with the previous drivers: “older adults are likely to experience significant skills obsolescence, particularly in the context of technological change, unless further training is available to upgrade what they learnt in initial education”.
• COVID-19 Pandemic: The pandemic accelerated and added to some of the challenges associated with the drivers identified above. It brought a significant shift to remote work, new and improved health and safety policies and widespread use of digital collaboration tools. It also highlighted the interconnectedness between countries through global value chains. The pandemic has brought into sharp focus many of the challenges facing underrepresented groups. Job markets are evolving, which has led to a surge in demand for many new skills.

Evidence gathered in key informant interviews indicates that external stakeholders unanimously acknowledged an ongoing need to fund innovative projects and generate research related to the future of work. The most pressing needs identified by interviewees were:

• mobilizing knowledge, such as scaling up projects
• fostering lifelong learning among those in the workforce because of continuous changes or transitions in the labour market
• designing training programs that help workers quickly master new skills and gain competencies
• researching megatrends, structural changes in the global economy

Most of these pressing needs predate the implementation of the Future Skills program but are still present in the Canadian skills development ecosystem. Relatedly, the Labour Market Information Council (2022) recently identified “8 persistent challenges” to Labour Market Information (LMI):

• a lack of local and granular LMI data
• the most frequently used LMI-gathering techniques are rooted in traditional methods, to the exclusion of innovation
• there is no commonly agreed-upon methodology for projecting future skills
• LMI data is often inaccessible, unreliable or not relevant to stakeholder needs
• the needs of end-users aren't being met by most current LMI tools
• the overall impact of LMI tools isn't being tracked
• we lack data that represents a diversity of population groups
• the private sector is innovating, but their data are challenging to validate

Finding 2. Canada’s skills development ecosystem continues to face ongoing challenges in the area of job-related training and labour market information.

Challenges resulting from the rapidly evolving skills requirements are compounded by the barriers Canadians face in accessing labour market information, skills training, and jobs. Some of the most common barriers identified in the two-year review and the literature review include:

• Limited employer-sponsored training makes it difficult for workers to adapt to changing conditions : Canada lags behind other countries in terms of the incidence and number of hours of employer sponsored training. Small and medium-sized enterprises are significantly less likely than larger employers to invest in employee training. The rise of the “gig” economy means that an increasing number of Canadians will find employment through independent contract work and won't have access to traditional employer-led training. Also, there are inherent inequities in employer-sponsored training, as better educated/skilled employees are more likely to receive training than those with lower levels of education. Moreover, only 46% of working-age Canadians take part in job-related training at all. Also, 31% say they want to participate in training but are facing barriers. Main barriers include insufficient time due to work or family commitments, high training cost, or lack of employer support.
• Shortfalls in essential skills required by employers : The Future Skills Council report (2020) stated that “Employers also tell us they need workers with foundational skills such as communication, teamwork, critical thinking, creativity and problem-solving in occupations across all sectors of the economy. They see these skills as integral for productivity and well-functioning workplaces. There is also increasing recognition that social and emotional skills will be important in workplaces being reshaped by new technologies and tools. While some call these “soft skills”, the Council indicated that they see these skills as durable skills that are central to helping people pivot, transition and succeed from one opportunity to the next, even if they are among the hardest skills to develop proficiency in. Labour market information needs to develop consistent, evidence-based, measurable definitions for the various transferable skills.
• Decentralized labour market information creates barriers to matching job seekers with employers : Canada has a lot of labour market information, but the data is fragmented, often hard to access and has many gaps. Information gaps include developments in the workplace, the balance of labour demand and supply in local markets, and the longer-term experience of college and university graduates in the labour market. Provincial and territorial governments collect a wealth of data for their regions, but its sensitivity and a lack of standardization in sampling and terminology make it difficult to use at the national level.

These barriers predate the implementation of the Future Skills program but are still present in the Canadian skills development ecosystem. As presented in the previous section, the Labour Market Information Council report also identified similar barriers.

Finding 3. Underrepresented and underserved groups in the labour market face more barriers.

The labour market changes and the barriers to skills training and information disproportionately affect those who are underrepresented and disadvantaged by the labour market. Research suggests that Canada’s current skills ecosystem isn't well-equipped to address these challenges. Canada Beyond 150 noted in its Future of Work Report (2017) that “the Government of Canada has articulated the importance of addressing pressures of globalization, digital transition, and protectionism by focusing on socio-economic inclusion”.

In a 2021 report, the Social Research and Demonstration Corporation (SRDC) noted that “groups underrepresented in Canada’s labour market include women, youth, Indigenous persons, newcomers, racialized groups, people who identify as 2SLGBTQI+, and persons with disabilities. These groups also tend to be among the least well served by training and employment programs”. While anyone can face a range of barriers to skills development and employment, research highlights “many systemic barriers faced by equity-seeking groups, regardless of skill level”. Employment rates and labour market participation rates are lower for these groups.

For example, the two-year review found that despite progress, Indigenous peoples don't enjoy the full benefits of Canada’s economy. The gap endures between Indigenous peoples and non-Indigenous people with respect to many outcomes such as employment, education and housing. Besides, certain jobs have a marked predominance by gender. For example, women disproportionately hold the service sector and administrative jobs, such as cashiers and clerical, that are now being automated at a rapid pace. Men still hold a disproportionate share of skilled trade occupations, which are both well paid and expected to be in demand well into the foreseeable future.

Additionally, a lack of disaggregated data exists to understand the diverse experiences and needs of different equity-seeking groups in Canada. There are some existing programs and initiatives targeted toward underrepresented groups in Canada (such as the Accessible Canada Grants and the Indigenous Skills and Employment Training Program). But as noted by SRDC, “with some exceptions, most national datasets lack appropriate identifiers to capture diverse identities. Data gaps are particularly pronounced for specific equity-seeking groups such as bisexual, trans and gender minority individuals, and about intersections with other identities such as Two-Spirit. Available sample sizes typically don't allow for stratified analyses, deepening inequities”. Furthermore, according to SRDC, there is a “lack of data on the specific experiences of equity-seeking groups [which] serves as a key barrier to designing programs and interventions to address inequities”.

For the Brookfield Institute for Innovation and Entrepreneurship (2017), the uneven labour market representation across different socio-demographics “underlines the need for thoughtful, tailored solutions that are designed in close collaboration with the populations they are intending to support, to ensure that youth from all backgrounds are afforded equal opportunities in Canada’s dynamic, growing tech economy”. For example, recent census data shows that almost one quarter of youth aged 20 to 24 were members of a visible minority. The unemployment rate for this group was 17.2% compared to 14.1% for their white counterparts. The unemployment rate for Indigenous youth in this age category was 22.6%.

Finding 4. The Future Skills Program complements existing programs and initiatives. Ongoing effort is being made to continue to avoid duplication.

The skills ecosystem in Canada includes multiple programs and initiatives to support education, training, and skills development for Canadian workers. However, within the existing system some ongoing gaps need to be addressed to ensure that Canadian workers and employers are provided with the information and training needed to succeed in the evolving labour market.

In 2020, the Future Skills Council released its report about knowledge gaps in the skills development ecosystem in Canada. The report recognizes that business, labour, education and training providers, Indigenous and not-for-profit organizations, and governments need to collaborate. The report presents 5 priorities that highlight the importance of collaboration and innovation across all sectors to build a skilled, agile workforce that is ready and able to shape the future:

• helping Canadians make informed choices by providing access to relevant, reliable and timely labour market information and tools so that all Canadians can make informed learning and training decisions
• equality of opportunity for lifelong learning by promoting, enabling and supporting learning throughout Canadians’ lives, in particular by removing structural and systemic barriers to participation
• skills development to support Indigenous self-determination by enabling First Nation, Inuit and Métis learning and skills development based on a commitment to reconciliation and self-determination
• new and innovative approaches to skills development and validation by promoting, enabling and validating skills development and training in all their diverse forms
• skills development for sustainable futures by developing a skilled workforce capable of adopting new technologies and business models while ensuring the well-being of communities and society

Despite the existence of multiple research and development projects and programs, ongoing and persistent challenges still exist in the development of labour market information (LMI) in Canada. Through its system-wide collaboration efforts and its focus on innovation and evidence generation, the Program aims to address these ongoing challenges by:

1. Focussing on innovation that is evidence-based for the future : existing initiatives are largely reactionary and focus on addressing current labour market challenges. For example, provinces and territories tend to spend more on labour market partnerships to find solutions to existing labour market issues, but spend less on research and innovation to identify “new” ways of solving problems. The Future Skills Program aims to experiment, foster innovation and disseminate knowledge on skills development and is geared towards building a more responsive, adaptable labour market.
2. Enabling a national, systemic, comprehensive approach : existing initiatives and programs, even where funding is provided by the federal government, tend to target knowledge development and training supports at a regional or provincial/territorial level. While some benefits are associated with this decentralized approach, it seems that a more national systemic lens is lacking to understand the challenges of the future and how to address those across the country. As a pan-Canadian initiative, the Future Skills Centre aims to “connect ideas and innovations generated across Canada so that workers and employers can succeed in the labour market and local, regional and national economies can thrive”^{Footnote 1}. The Centre aims to “bring together experts bridging supply-and-demand-side perspectives, reflecting the diverse interests and supporting approaches that are both people-centric and employer-informed”^{Footnote 2}.
3. Targeting underrepresented groups : despite the number of existing federal programs that target underrepresented groups in Canada (Youth Employment and Skills Strategy, Opportunity Funds for persons with disabilities, Indigenous Skills and Employment Training Program, etc.) there is a need to strengthen disaggregated data to better understand the diverse experiences and needs of these underrepresented groups. The Program contains a specific focus on including underrepresented and disadvantaged groups:
   • at least 50% of the Centre’s contribution amount is allocated to address the needs of underrepresented and disadvantaged groups
   • up to 20% of the Centre’s contribution amount is allocated to address the needs of youth.

More recently (February 2022), the Labour Market Information Council proposed 6 recommendations to address the 8 challenges for Canada’s LMI Landscape mentioned earlier. Of note, the first 4 are elements that the Future Skill’s Program specifically aims to address:

• collaborate to close gaps in LMI generation
• prioritize innovation in LMI
• develop, endorse, adopt, and advocate for best practices and principles
• convene the ecosystem
• champion open access to data and tools
• develop impact and evaluation frameworks for LMI

Most of the work the Program accomplishes complements existing programs and initiatives in the skills ecosystem. With a program as complex and with a scope of work as large as the Future Skills Program, some duplication is inevitable: some key informants perceived some duplication as being unavoidable.

But all key informants in the two-year review agreed that the work done by the Future Skills Program complements existing programs and initiatives. The most common reasons offered by interviewees to support this view was the Program’s focus on future-based experimentation, innovation, and research.

In addition, evidence shows that the Future Skills Centre actively tries to avoid duplication by bringing together stakeholders across the country. This has allowed for the development of a community of practice, breaking down of silos, and sharing of information/learning beyond what had been done to date. This allows various actors to build-off each other rather than duplicate.

Finding 5. The COVID-19 pandemic presented challenges in the early years of the Program and has resulted in delays in achieving outcomes.

The COVID-19 crisis led to unprecedented economic disruption, uncertainty, and hardship for many Canadians. Some sectors, regions, and populations were hit harder than others. Notably, two-thirds of the record 3 million job losses in March and April 2020 occurred in sales and services occupations — jobs where young people, women, Indigenous, immigrant, and racialized workers were all over-represented.

For the Future Skills Centre, it created both operational challenges for an organization trying to develop as a pan-Canadian entity in a time of no travel and systemic challenges that led to disproportionate effects on specific sectors of the workforce.

Specific challenges included:

• delays in implementing projects: most projects funded by the Centre have experienced delays due to challenges related to the pandemic (13 out of 16 in the first year). Thus, producing results data has also been delayed;
• the Centre promptly shifted its activities' focus by privileging projects aiming at helping employers and workers address the immediate challenges of the pandemic. In particular, it funded 64 innovation projects through a Call for proposals to explore ways in which skills innovation could promote resilience and new ways forward in the face of social and economic shocks like COVID-19. Proposals had to examine new insights and models within or across 3 levels of the skills ecosystem: innovation in support for individuals, innovation in support for organizations, and systems change; and
• adapting to using virtual technology to continue efforts to set up a pan-Canadian approach to innovation in the skills ecosystem during a period of restricted travel.

Moreover, the pandemic also impacted the pace for Future Skills Centre Call for proposals and implementing the subsequent selected projects. Figure 1 shows the evolution in the number of funded projects with 95 projects out of 230 approved after March 1, 2021.

Text description of Figure 1

"The figure shows the evolution of the number of projects funded by the program between 2019 and 2023, which increased from 6 (March 1, 2019) to 46 (March 31, 2020), 135 (March 31, 2021), 176 (March 31, 2022) to reach 230 by December 31, 2022.

The figure also presents the cumulative total of funding allocated to the projects by the program, which went from $11.5 M (March 31, 2019) to $73.6 M (March 31, 2020), $135.4 M (March 31, 2021), $206.4 M (March 31, 2022) and finally $236.7 M as of December 31, 2022."

• Source: Future Skills Centre, extracted from 17 different Quarterly activity reports.

Finding 6. The Program has been effective in engaging a pan-Canadian network of diverse stakeholders and experts across the skills development ecosystem.

The Program developed a pan-Canadian network of diverse stakeholders to enable a national, systemic, comprehensive approach to testing innovation, developing knowledge, and replicating and scaling innovation approaches in the skills ecosystem. Since the Program’s start, both the Council and the Centre have brought together actors from across the skills ecosystem to support Canada’s skills development systems in being future-oriented, evidence-based and equitable.

Finding 7. The Program is setting the stage for innovation in the skills ecosystem. But the extent to which it is experimenting with innovative approaches is unclear and too early to assess.

Overall, assessing the effectiveness of the Program in testing and experimenting with innovative approaches was challenging. Evidence indicates that the Program, largely through the activities of the Centre, has encouraged a new level of innovation in the skills' ecosystem through both its collaborative efforts and its funding to third-party projects that focus on testing the following 3 different types of innovation:

• Core innovation: focuses on incremental iterations on proven models to improve the experiences and outcomes of people accessing existing skills training and development programs. These projects are expected to use existing knowledge, infrastructure and resources to improve and scale proven solutions for increasingly larger numbers of people.
• Adjacent innovation: focuses on adaptations of proven skills development models, applied in new ways and for new needs, and sometimes for new markets. These projects are expected to strengthen the ecosystem’s ability to adapt proven models in new ways and in response to emerging opportunities and threats.
• Disruptive innovation: focuses on brand-new ideas aiming to make breakthroughs and invent skills development models for markets that are nascent or that don't exist yet, and which have the potential to make existing approaches obsolete. These projects are expected to foster truly new and nascent ideas, with the potential to replace current practices, ways of thinking or value systems.

But according to the key informants interviewed, the extent to which these efforts are resulting in approaches that are ‘truly’ innovative is unclear. Thus, it's too early to provide a comprehensive assessment of the evidence generated and the nature of the innovations. Key informants also suggested that lately, more emphasis is placed on supporting projects that are testing innovative approaches.

Some interviewees felt the support for testing innovative projects was limited and not yet exploited to its full potential because of risk aversion. Some key informants expressed the opinion that the Centre is acting too much like a traditional government-funding program designed to safely invest to showcase successes instead of selecting riskier projects which might have a greater chance of failure. Most of these interviewees felt that the Centre has the potential to promote innovation and collaboration in the skills ecosystem beyond what is typically possible, but has not yet fully achieved this potential.

As shown in Table 9, the majority (55%) of funded projects pursue core innovations which focus on incremental iterations with existing resources. Very few (6%) projects target disruptive innovations which seems to be more in line with key informants’ opinions of what is ‘true’ innovation.

• Source: Future Skills Centre, "KPI Framework and Performance Update", January 31, 2023

Key informants who felt that the Centre has not yet reached its potential in driving innovation provided 2 main reasons as to why they believe this:

• The program relies too much on funding safe projects that are likely to succeed when it needs to support more projects that are risky and might fail. It was observed that failure is a key part of experimentation and is very important when the focus is on innovation. What didn't work is as important to know as what did work
• Testing of innovative approaches has only been accomplished to a limited extent because most calls for proposals were not designed as test per se (for example, no hypothesis being tested). In this regard, a 2020 OECD review of Future Skills recommended that the Centre should set up quality standards for evaluations to improve the impact evaluation culture in Canada, possibly following the Maryland Scientific Methods Scale.

Finding 8. Early results indicate that there was knowledge developed for the skills ecosystem. But there isn't enough evidence to provide a reliable assessment of the usefulness of the knowledge produced.

Review of administrative documents shows that the work undertaken since 2018 by the Future Skills Centre and ESDC has contributed to addressing the gaps of information available in the skills development ecosystem by:

• making skills development and LMI data more accessible,
• increasing information on a diversity of population groups and especially underrepresented groups in the labour market, and
• fostering exchange of LMI amongst the labour market stakeholders.

As of March 31, 2023, the Centre has produced 120 publications focused on skills innovation, workforce development and labour market issues. These reports are accessible via a searchable database, cover the impact of automation, sustainable livelihoods, career pathways, in-demand skills, and take a skills perspective on labour market information. Also, the Future Skills website proposes a searchable database of innovation projects (by keyword, theme or region) with 128 projects listed and an interactive map (also searchable by target population, sector, skills type or project type) currently presenting only 110 projects.

There is no evidence in reviewed documents or administrative data as to the extent to which these tools are used by skills development partners or Canadians.

External interviewees agreed products were useful to at least some extent in terms of providing insights into emerging skills relevant to Canada’s future economy and labour market. But it was more difficult for external interviewees to identify ways in which products or activities have been useful in providing evidence on valuable training solutions relevant to Canada’s future economy and labour market.

A few interviewees indicated that there needs to be more focus on what does not work and what is a failure when it comes to disseminating evidence. They felt that learning from what does not work is part of research and development. The literature reviewed also refers to the importance of learning from failures to support innovation.

Leveraging from promising early results of its various projects and activities, the Centre has established a plan to ensure that information gathered from its funded projects results in valuable insights to the ecosystem. The Future Skills Centre is developing a comprehensive compendium of evidence from each project to carry this out.

Given the limited number of completed projects to date, the knowledge generated or its usefulness can't really be assessed yet.

Finding 9. The Program has undertaken a variety of knowledge mobilization activities to help inform and influence skills-related policies and programs. But impacts of these efforts on policies and programs aren't clear yet.

Both the Future Skills Centre and ESDC play a role in the knowledge mobilization on the skills of the future and on influencing policies and programs. It's too early to assess any impact related to knowledge dissemination and influence on policies and programs given the limited number of completed projects. But early results show progress in this area. Most interviewees assessed the Program’s ability to disseminate evidence in a positive way.

Some interviewees, largely external, felt results have been limited so far and raised the following points:

• While the program has many accessible reports and bulletins, there isn't enough yet to actually tell "what works" and why, or “is it replicable?”
• While evidence is growing, much of the data coming out still needs to be analysed. Clear evidence about what works, and lessons learned will take more time as there isn't enough evidence yet in the scaling portfolio.

Review of available administrative documents provides an overview of ESDC's knowledge mobilization activities:

• Knowledge Blast - curated content (including evidence, insights, and information) about Future Skills Centre research, projects, and knowledge dissemination events that is relevant to the work of various government audiences. The Knowledge Blast goes out to 140 recipients per month across ESDC and other federal government departments (including Agriculture and Agri-Food Canada, Canada Border Services Agency, Crown-Indigenous Relations and Northern Affairs Canada, Fisheries and Oceans Canada, and Environment and Climate Change Canada).
• ESDC Skills and Employment Branch Talks— a learning series that promotes discussion, collaboration, and knowledge transfer within the Branch. These talks are promoted to working level and senior management staff. Since September 2021, 5 widely-attended talks were hosted that covered the following topics:
  • insights and evidence generated from Future Skills implemented projects,
  • how they have scaled-up promising projects,
  • their relationships with provinces and territories,
  • thematic findings on digital platforms, and
  • engaging Northern communities in Canada.
• Forum for Labour Market Ministers – the Centre has been engaging with a designated task team to explore opportunities to collaborate on either new or existing projects. ESDC facilitated this engagement opportunity between the task team and the Centre.
• Assistant Deputy Minister Interdepartmental Committee on Jobs, Skills and Training – this Committee was originally created to help outline the role and mandate of the Future Skills Centre. Fifteen (15) federal departments are currently participating. The ongoing objectives of the Committee are to engage in a discussion on:
  • how the Centre can support federal departments and stakeholders address anticipated workforce development issues
  • the Centre’s Strategic Initiatives Fund
• Director General Interdepartmental Table on Jobs, Skills and Training - to support horizontal collaboration and information sharing on workforce development issues. It's chaired by Skills Employment Branch and includes fifteen (15) federal departments.
• Other Government Department Engagement –ongoing information exchanged, engagement and meetings held to inform higher level tables.

While documents reviewed show that ESDC undertakes a variety of activities to help mobilize knowledge, most key informants, both internal and external, felt the work to date in this area has been limited. The review also suggests that in the implementation of the Future Skills initiative, external engagement and communications activities were mainly carried out by the Future Skills Centre. Only a few internal key informants were aware of ESDC’s knowledge mobilization actions. The following challenges were expressed:

• ESDC limited direct contact with provinces and territories while the Future Skills Centre has relationships with these governments
• Insufficient evidence to mobilize knowledge generated to influence government policies and programs
• ESDC needs to increase its visibility to ease information-sharing. Scaling up/knowledge mobilization will likely require some facilitation through government partnerships.

On the other hand, the Future Skills Centre created structures, platforms and relationships to disseminate information and also made significant investments in this regard.

A 28.2 millions of dollars project was funded to develop and support Future Skills Centre’s digital infrastructure platform, undertake knowledge dissemination and mobilization activities, support the development and maintenance of the Future Skills Centre website and facilitate a Community of Practice platform. The platform aims to cultivate a pan-Canadian community in order to foster learning and the shared goal of strengthening Canada’s labour market for the future.

Most interviewees assessed evidence dissemination in a positive way. Reasons given were mostly related to activities and products, including the Future Skills Centre visibility and promotion of newsletters, reports, webinars, podcasts, and speaking engagements.

While the evidence shows a promising start to the Centre’s knowledge mobilization efforts, a few interviewees from the two-year review felt that there should be greater sharing of results and best practices to enable innovation project proponents to learn more from other innovation projects including their challenges and successes.

The two-year review also found that in most areas assessed the reach of the Centre’s knowledge mobilization activities had increased. Review of available data since then has confirmed this with a constant increase in website visits, social media followers (Twitter, LinkedIn, and Instagram), as well as distinct reports downloads.

Finding 10. As per the Program objective, most funded projects focus on underrepresented, disadvantaged and equity-deserving groups. But it's too early to assess the impacts on these groups.

The Program includes a specific focus on including underrepresented and disadvantaged groups so all Canadians can benefit from emerging opportunities. But the evidence on results of these approaches and outcomes for these groups is still very limited.

The Contribution Agreement with the Future Skills Centre stipulates that at least 70% of the funding must address the needs of disadvantaged and underrepresented groups with at least 50% of the Centre’s contribution amount to address the needs of underrepresented and disadvantaged groups, and up to 20% of the Centre’s contribution amount to address the needs of youth.

The Centre has exceeded this requirement by attributing 63% ($148,378,217.59 out of $236,699,063) of its funding to projects that target these groups. This accounts for 73% (168/230) of funded projects.

• Note: Many projects focus on more than one underrepresented group, so totals add up to more than 230 projects and 100%.
• Source: Future Skills Centre, Quarterly Activity Report, December 31, 2022

Key informants identified some approaches that facilitated the targeted focus through implementation of:

• an evaluation process for project applications with a focus on how they will address the needs of equity-deserving groups
• a funding stream dedicated to supporting skills development for equity-deserving groups
• a 50/30 challenge – a commitment that 50% of leadership in projects is female and 30% racialized
• a Centre’s advisory board that reflects equity-deserving groups in its membership and holds itself to the same standards for diversity and inclusion as those applied to the funding project applications assessment
• the Facilitating Access to Skilled Talent (FAST) program for newcomers by the Immigrant Employment Council of BC
• a network or community of practice, such as bringing together individuals and organizations that deal with members of equity-deserving groups.

A few interviewees emphasized that more could be done to reach groups in the North, including focusing on particularly significant issues for the North in general and the Inuit. The Future Skills Centre has started to act on this aspect with its Northern Strategy that is currently being developed.

Other specific groups identified as challenging to reach were:

• persons with disabilities
• Veterans
• individuals living in rural and remote areas
• those who need income supports
• low-skilled individuals and individuals without post-secondary credentials
• members of the 2SLGBTQI+ community.

Data on outcomes for these groups is still very limited. It should also be noted that there was little evidence available to show plans for collecting data on outcomes for these groups. Even though the Centre has to report annually on outcomes for participants, this information wasn't available in reports provided to ESDC. Available data only shows numbers of participants and does not allow for an assessment of the participants' results, so outcomes by groups can't be assessed either.

Finding 11. The Future Skills Council has fulfilled its mandate.

In Spring 2019, the Future Skills Council completed the first milestone in its mandate. The Council undertook a first round of engagement activities to gather perspectives on emerging skills and workforce trends from about 400 individuals from over 150 organizations across the country representing the public, private, labour and not-for-profit sectors. The Council prepared a high-level summary report of recurring themes on opportunities and challenges that surfaced across a broad spectrum of partners and stakeholders.

In the course of 2020, the Council drafted its final report titled Canada - A Learning Nation; A skilled, agile workforce ready to shape the future. The report included recommendations that informed Budget 2021 training announcements and ongoing efforts across federal departments to advance workforce development to support whole of government priorities.

Following the release of the report, Council members led a range of activities to champion action on the priorities listed in their report. Podcasts were created, and the Council hosted 5 digital dialogues that had over 180 participants to promote a national conversation on skills for the future of work. This promotion and engagement culminated with the virtual Future Skills Forum on June 8, 2021. About 200 stakeholders from across sectors nationwide participated. To allow for engagement leading up to the Forum, the final term of Council members was extended by 6 months to June 30, 2021. This marked the end of the Council’s term.

Finding 12. The third-party delivery model is seen as an effective model for delivering on the objectives of the Program. But roles and responsibilities of the department and the Future Skills Centre aren't always clear.

The third-party delivery model is associated with multiple benefits and interviewees consistently perceived it as the most effective model for delivering on the objectives of the Program. The establishment of the Future Skills Centre, independent of government, is one of the greatest strengths of the governance model. It's viewed as crucial to ensuring the success of the initiative in terms of generating buy-in at the pan-Canadian level. In particular, this is seen as helping to overcome provincial and territorial reluctance to take part in the Program. Other benefits of this model identified by interviewees included:

• it allows the Future Skills Centre to be in touch with what is going on in different sectors, but also to connect with government so that it can still influence government policy.
• it offers greater connectivity as government could not do the same job of knowledge mobilization that the Future Skills Centre has done. It nurtures relationships beyond government-to-government and government-to-industry and mobilizes expertise that government may not have.
• it provides more flexibility for testing innovative ideas and considering new approaches.

While evidence indicates that the third-party delivery model is a promising approach, interviewees identified 2 key challenges that were confirmed by the documents review:

1. Clarity of roles and responsibilities : in the first years of the Program, distinction between the roles of the Council and ESDC (via the Future Skills Office) was clear. Since the end of the Council's mandate, ESDC's remaining role is perceived to be less clear. Most key informants (from inside and outside the federal government) indicated a lack of clarity that pertains to the ESDC's role vis-à-vis the role of the Centre. Those interviewed as part of the two-year review and the evaluation showed a lack of familiarity with ESDC's role in the program. It was felt that there is a need for ESDC to increase its visibility and communicate more clearly its role while respecting the terms of the Contribution Agreement.
2. Low synergy and information-sharing between ESDC and the Future Skills Centre : the two-year review found that the degree of interaction and synergy between the 3 pillars of the Future Skills Program has been limited. Indeed, it's not clear how ESDC works with the Centre and how information is shared, aside from the required activity reports and annual reports as per the Contribution Agreement.

Finding 13. The consortium model used to govern the Future Skills Centre is a non-traditional approach to achieving the objectives of a federal program. But there are challenges related to lines of accountability and lack of clarity in responsibilities with this type of partnership.

While the Toronto Metropolitan University (TMU) is the sole signatory in the Contribution Agreement, the project is based on a consortium of partners including TMU, Blueprint-ADE, a non-profit research organization, and the Conference Board of Canada, a non-profit applied research organization.

• TMU is the signatory to the Contribution Agreement with Employment and Social Development Canada. TMU is responsible for offering administrative services (finance, IT, human resources) to the Future Skills Centre. The project team at TMU includes the Diversity Institute and Magnet.
  • Diversity Institute has developed new approaches to understanding future skills and meeting employer needs with a focus on removing barriers for women, racialized people, immigrants, Indigenous people, persons with disabilities and others.
  • Magnet (co-created by TMU and the Ontario Chamber of Commerce) is a social innovation platform addressing the unemployment and underemployment of Canadians. Magnet brings an extensive, technology enabled network of multi-sectoral partnerships to the Future Skills Centre with purpose-built digital infrastructure to grow and sustain a pan-Canadian network that will support Future Skills Centre initiatives.
• Blueprint-ADE draws from its extensive experience in program design, evaluation, policy analysis and advanced data analytics, to lead the evidence generation strategy and evaluation of Future Skills Centre’s innovation projects. Blueprint-ADE collaborates with policymakers and practitioners to generate rigorous evidence to understand how they can learn, improve, and maximize their impact.
• The Conference Board of Canada leads the Centre’s knowledge dissemination and mobilization activities and contribute to the Centre’s research program.

The Future Skills Centre also has an Advisory Board which provides strategic and forward-looking advice and input to the Centre’s management to support delivering on its mandate.

The consortium model was regularly credited for the early success of the initiative by leveraging the networks and strengths of each partner (Future Skills Centre, Conference Board of Canada, and Blueprint-ADE). Interviewees felt that the governance model is working mostly as intended. Interviewees believed that the Program was able to immediately start and quickly ramp up activities by relying on the expertise, resources, infrastructure, and pre-existing networks in the ecosystem of each of the consortium partners.

But the governance of the Future Skills Centre is complex and there have been some challenges related to accountability and lack of clarity on roles and responsibilities. Perceived challenges identified by interviewees from both the evaluation and the two-year review, included the following (each mentioned by a few interviewees):

• the consortium model is complexified by various contributions agreements (for example, TMU has an agreement with the Future Skills Centre, and the consortium partners have an agreement with TMU) which creates a potential for tension around lines of accountability. This was mitigated by establishing a leadership table with equal representation of each consortium partner.
• the fact that the Future Skills Centre and all members of the Consortium are located in Ontario adds pressure to the pan-Canadian focus to the program.

Finding 14. Data available from the Future Skills Centre meets the reporting requirements of the Contribution Agreement. But the information is insufficient for ESDC to monitor and report on performance over time and measure impacts, particularly on underreprensented groups.

The Program’s Performance Information Profile provides clear linkages between activities, outputs, and expected outcomes. But inconsistencies and gaps in key performance indicators collected from the Centre limit the ability to efficiently monitor progress towards achieving expected outcomes.

The type of data (such as which indicators are reported on) and the measures used to report on each indicator are sometimes inconsistent. For example, information about projects addressing the needs of underrepresented and disadvantaged groups used an estimated proportion of funds spent in the 2019-2020 January to March quarterly activity report. But in 2021-2022 this same indicator is reported on with absolute numbers rather than estimated proportions. This leads to inefficiencies and risks errors in the analysis of the data.

A variety of information and data exist about the type of activities and outputs funded and produced by the Centre. But there still is a lack of data available on the outcomes of the funded projects and their impacts on participants.

The evaluation found no tangible evidence available on results of the projects for the participants. The Program’s performance data is limited to results at the project level with data available on actors in skills development (stakeholder institutions) and knowledge produced instead of actual participants. This will make it difficult to fully assess the outcomes of the Program on individual participants and sub-groups of participants, especially those from underrepresented groups.

It's also unclear how and to what extent the Program uses the performance information collected from the Centre. While the Centre has provided the required quarterly activity and annual reports, there was no evidence available to show if and how the Program utilized that information to inform the Program's management.

The Future Skills Centre has developed a database of projects funded and began with a thematic coding of strategic learning orientation and topical/thematic groupings. But, how much or in what ways the Program uses or expects to use that database is unclear.

Recommendation #1: Continue to improve and strengthen data collection and reporting on the Program’s performance to support better decision-making.

As the Program continues to mature and an increasing number of funded projects are reporting on results, it will be important to improve upon the performance measurement strategy. The strategy would benefit from focusing more on collecting consistent and meaningful outcome data and effectively utilizing the performance data to support the management and the accountability of the Program.

The Program should explore opportunities to collaborate with the Centre to better align and streamline data collection efforts around key performance indicators. The Program could also explore the Centre’s Research, Evaluation and Knowledge Mobilization’s “Learning and Evaluation Database” to find potential opportunities to support more timely, targeted, and advanced analysis of project information to support performance measurement needs.

Also, the Program could give further consideration to how it uses the performance information collected to better support decision-making. Indeed, it isn't clear how and to what extent the collected performance information from the Centre is currently used.

Recommendation #2: Strengthen knowledge synthesis and mobilization activities for evidence generated by the Future Skills Centre.

A key program objective of the Future Skills Program is to support transformative labour market policy and program innovation to ensure that Canada’s labour market and training systems remain future-fit. To that end, the Program aims to:

• find emerging in-demand skills now and into the future;
• support experimentation by prototyping, testing and rigorously measuring innovative approaches;
• publicly disseminate evidence on what works; and
• foster the enabling conditions for replication and scaling.

The Program has demonstrated significant progress in identifying skills and supporting experimentation. It has also effectively established relations with provincial, territorial, and federal partners through the work of the Centre and the Office. With the majority of projects funded by the program ending in fiscal year 2023 to 2024, the Program is encouraged to place greater emphasis on producing a more comprehensive analysis and reporting on lessons learned on what is working and not working to foster the enabling conditions to meet job seekers, workers and employers evolving needs. The Program should also aspire to more cohesive knowledge synthesis and mobilization to maximize insights from evidence generated by the Centre to support replication and scaling of promising practices.

Recommendation #3: Take steps to clarify the role of ESDC and increase its visibility to support efforts to mobilize knowledge and influence labour market policies and programming.

In the implementation of the Future Skills initiative, internal and external engagement and communications activities were mainly carried out by the Future Skills Council and Future Skills Centre. This has resulted in a lack of awareness and clarity about ESDC’s role to fulfill the Program’s objectives with respect to supporting transformative labour market policy and program innovation. As the Program continues to mature and more knowledge is produced, the role and visibility could affect its efforts to influence government policies and programs.

Furthermore, working with the Future Skills Centre to make sure roles and responsibilities are clearly defined can help ensure better support and complementarity of eachothers’ work and minimizes inefficiencies associated with potential duplication.