Summative Evaluation of the Foreign Credential Recognition Program

Official title: Joint Summative Evaluation of the Foreign Credential Recognition Program and the Inter-provincial Labour Mobility Initiative.

Executive Summary

This report presents the findings of the Joint Summative Evaluation of the Foreign Credential Recognition Program (FCRP) and the Interprovincial Labour Mobility Initiative (ILMI) covering five-year period from April 2008 to March 2013. The FCRP invests in regulated and non-regulated occupations to facilitate the foreign credential recognition process and the timely integration of internationally-trained individuals into the Canadian labour market. The ILMI, which ceased in 2013-2014, delivered funding support to facilitate the elimination or reduction of inter-provincial barriers that restrict or impair mobility in Canada. The key questions covered by this evaluation relate to program relevance and performance (including effectiveness and efficiency). Seven technical reports analyzing both quantitative and qualitative data were prepared in support of the evaluation.

Key Findings

Based on the context in which the program operates (i.e., federal/provincial/territorial, regulatory, individual professions and trades, etc.) the evidence gathered for this evaluation provides insights into several aspects of the program and areas in which progress has been made. A complete assessment of all aspects of the program, including an occupation-by-occupation analysis, was beyond the scope.

  • Continued need: There is a strong need to support national and regional organizations to address issues related to foreign credential recognition (FCR) and the evaluation found a continued role for the federal government in FCR-related issues. Barriers in FCR remain strong, and the related needs are diverse and vary among the many stakeholders involved. Barriers to inter-provincial labour mobility: Some barriers still remain. Standards still vary in a number of occupations between some P/Ts on the recognition of education, skills and experience.
  • FCRP’s alignment: FCRP’s function is consistent with both Employment and Social Development Canada (ESDC) and Government of Canada priorities. However, the extent to which the federal government should be active in this area is challenging given that the regulation of occupations is a P/T jurisdiction.
  • FCRP’s outreach: Efforts to raise awareness and build support for FCR issues were described as good although FCRP’s efforts related directly to the Framework were considered insufficient. Stakeholders would like to see continued opportunities for information sharing by, and facilitated through, FCRP.
  • FCRP’s support for horizontal linkages: FCRP supports horizontal linkages through various mechanisms. Those consulted for the evaluation identified the FQRWG as the key mechanism for supporting horizontal linkages in FCR but its achievements were considered by most to have fallen short of expectations.
  • FCRP’s support to develop systemic capacity to implement the Framework: The evaluation found evidence that systemic capacity has been built in many occupations. Progress against the four objectives of the Framework – fairness, transparency, timeliness, and consistency – is mixed with the greatest movement observed in transparency and fairness. However, the level of progress varies by occupation.
  • Availability and use of FCRP-funded tools and processes: FCRP does not have a formal mechanism to share project outputs, lessons learned or best practices. Despite this, the evaluation revealed that FCRP-funded tools and processes have been successfully shared within occupations and, to a lesser extent, between occupations. Increased sharing between occupations could result in more efficient allocation of resources.
  • ILMI objectives: The majority of ILMI projects achieved their objectives, most of which were aimed at understanding/complying with the amendment to Chapter 7 of the Agreement on Internal Trade.
  • Operational costs: Program expenditures have fluctuated for both FCRP and ILMI. Both have also underspent their budgets.
  • Efficiency: FCRP has funded a total of 142 projects over the last five years. Projects tend to be large, with an average project value of almost $600,000. The programs’ investment approach prioritizes projects with target regulated occupations and P/Ts. However, the evaluation found many examples of FCRP-funded projects that did not result in a sustainable solution and the evaluation identified a few instances where there was duplication between these projects.

Recommendations

Based on the evidence gathered in this evaluation it is recommended that the Foreign Credential Recognition Program:

  1. Continue to leverage on coordination and consultation efforts of the FLMM and to fund and hold consultation and coordination activities.
  2. Further improve awareness of funded projects, their outputs and lessons learned from implementation, and establish a mechanism for information sharing.
  3. Further explore the issues and the areas in need of support around interprovincial labour mobility.
  4. Renew and share an FCRP investment strategy and project selection criteria that align with other federal, FLMM and occupational priorities.
  5. Advance performance measurement activities, including collecting and collating performance data and conducting follow-up activities.

Management Response

Introduction

The Skills and Employment Branch (SEB) would like to thank all those who participated in conducting the joint summative evaluation of the Foreign Credential Recognition Program (FCRP) and the Interprovincial Labour Mobility Initiative (ILMI). In particular, SEB acknowledges the contribution of provinces and territories, key informants, and participants who participated in the surveys. SEB agrees with the evaluation findings, and proposes the following Management Response.

The joint summative evaluation of the FCRP and the ILMI covers the five year period from April 2008 to March 2013. While the main focus of the evaluation is a quantitative and qualitative review of the FCRP, it is complemented by a document review of the ILMI. The key questions covered by this evaluation relate to FCRP’s relevance and performance.

Background

In order to find work commensurate with their skills and experience immigrants must undergo a foreign credential (or qualification) recognition (FCR) process. If they want to work in a regulated occupation, they must also obtain a license to practice. This assessment and recognition process serves to verify that the knowledge, skills, work experience and education obtained in another country are comparable to the standards established for Canadian professionals and tradespersons.

In Canada, the regulation of occupations is mainly a provincial and territorial (P/T) responsibility and is often delegated to regulatory bodies. These organizations are primarily mandated through legislation to protect the integrity of professions and to protect the public. Most regulators acknowledge that FCR is an important issue but it is often one of the many priorities that regulators have to address. Given the limited levers available to the federal government and the complex interplay of jurisdictions, meaningful change cannot be achieved through federal efforts alone, but rather requires active and engaged P/T participation. The federal government’s response to these challenges is, in essence, a strategy of incenting change through federal leadership with P/T counterparts and direct financial support with regulators and related stakeholders.

At the federal level, FCR issues are addressed across three portfolios. Employment and Social Development Canada (ESDC) provides expertise on labour market integration, funds stakeholders to improve FCR processes, promotes labour mobility across Canada, and provides labour market information to internationally trained workers and employers. Citizenship and Immigration Canada (CIC) provides internationally trained workers with essential information to improve social and economic integration as well as settlement services, such as language training. Health Canada (HC) works with health stakeholders to address health human resource needs by improving the assessment and recognition of internationally educated health professionals.

At ESDC, the Foreign Credential Recognition Program (FCRP) was launched in 2003 to provide federal leadership and facilitate national coordination among key players on FCR. A key priority of the FCRP is to implement the 2009 Pan-Canadian Framework for the Assessment and Recognition of Foreign Qualifications (the Framework). The Framework, launched by the Forum of Labour Market Ministers (FLMM), is a joint vision for federal, provincial and territorial (F-P/T) governments to work together to ensure that regulatory authorities have FCR processes in place that adhere to the Framework’s principles of fairness, consistency, transparency and timeliness. The Foreign Qualifications Recognition Working Group (FQRWG) supports the FLMM in facilitating the implementation of the Framework through collaboration with stakeholders, including regulatory authorities and national professional associations.

While the principles of the Framework apply to all regulated occupations, F-P/T governments first targeted two sets of regulated occupations (14 occupations in total) as the common focal point for individual and collective actions. On July 11, 2014, the FLMM announced the addition of 10 new target occupations to the Framework, including five regulated occupations and five trades. Governments are working with these additional occupations ensure they meet the Framework’s principles, including the one year commitment to timely service.

In November 2014 the FLMM Ministers endorsed a renewed multilateral approach called An Action Plan for Better Foreign Qualification Recognition that focuses on improved pre-decision supports, including access to the first steps in assessment overseas, clearer communication and transition supports to ensure that the skills and experience of immigrants are fully utilized in the Canadian labour market as well as measuring and monitoring FCR progress regularly.

Through contribution agreements, the FCRP provides financial support to streamline FCR processes for occupations targeted under the Framework as well as other occupations. The FCRP also supports P/T governments in building their FCR capacity. Over the evaluation’s five year period, the FCRP funded a total of 142 projects. These projects have led to significant improvements by making FCR processes more timely, fair, consistent and transparent for internationally trained workers (ITWs), including:

  • Pharmacists and physicians have streamlined the process and making it more fair for internationally-trained candidates by developing pan-Canadian standards and single window application systems (Pharmacists’ Gateway to Canada);
  • Three separate nursing occupations together developed one single access point for international nurses (National Nursing Assessment Service), making it easier for foreign candidates to get information and apply to become a nurse;
  • Architects and dentists have created expedited processes that significantly reduce the cost and the time it takes for foreign trained candidates to get licensed;
  • Engineers can now begin the credential assessment process on-line, as well as undergo a competency based assessment before they arrive in Canada, providing them better expectations on their likelihood to become a licensed engineer in Canada;
  • Medical Laboratory Technologists have extensive alternative career resources to help unsuccessful candidates transition into other related career pathways;
  • Several regulators have developed capacity to assess credentials pre-arrival, resulting in faster FCR processes once in Canada;
  • The FCR Loans Pilot assisted over 1,600 newcomers to obtain financial assistance to get their credentials recognized faster.

This progress has led to streamlined, more timely and fair FCR processes that support newcomers getting jobs that are commensurate with their skills and experience. Furthermore, FCRP activities complement domestic labour mobility initiatives, notably the Agreement on Internal Trade (AIT). The Labour Market Integration Directorate (LMID) is leading work to implement Chapter 7 of the AIT by facilitating national coordination among key players to reduce assessment and recognition barriers faced by internationally trained workers as well as Canadians as they move across P/Ts. While there has been no dedicated funding source to address interprovincial labour mobility since 2013-2014, it remains a departmental priority. Consequently the FCRP considers financial support for FCR projects that also seek to facilitate the elimination of interprovincial barriers that restrict the mobility of workers in Canada.

Key Findings

Overall, the summative evaluation found the FCRP to be effective, relevant, and successful in advancing towards its longer-term outcomes. All professions funded by the FCRP have achieved some level of progress towards the Framework’s principles of timeliness, consistency, fairness and transparency, most notably among high volume occupations. In fact, 91% of funding recipients who responded to the survey indicated that FCRP had enabled their organization to develop systemic capacity to implement the Framework. Progress has also varied among occupations and in certain aspects of the FCR process, from developing language assessment tools to harmonized pre-arrival processes.

Key findings and conclusions complement those of previous FCRP evaluations in that they underscore the importance of a continued role for the Government of Canada through the FCRP to support partners and key stakeholders in improving FCR processes. This demonstrates that the program has laid the foundation for improved FCR processes in regulatory practices, has established key relationships, and has built the trust necessary to work towards common goals.

The evaluation also demonstrates a positive impact of the ILMI and its link in supporting FCR work.

Recommendations

Recommendation 1: Continue to leverage on coordination and consultation efforts of the Forum of Labour Market Ministers and to fund and hold consultation and coordination activities.

For example, over the period examined by the evaluation, the FCRP, in conjunction with the FQRWG, conducted a series of consultations with occupations targeted in the Framework. Each consultation examined in detail all aspects of occupations’ FCR processes, framing the discussions around the long-term vision articulated by the Framework and identified areas where improvements could be made to FCR processes. This in turn set the stage for coordinated action on FCR by regulators, educational institutions and governments.

LMID agrees with this recommendation. Addressing FCR issues is a shared FPT responsibility that can only be achieved through sustained and coordinated dialogue. LMID has a long standing commitment to share information and coordinate FCR activities with multiple levels of government and stakeholders. For example, over the period examined by the evaluation, the FCRP, in conjunction with the FQRWG, conducted a series of consultations with occupations targeted in the Framework. Each consultation examined in detail all aspects of occupations’ FCR processes, framing the discussions around the long-term vision articulated by the Framework and identified areas where improvements could be made to FCR processes. This in turn set the stage for coordinated action on FCR by regulators, educational institutions and governments.

For example, in January 2014, the FQRWG published a report on Alternative Careers to contribute to a better understanding of alternative careers for internationally trained workers (ITWs), to analyze the structure and delivery of effective alternative career supports, and to make recommendations on next steps. The release of this report confirmed FCRP’s investment approach to supporting alternative careers projects.

In October of 2014, ESDC announced the Panel on Employment Challenges of New Canadians to identify and report on innovative approaches as well as challenges related to the licensing, hiring and integration of immigrants. LMID supported the work of the Panel, notably during the consultation process when the Panel met with a broad range of stakeholders, such as employers, immigration serving organizations and academic institutions. The Panel also conducted an online survey open to all Canadians. The Panel published its report in April 2015 and its work helped to continue the dialogue on FCR with key stakeholders, beyond those engaged in the Framework consultations.

Moving forward, LMID will continue working closely with key stakeholders to support dialogue and joint efforts that address current and emerging FCR issues. This includes:

  • Developing a more proactive outreach strategy to raise awareness of FCRP activities and to enhance the sharing of information and progress of FCRP activities.
  • Exploring ways to regularly bring the conversation on FCR issues and related investments to the senior level within governments to ensure a common vision.
  • Using the Panel’s findings and conclusions, as well as other key sources of information, to help further support, facilitate and coordinate FCR-related dialogue and activities among key stakeholders.

Recommendation 2: Further improve awareness of funded projects, their outputs and lessons learned from implementation, and establish a mechanism for information sharing.

LMID agrees with the recommendation. Better sharing of project results, lessons learned and promising practices across occupations, sectors and jurisdictions will contribute to the development and delivery of more effective projects.

To further share information and coordinate FCR activities, LMID regularly dedicates resources, time and effort to closely liaise with P/T governments, regulatory bodies, non-governmental organizations, employer associations, immigrant serving organizations and international forums. For example, LMID requires that each of its FCRP project recipients submit a communication and dissemination plan that identifies the target group, mechanisms for sharing project information and results with the target group and the greater stakeholder community (e.g. newsletters, meetings, websites, mail outs). LMID also shares information about FCRP P/T projects regularly at FPT meetings.

Furthermore, since 2013, LMID has held several workshops with key stakeholders to share FCR practices and highlight future federal directions. Most recently, this included workshops on Examinations in the FCR Process (February 2015) and also on Pan-Canadian Systems and Mutual Recognition Agreements (May 2014). LMID funded events are also held to help stakeholders better understand key challenges and opportunities related to FCR. In 2012, FCRP funded the Canadian Foundation for Economic Education to organize a conference on Mutual Recognition Agreements (MRAs). Following the event, the Royal Architectural Institute of Canada applied to FCRP for a project aiming to develop MRAs with countries in Europe, Asia, Eurasia, and beyond. This investment allows ongoing MRA negotiations with European architectural counterparts and has already generated new agreements with New Zealand.

LMID also regularly uses opportunities such as the annual meeting of the Canadian Network of National Associations of Regulators (CNNAR) to share information and best practices among relevant stakeholders on FCRP funded initiatives. For example, the Canadian Association of Medical Radiation Technologists presented to the audience of CNNAR in 2014 on their efforts to develop harmonized standards for the assessment of internationally trained medical radiation technologists, a project funded by the FCRP.

While the report flags overlap as a potential issue, the three federal departments (ESDC, CIC and HC) that address FCR issues have been taking measures, and will continue to do so, to prevent potential duplication of FCR related activities. For example, LMID regularly meets with its federal partners to ensure FCR related activities complement and support one another. FCR is also a regular agenda item at the ESDC-CIC Assistant Deputy Minister Roundtable. Furthermore, CIC and HC are active members on the FQRWG and LMID participates on their respective FPT tables (CIC’s Immigration Table and HC’s Committee on Healthcare Workforce).

Moving forward, LMID will continue to build on existing communication mechanisms and strategies to support awareness and information sharing efforts that advance the FCR agenda. This includes:

  • Organizing an annual workshop on FCR with key stakeholders that demonstrate progress towards achieving the principles of the Framework, and reiterate FCRP’s mandate and funding opportunities.
  • Continuing to work with P/Ts through the FQRWG and FCRP funding agreements to share approaches, achievements and challenges across P/Ts.
  • Enhancing its web presence and ensuring that information regarding the program is clear, concise, current, and accessible.
  • Applying FCR promising practices in the program’s future consultations with occupations.
  • Creating a publicly available project database that will include timely and accurate project information.

Recommendation 3: Further explore the issues and the areas in need of support around interprovincial labour mobility.

LMID agrees with the recommendation. Seamless mobility is integral to an effective FCR process, as it encourages the development of key instruments (e.g. national competency profiles, indicators and performance benchmarks) to assess both domestic and internationally trained workers.

LMID recognizes that there is a need for the federal government to continue taking actions to address barriers to labour mobility and that the FCRP is well positioned to continue assuming this role. The FCRP’s current funding strategy has been successful in addressing labour mobility issues. For example, a project funded by FCRP to the Canadian Alliance of Audiology and Speech-Language Pathology Regulators was successful in helping to address barriers to workers mobility across the country by removing some of the variances in the application process, and allowed the profession to begin work on harmonization and standardization of the tools that will be used to assess both Canadian and internationally trained applicants.

Moving forward, LMID will continue to address interprovincial labour mobility issues. This includes:

  • Continuing support for foundational projects, including labour mobility projects that seek to remove mobility barriers for workers in regulated occupations.
  • Reviewing FCRP’s terms and conditions to better reflect the importance of addressing labour mobility barriers.

Recommendation 4: Renew and share an FCRP investment strategy and project selection criteria that align with other federal, Forum of Labour Market Ministers and occupational priorities.

LMID agrees with the recommendation. Since its inception, the FCRP has worked closely with occupations to develop diagnostiques and projects that seek to improve FCR tools and processes. The FCRP has also worked closely with P/T governments to define priorities and identify key occupations in need of support.

Moving forward, supporting continuous enhancements to the program's investment strategy will help achieve its intermediate and long-term goals. This includes:

  • Reviewing, updating and sharing the FCRP investment strategy and its selection criteria to ensure that it remains aligned with the renewed federal priority on FCR and labour mobility as well as with the priorities of P/T governments through the FLMM and occupations.
  • Ensuring that subsequent projects within the same occupation will build further on past successes, practices and lessons learned and will be linked with an overall strategic vision to increase the likelihood of meeting the program’s medium and long-term outcomes.
  • Ensuring that FLMM Senior Officials have an opportunity to discuss FCRP and P/T priorities at their regular meetings. This will ensure that P/T projects align with the joint vision on FCR and raise awareness of the Framework among relevant stakeholders so that FCRP funding recipients share information on their success and results.

Recommendation 5: Advance performance measurement activities, including collecting and collating performance data and conducting follow-up activities.

LMID agrees with the recommendation. FCRP’s performance measurement strategy is currently being updated and there are opportunities to strengthen the plan for the collection and use of performance data.

Since 2012-2013, LMID has mostly relied on saved project records and the Common System for Grants and Contributions for results tracking. ESDC has had limited success in collecting data about individuals from its key stakeholders, such as regulators, as they generally have access to limited data that is often not comparable among occupations, making it difficult to track individuals’ progress throughout the licensing pathway.

LMID would benefit from more refined results at a Pan-Canadian level and the FLMM’s new F-P/T Action Plan identified a commitment to strengthen the monitoring and evaluation strategy. Through this work, the F-P/T governments have committed to working with regulators and other stakeholders to define measurement criteria, monitor FCR progress over time and demonstrate progress in FCR pathways. However, more comprehensive results on newcomers’ labour market integration outcomes may remain very difficult to gather, given the limited ways to track immigrant successes.

To further strengthen the collection and use of performance data, LMID and POB will work closely to ensure that the department is adequately measuring both project results and program objectives and is better at collecting and aggregating information related to project successes.

Conclusion

The findings from the summative evaluation are positive and demonstrate FCRP’s relevance and performance in achieving its outcomes. The program will apply these findings to improve its current and future approaches to the assessment and recognition of foreign credentials. Since 2003, the activities of the FCRP have evolved and will continue to do so as the program seeks to further improve FCR. While F-P/T support for the Framework demonstrates the advantage of a pan-Canadian approach to FCR, it also highlights the challenges of implementing, monitoring and reporting on joint commitments and desired outcomes. The Government of Canada will continue to play an important leadership role to address FCR and labour mobility issues and the FCRP will continue to be a key contributor to these efforts. Moving forward, FCRP will work to ensure that its programming is responsive to the needs of its stakeholders, and to new and emerging labour market issues impeding immigrants from accessing jobs commensurated with their skills and experience.

1. Introduction

This report presents key findings and recommendations resulting from the Joint Summative Evaluation of the Foreign Credential Recognition Program (FCRP) and the Interprovincial Labour Mobility Initiative (ILMI). The evaluation which was identified in Employment and Social Development Canada’s (ESDC’s) five-year evaluation plan, approved by the Departmental Evaluation Committee in July 2013, covers a five-year period from April 2008 to March 2013. At that time, it was decided that the main focus of the evaluation should be the FCRP, complemented by a document review of the ILMI.

Undertaken as a requirement of the Financial Administration Act and pursuant to the Treasury Board Policy on Evaluation requirements, this evaluation assesses issues related to program relevance and performance (including effectiveness and efficiency).

It should be noted that FCRP has been previously evaluated. The report, released in 2010, considered the performance of the program between 2004-2005 and 2007-2008.

Overview of Foreign Credential Recognition Program

Foreign credential or qualification recognition is the process of verifying that the knowledge, skills, work experience and education obtained in another country are comparable to the standards established for Canadian professionals and tradespersons Footnote 1 . The stated objectives of FCRP is to provide federal leadership and promote national coordination among key players to facilitate foreign credential recognition by ensuring that professionals and tradespersons who have obtained their credentials in another country are better integrated into the labour market and can fully use their talents and skills.

Since 2003, FCRP has invested in regulated and non-regulated occupations to facilitate the foreign credential recognition process and the timely integration of internationally-trained individuals in the Canadian labour market.

Through contribution agreements, FCRP seeks to provide strategic financial support to organizations and regulatory authorities in streamlining credential recognition processes of key in-demand occupations targeted under the Framework, as well as other occupations. FCRP also supports provincial and territorial (P/T) governments in building their foreign credential recognition capacity.

The regulation of occupations is a responsibility that is delegated from Provincial and Territorial (P/T) governments to regulatory bodies for each of the occupation. The process for the recognition of credentials in Canada is complex, with nearly 500 regulatory bodies in Canada, at least five recognized credential assessment agencies, numerous professional associations, post-secondary and vocational institutions, and employers, throughout 13 jurisdictions, all of which are involved in various aspects of credential assessment. Given this complexity, the FCRP is interacting with a multitude of players that may have differing priorities, capacities and engagement.

FCRP is delivered by the Labour Market Integration Directorate of the Skills and Employment Branch of ESDC. Approved funding for FCRP for 2009-2010 to 2013-2014 was $49.5 million. The program received additional funding through the 2009 Budget in the amount of $94.75 million for 2010-2011 to 2014-2015. The table below presents FCRP’s planned funding, by fiscal year.

FCRP Budgetary Financial Resources (Planned - $ millions)

  • 2009-2010: 26.4 million
  • 2010-2011: 29.8 million
  • 2011-2012: 29.4 million
  • 2012-2013: 26.9 million
  • 2013-2014: 26.9 million
  • Total: 139.5 million

According to the program’s logic model (Annex A), one of the FCRP’s outputs is the promotion of the Pan-Canadian Framework for the Assessment and Recognition of Foreign Qualifications (the Framework). The Framework, launched by the Forum of Labour Market Ministers (FLMM) in 2009, is a joint vision for Federal and Provincial/Territorial (F-P/T) governments to work together to ensure that regulatory authorities have foreign credential recognition processes in place that adhere to the principles of fairness, consistency, transparency and timeliness. The Framework, while not being a legally binding instrument, is a key commitment by all governments to take action on FCR.

While the principles of the Framework apply to all regulated occupations, 14 regulated occupations were initially targeted to serve as the common focal point for individual and collective actions. An ESDC official is the co-chair of the Foreign Qualifications Recognition Working Group (FQRWG), an entity that reports to the FLMM. The FQRWG is mandated to advance progress on the Framework.

Overview of Interprovincial Labour Mobility Initiative

Barriers to labour mobility prevent labour reallocation across sectors and regions, thus preventing labour to flow from regions where it is in over supply to others facing shortages. This results in efficiency costs for the economy. Some of these costs are incurred in the short run, while others are more in the long term. The consequences for regions where labour is in over supply are higher unemployment, lower wages, and a production sector operating below its potential. In the short run, this results in lower labour earnings (potentially causing more reliance on the social security system), falling consumption and investment spending, and an overall decline in the population welfare.

In the long run, unemployment results in losses of skills for laid-off workers, and a decline of the average productivity of the labour force (which may require upskilling and training programmes). For regions facing shortages, the opposite phenomena are observed: higher wages raise labour costs, which adversely affect the competitiveness of the economy, keep the production sector operating below its potential, while more resources are spent on searching for workers and in hiring. In the long run, limited supply of labour depresses investments in capital and high demand for workers and rising wages discourage education and investment in human capital and skill development.

Canadian First Ministers signed the Agreement on Internal Trade (AIT) in 1994, to ease the inter-provincial movement of goods and labour. Chapter 7 of the AIT is specifically designed to eliminate or reduce inter-provincial barriers that restrict or impair labour mobility in Canada, in particular the movement of certified workers in regulated professions, including the skilled trades.

Chapter 7 was amended in 2009 to allow certificate recognition. This recognition has compelled regulators to recognize the due diligence applied by other P/T regulators in determining that a worker is competent to practice without additional material requirements. To facilitate the elimination or reduction of inter-provincial barriers that restrict or impair mobility in Canada, the ILMI delivered funding support to regulatory bodies and national associations of regulators through contribution agreements. Funding for the ILMI ceased at the end of 2013-2014. ILMI initiatives that include an FQR component can now be considered for funding under the FCRP (EI part II).

The ILMI was delivered by the Labour Market Integration Directorate of the Skills and Employment Branch of ESDC. ESDC began funding occupational organizations as early as 1996-1997 to address labour mobility barriers. Annual funding of this initiative for the 2008-2009 to 2012-2013 period varied from a low of $1.2 million in 2009-2010 to a high of $2.7 million in 2012-2013 and originated from the funding envelope under the Employment Insurance Part II.

Evaluation

The joint evaluation was identified in ESDC’s five-year evaluation plan, approved by the Departmental Evaluation Committee in July 2013. The main focus of the evaluation is FCRP, but it is complemented by a review of the ILMI. With emphasis on FCRP, the evaluation assessed the program’s medium and long-term outcomes as well as the extent to which it has developed systemic capacity among the main FCR players. Based on the evaluation objectives and three broad issues, as listed below, areas consistent with the 2009 Treasury Board Policy on Evaluation, ten specific evaluation questions with corresponding indicators were developed.

Relevance

  1. To what extent: a) is there a continued demonstrated need for FCRP and b) is FCRP clearly distinctive from other federal programs/initiatives (such as the Foreign Credential Referral Office) and P/T programs?
  2. To what extent are there barriers to inter-provincial labour mobility?
  3. To what extent are the objectives of FCRP consistent with government and departmental priorities as well as federal roles and responsibilities?

Performance (Effectiveness)

  1. To what extent has FCRP conducted its outreach activities with stakeholder groups, particularly regulatory authorities and professional organizations, to raise awareness about the Framework and build support for its implementation?
  2. To what extent has FCRP supported horizontal linkages through leadership in F-P/T fora, such as the Forum of Labour Market Ministers’ Foreign Qualification Recognition Working Group and its task groups and through coordination within the federal system?
  3. To what extent has FCRP supported provinces, territories and key stakeholders to develop systemic capacity to implement the Framework across occupations?
  4. To what extent have Pan-Canadian FQR processes and tools in targeted occupations and sectors been made available and used as a result of FCRP efforts?
  5. To what extent did ILMI projects achieve their objectives?

Performance (Efficiency and Economy)

  1. What are FCRP/ILMI’s operational costs vis-à-vis total program funding?
  2. To what extent are FCRP/ILMI funds delivered efficiently and effectively? Do alternative methods of funding delivery exist?

The evaluation was designed to gather information on each of the evaluation issues using a multi-method approach. Where possible, there is a balance between quantitative and qualitative methods, with qualitative methods providing further description and explanation for the quantitative information. The seven evaluation methods implemented are presented in Annex B. Details on the evidence obtained from the evaluation methodology are available in separate Technical Reports. Annex C provides the list of reports available.

2. Key Findings – Effectiveness

This section presents two key findings related to effectiveness, covering how well FCRP has increased the awareness about FCR issues and the Framework and the extent to which FCRP funding has helped develop systemic capacity among the main FCR players. This section also includes a brief description on horizontal linkages and the state of FCRP performance measurement.

Awareness about FCR Issues and the Framework

FCRP has continued to raise awareness, knowledge and sensitivity of FCR issues in Canada and has articulated the need for change through the funding of projects and participation in the FQRWG. Outreach to raise awareness of the Framework was considered by funded recipients to be adequate; however, external key informants characterized FCRP outreach overall as limited.

The evidence from documents and interviews illustrates that FCRP has been active, alongside the FQRWG, in promoting the Framework among stakeholders and strengthening partnerships with a broad range of stakeholder groups including but not limited to the regulatory communities and professional organizations. Most of those interviewed gave FCRP credit for raising awareness about challenges in FCR and focusing attention on the matter. This is consistent with the previous evaluation where FCRP was also found to increase understanding of FCR-related issues among stakeholders and partners as a result of funding national-level diagnostiques, contributing to organizing national events and stimulating dialogue with multiple levels of government and stakeholders.

At the project level, stakeholders have been actively engaged in outreach activities. More than two out of three funded projects included outreach activities such as consultation meetings with stakeholders, stakeholders’ involvement in piloting tools, and developing processes. A wide range of stakeholder groups took part in these activities including industry representatives, regulatory authorities, and professional organizations.

At the program level, documents reviewed for the evaluation illustrated that FCRP is engaged in a range of outreach activities (e.g. conferences and workshops, FQRWG-related committee and working group meetings, and roundtables). Evidence of conferences funded by FCRP during the five-year period of the evaluation, included two hosted by the Canadian Foundation for Economic Education (in 2009 and 2012). Also, FCRP representatives regularly attend conferences and workshops hosted by Employment and Social Development (ESDC) and the FQRWG as well as high profile events. The FQRWG also organized roundtables with 14 occupations in 2010 and 2011 to develop occupational action plans as well as a roundtable with regulators to follow-up on engagement activities in March 2013.

The majority of surveyed funding recipients (57% or 17 out of 30 who answered the question) indicated that FCRP’s outreach efforts to raise awareness about the Framework were adequate and another 13% (4 out of 30) indicated they were excellent. Almost a third (30%, 9 out of 30) indicated they were limited.

Some key informants were able to mention conferences, roundtables and working groups which supported horizontal linkages, although most of the external interviewees were unclear about who sponsored the events. Their assessments of these fora were generally positive, and they welcomed opportunities to meet with other stakeholders and discuss areas of common interest. When asked specifically about outreach conducted by the FCRP, most respondents from all external respondent groups indicated it had been limited.

While most external interview respondents, case study interviewees and surveyed funding recipients were generally familiar with the Framework, their support for it could be characterized as modest. For example, when asked to describe the extent to which the Framework has helped improve the integration of internationally trained individuals into the Canadian labour market, 40% of those who responded (12 out of 30) indicated the Framework had done a good or excellent job and 40% were neutral. Another 17% (5) indicated it had done a fair job and only one person thought it had done a poor job.

Key informants describe the Framework as overly general and lacking in authority, and felt that assessing achievements of the Framework requires better data on outcomes for internationally trained individuals (ITI)s. Case study respondents reported it to be helpful in areas such as generating awareness and spurring collaboration, but also had criticisms. In particular, they were concerned that it raised expectations, was not well understood, and lacked accountability.

Development of Systemic Capacity to Implement the Framework

One of the key expected outcomes of FCRP is the development of systemic capacity to address FCR issues. The evidence confirms that the program has funded organizations to increase their knowledge, skills and abilities to address FCR issues. Moreover, systemic capacity has been built to increase the transparency related to FCR information and processes. Other objectives have seen more modest progress, including the issues of fairness, timeliness and consistency.

Over the ten-year timeframe of the program, FCRP’s focus has evolved from the funding of awareness-raising projects and events, to the development of standardized tools and processes, to the funding of projects that aim to develop systemic capacity of organizations to implement the Framework according to its four objectives: the improvement of the transparency, fairness, timeliness and consistency of processes.

Documents produced throughout the timeframe of the evaluation confirm that implementing the four objectives of the Framework has been a priority for FCRP since the Framework was launched in 2009. Footnote 2 At the project level, more than 80% of project descriptions either explicitly mentioned the Framework in their objectives or had objectives that aligned with one of the four objectives of the Framework. Progress against each objective is described below, although the evaluation also looked at progress against the extent to which FCRP has helped to develop systemic capacity among funding recipients.

Systemic Capacity

Funding recipients who responded to the survey indicated that FCRP had enabled their organization to develop systemic capacity to implement the Framework; 69% (22 out of 32 respondents) indicated it had to some/great extent, and another 22% (7 of 32 respondents) indicated it had to a little extent. Responses were similar when funding recipients were asked about the extent to which FCRP had enabled P/Ts and other stakeholders to develop systemic capacity: 62% (16 of 26 respondents for P/Ts and 11 of 18 for other stakeholders) said it had to some/great extent and 31% (8 of 26 respondents) and said a little for P/Ts and 28% (5 of 18 respondents) said a little for stakeholders.

The file review shows that the eleven FCRP projects (where a P/T is the contribution agreement holder) contributed to FCRP’s intended outcomes in various ways Note that seven different P/Ts were contribution agreement holders. Specifically, 82% (9 of 11 of the projects related to the increased capacity of stakeholders to address FCR issues and to better integrate ITIs into the labour market, 55% (or 6 projects) increased FCR recognition and labour market integration of skilled immigrants, 55% (6 projects) increased understanding, consensus, collaboration and commitment among stakeholders and partners on issues and potential solutions related to FCR, and 55% (6 projects) improved labour market outcomes for ITIs.

The interview evidence suggests that FCRP’s success in developing systemic capacity to implement the Framework has been limited. Knowledge, skills and abilities to address FCR are thought to have been strengthened, with variations between occupations and jurisdictions. While external key informants indicated that progress has been made since 2008, progress is considered moderate. According to internal ESDC respondents, small jurisdictions with limited capacity have been able to develop FCR programming as a result of FCRP.

Transparency

The findings from the previous evaluation showed that progress had been made, although at the time, transparency and access to information were considered longer term outcomes and the extent to which FCRP could directly influence transparency was not clear. While survey respondents for the current evaluation did not attribute these results to FCRP, since 2008, 92% (34 out of 37) of funding recipients surveyed believe that ITIs have more access to relevant information and appropriate services and 90% (17 out of 19 respondents from regulated occupations only) believe FCR processes have become more transparent.

The case studies conducted for the current evaluation provide evidence of increased transparency and attribution of this improvement to FCRP. According to the case studies, transparency is the area under the Framework in which the greatest strides have been made in the five-year timeframe of this evaluation. The majority of occupations report a larger quantity and better quality of information; tools and processes are reported to be more widely understood by ITIs. External respondents spoke to the need to continue to improve communication and provide transparent information to ITIs, particularly at the pre-arrival stage.

Fairness

As with the case for transparency, the previous evaluation suggested some progress but was less certain about FCRP’s direct contribution. According to survey respondents for the current evaluation, without attributing the results to FCRP, about three quarters, or 78% (29 of 37) of survey respondents believe that the FCR process has become more fair and equitable for ITIs.

When looking at the occupations covered through the case studies, fairness is an area where progress is slow. However, the FCR processes became fairer for the nursing profession through the development of a single-window process. Also, the medical radiation technologist and ICT professions have had successes attributable to improved processes funded by FCRP.

Timeliness

Evidence from the review of documents indicated that progress has been reported from the occupation stakeholders toward the achievement of the timeliness objectives. The FLMM Progress Report 2013 “Timely service: An Important Benchmark” indicates that:

“all of these occupations have confirmed that they meet the Framework commitment to timely service (p.5). This means that regulators in these occupations, upon receipt of a complete application, are able to review the qualifications of internationally trained applicants and, within one year, provide a decision as to whether they meet the standards for registration, if they need to meet additional requirements, or if their qualifications are better suited to an alternative occupation that is more closely linked to their skills. Many regulators have reported they are able to communicate decisions in a matter of weeks”

While interviewees were not asked to comment on timeliness specifically, among most external interview respondents, time and cost are still, since the last evaluation, cited as key barriers to ITIs in the recognition of foreign credentials. Almost none of the occupations included as case studies are assessing and recognizing credentials faster than before, with FCR processes remaining multi-year endeavours in many cases (although progress in a few of the occupations was such that it was too early to determine the ultimate impact on timeliness). The occupations that have faster processes include ICT and the non-FCRP funded occupation, veterinarians.

The case studies found that a few funded occupations may actually be affecting the length of processes, by adding or planning to add pre-qualifying tests. That said, increasingly the regulated professions are shifting parts of their assessment processes to the pre-arrival stage. This may potentially help ITIs spend a reduced amount of time in Canada underemployed or unemployed.

Consistency of Processes

The file review revealed that standardized FCR processes have been implemented in a variety of regulated occupations: 27 occupations (including all 14 target occupations identified in the Framework) have been funded by FCRP to standardize their FCR processes and 21 of these occupations (including 13 of the 14 target occupations) reported making progress towards this objective.

External interview respondents noted that similar or even standardized processes and tools are being used in a few occupations, these respondents indicated that they usually do not cover all aspects or steps of the foreign credential assessment process. There are significant challenges with applying a single system or process to an occupation that is governed by a national body and 13 different P/T regulators. While most occupations are working towards developing a single system or process, no occupation to date has been able to achieve this goal.

This was confirmed in case studies. Of the five regulated occupations considered, three have standardized processes or tools (medical radiation technologists, nurses, physicians), but these are partial as they do not cover all jurisdictions or all parts of the FCR process. A notable achievement is that the medical radiation technologist profession has developed a national standard for the assessment of credentials, which is used across all jurisdictions in Canada. Nursing professions have harmonized the data collection and process for the first four (of ten) steps in the licensure process. Physicians have created a centralized application process, though legislative barriers have prevented further harmonization across jurisdictions.

Evidence shows that FCRP funding has enabled all of these successes. The previous evaluation of FCRP also explored the degree of standardization (and the extent to which FCR-related tools/processes had been developed and disseminated) in case study occupations. Some progress was found in two of the case study occupations (engineers and physicians), noting that these had received funding for multiple projects since the beginning of FCRP.

Though ICT is an unregulated occupation and the barriers to ITIs are fewer, it has also made big improvements in standardization since 2008, with the implementation of a nationally recognized competency-based assessment and recognition tool. A few professions, namely veterinarians, biotechnologists, and medical laboratory technologists, already had harmonized standards in place prior to FCRP funding.

According to the survey of funding recipients, 68% (13 of 19) believe the FCR process for their regulated occupation has become more simplified (particularly the assessment and application processes, but less so the registration process or testing). As well, 78% (21 of 27) of respondents from regulated occupations said that regulatory bodies for other P/Ts are more likely to use similar FCR processes while 19% (5 of 27) said that there has not been any change since 2008. However, among respondents from regulated occupations, only 11% (2 of 18) respondents said that all jurisdictions use the same tools and processes and another 44% (8 of 18) said that between 7 and 12 jurisdictions use the same processes and tools. A full 39% (7 of 18) indicated that between 0 and 3 jurisdictions use the same processes and tools.

Interviewees also provided a few examples of projects that were funded by FCRP that were not successful, either because there was no realistic plan for the sustainability of the solution being developed or because the necessary players and their buy-in were not appropriately engaged in the FCRP-funded project.

Support of Horizontal Linkages and Leadership through F-P/T Fora

FCRP supports horizontal linkages through various mechanisms, although the most commonly cited mechanism was the FQRWG (of which FCRP is only one among many other federal and P/T representatives). While the FQRWG has hosted various types F-P/T fora, it was also criticized by most external federal and P/T interview respondents has falling short of expectations, and would gain from additional coordination mechanisms and better horizontal linkages among F-P/T governments.

Many of FCRP’s intended activities and outputs focus on engaging stakeholders and P/Ts and thus supporting horizontal linkages and other partnerships in the area of FCR. Specifically, the evaluation sought to understand how this objective was achieved through the program’s leadership in F-P/T fora.

While the program interacts with P/Ts in many ways (including national level conferences hosted by FCRP and others, as well as through bilateral meetings), the most commonly cited F-P/T fora where FCRP is seen to have a presence, according to the program documents and interviewees, is the FQRWG. It is important to emphasize that the evaluation of the FCRP is not an evaluation of the FQRWG; however, it is the most visible mechanism through which the FCRP supports horizontal linkages in the area of FCR.

According to the FQRWG Terms of Reference, the ad hoc working group operates collaboratively to guide and support implementation of the Framework within and across different orders of government. The FQRWG includes a federal co-chair from ESDC as well as a rotating P/T co-chair. Membership includes federal representatives from ESDC, Citizenship and Immigration Canada (CIC) and Health Canada and P/T representatives chosen by each jurisdiction from among labour market, immigration, education and health portfolios.

A large majority of the federal department representatives and P/T representatives interviewed for the evaluation identified the FQRWG as the main mechanism for supporting horizontal linkages in the area of FCR. The evaluation found evidence of a number of roundtables, conferences and workshops hosted by the FQRWG that served to supported horizontal linkages. Despite this, the FQRWG was otherwise roundly criticized by most key informants from federal and P/T governments. In general, it was seen to have fallen short of expectations for F-P/T coordination and would gain from additional coordination mechanisms and better horizontal linkages among F-P/T governments. However, the federal government is only one member of the FQRWG and should not be held accountable for its successes or limitations.

Adequacy of Monitoring/Performance Measurement

Data and/or aggregate reports to assess the results of funded projects were not available. Data in the Common Systems for Grants and Contributions (CSGC) did not include details linking project results with the expected outcomes of the program. FCRP’s performance measurement strategy is inconsistent with more recent program documentation, includes unclear terminology and would benefit from improved/additional data sources.

The document review did not identify a performance report or other such documentation that reported on the effectiveness of FCRP investments. As well, there was a lack of available outcome data for projects. The evaluation did not have access to any kind of aggregate information pertaining to project successes or results. Final reports were available for some projects in electronic files, but these were not linked with the expected outcomes of the program. The data in the CSGC did include a field to indicate whether each closed project’s objectives were met, but no details were provided regarding what the objectives were for those projects or how they were linked with the logic model for the program.

Evidence indicates that FCRP’s Performance Measurement Strategy (developed in 2011-2012) would benefit from an update. In the planning phase of the evaluation, it was found that the logic model was not a current representation of the program’s objectives. As well, there is a lack of clarity around some of the terminology in the Strategy (such as the concepts of standardization versus harmonization). Finally, it is the opinion of the evaluator that the indicators and data sources should be updated to identify more appropriate data sources (i.e., using close-out and final reports rather than contribution agreements and ensuring data is available in project/stakeholder databases).

3. Key Findings – Relevance

Continued Need

There is a strong need for supporting national and regional organizations to address issues related to FCR. However, FCR-related needs and barriers vary by stakeholder group and are diverse and evolving.

Recent statistical studies Footnote 3 find that immigrants experience a disadvantage in regards to employment and earnings as compared to native-born Canadians. This is the case particularly for immigrants with a foreign education specifically those immigrants who obtained their education from certain regions of the world (i.e. Central Africa, East Asia, North Africa, Middle East and East Africa). Footnote 4 Although non-recognition of foreign qualifications is likely one cause of the gap in economic performance, its direct effect has proven difficult to establish. To the extent that non-recognition is a factor that contributes to the gap, measures that result in an improvement in the recognition of foreign qualifications should decrease it. But since the gap has other causes (e.g. low language skills, ethnic discrimination), credential recognition alone may not eliminate it.

At a cost of up to $5 billion a year, underemployment among immigrant workers weighs heavily on the Canadian economy. Footnote 5 Further need for FCRP is indicated by increasing demographic pressures on the labour market more generally, including the need to replace the growing number of retiring professionals and skills shortages in certain occupation and sectors in the coming years.

Those consulted for the FCRP/ILMI evaluation confirmed that a need remains to address FCR issues and reduce FCR barriers in Canada. When asked to identify the barriers (or areas of need pertaining to FCR) that the various players (including ITIs, regulatory bodies, employers) continue to face, all respondents offered reasons for the barriers, but there was little consensus among the responses.

ITIs face multiple barriers when attempting to have their credentials recognized. Studies reviewed for the evaluation demonstrate that immigrants perceive the FCR system to be inadequate, citing a lack of clear information about the process, delays by regulating authorities, a disconnect between the point-based immigration system and recognition by regulators and employers, and a lack of opportunities to gain Canadian work experience or practical training. Many of those consulted for these studies also mentioned financial constraints, which contrasts statistical evidence that suggests that only 3.7% of immigrants were prevented from getting their credentials assessed due to financial problems. Footnote 6 These studies generally argue that better information and a more transparent FCR process should reduce these concerns among newcomers.

Those interviewed for the evaluation suggested that ITIs face differing P/T regulations, high FCR-related expenses, delays in receiving documentation from their home country, delays in having their applications processed in Canada, and an overall complex system that usually differs by jurisdiction, occupation and, for non-regulated occupations, by employer.

There is some evidence indicating that barriers to credential recognition increase the likelihood that immigrants engage in post-migration education, thereby delaying their entry into the labour force. Footnote 7 Thus, appropriate recognition of ITIs’ credentials should reduce the need to undertake unnecessary additional training and better address the anticipated labour shortages in many occupations. A few respondents interviewed for the evaluation suggested that there is an evolving need for online training to be made available for ITIs to complete before they arrive in Canada. Some external interview respondents also suggested that ITIs would benefit from training in Canadian workplace language and/or workplace culture.

According to evaluation interview respondents, the main barrier to FCR, particularly for non-regulated occupations where employers are responsible for assessing foreign qualifications and experience, is a lack of understanding and/or capacity among employers to properly interpret an ITI’s experience and other qualifications. Most respondents to the evaluation who work in organizations associated with non-regulated occupations noted that a key barrier was the absence of a coordinated approach in dealing with FCR challenges.

Regulatory body representatives interviewed for the evaluation reported they continue to want opportunities for information sharing, learning and best practices as well as assistance to facilitate national solutions, all of which are aligned with FCRP’s objectives. Some representatives from government organizations suggested that regulatory bodies also need funding to address FCR-related issues. A few P/T interviewees also cited a lack of knowledge and information on best practices in assessing credentials as barriers to FCR among regulatory bodies.

Despite progress, all occupations covered in the case studies identified similar barriers as in the previous evaluation. The need for harmonized and standardized processes for the assessment of work experience, cultural competencies or soft skills, were identified as among the strongest.

When considering these various barriers and needs, it is clear that most of the needs closely correspond to the main objectives of FCRP indicating a continued need for a program such as FCRP.

However, some key informants believe that some priority areas for FCR have not been strongly addressed by FCRP in the five-year timeframe covered by this evaluation. These areas include understanding and addressing the needs of employers (especially in unregulated occupations), having more information, assessment and enabling skills development and bridging at the pre-arrival stage. It should be noted that some of these areas, such as providing information to immigrants abroad, is the responsibility of CIC. The persistent disconnect between qualifying for immigration (as a skilled worker) and qualifying in one’s occupation once in Canada remains a key barrier, as does the general complexity of FCR processes in Canada.

Federal Interest

Distinctive programs

There are several existing federal programs and initiatives that address similar challenges as FCRP. The evidence suggests that most of these other programs complement the work of FCRP (such as the Canadian Immigration Integration Program and the Internationally Trained Workers Initiative both funded by CIC). However, some run the risk of duplicating the work of FCRP (including the Foreign Credential Referral Office (FCRO) at CIC and the Internationally Educated Health Professional Initiative (IEHPI) at Health Canada). Key informants also recognized the potential for overlap with these programs and other government departments, P/T respondents in particular, suggested that there could be a stronger distinction between these three programs.

The possible overlap with FCRO is most likely in the production and distribution of information on credential assessment, licensing, and registration. The most recent FCRO evaluation found that key informants perceive that FCRO and FCRP fund similar projects. Footnote 8

The possible overlap with the IEHPI is in both capacity development and information. The Initiative not only focuses on occupations that are also targeted by FCRP, it also has similar goals (i.e., standardization of testing and registration procedures and the implementation of the Framework). Like FCRP, IEHPI makes funding available for provincial governments to carry out FCR-related projects. Thus, overlap between the two programs can occur if they fund similar projects at the provincial level.

The extent to which these federal programs overlap or complement each other depends on the degree of coordination among them. According to both this and the previous FCRP evaluation, representatives of the three programs (i.e. FCRP, FCRO and IEHPI) have regular meetings to discuss and exchange information on new and existing activities and to build on existing synergies. The survey of funding recipients found that 7% of respondents (3 of 41) also received funding from CIC for their FCRP-funded project.

Although information on the sources of funding for FCR-related projects at the provincial level was not available for all jurisdictions, there is evidence that FCRP funding seems to have supported and complemented provincial initiatives. For instance, FCRP funding has been used to support the work of the Review Officer and the International Qualifications Recognition Funding Program in Nova Scotia, as well as some FCR-related work (such as FCR information on the Welcome BC Web site) in British Columbia. The evaluation found evidence to suggest that some activities at the provincial level are similar to federal efforts, especially in the production of information.

For example, the development of provincial occupational fact sheets or a dedicated Web site may overlap with the efforts of both FCRO and FCRP. When provincial regulations differ significantly, separate province-specific information may be justified. As pan-Canadian standards are implemented, it is expected that occupational information would become less fragmented.

Federal Interest

FCRP’s interest is consistent with both ESDC and GoC priorities. FCRP supports ESDC priorities by directly contributing to the strategic outcome of “enhanced Canadian productivity and participation through efficient and inclusive labour markets, competitive workplaces, and access to learning.” As outlined in the most recent ESDC Report on Plans and Priorities (2013-2014), ESDC has committed to speeding up and improving consistency for foreign qualification recognition processes. As well, the 2010 Speech from the Throne acknowledged working with P/Ts to strengthen recognition of foreign credentials through the Pan-Canadian Framework and the 2013 Speech from the Throne articulated the need to create jobs and secure economic growth as a top priority. The 2013 Economic Action Plan stated that the Government would support the development of a “pan-Canadian framework for foreign credential assessment and recognition and ensure that immigrants are better integrated into the Canadian labour force.” Footnote 9

Most key informant interviewees and funding recipient survey respondents (58%) identified coordinating stakeholders at the pan-Canadian level on solutions related to FCR as a pressing area for federal government support. Some interviewees also felt that the program’s provision of financial support is a useful and appropriate role for the federal government to play. The area where federal support is considered most pressing by funding recipient survey respondents is greater availability of tools and processes to assess foreign qualifications (34%).

Interviewees felt that the extent to which the federal government should be active in dealing with challenges with provincial regulations is a delicate matter with some wishing for a greater federal presence and some warning of jurisdictional issues. Interestingly, 41% (n=17) of surveyed funding recipients identified “greater leadership to advance the FCR agenda at the P/T level” as one of the greatest needs and 59% of those or 24% overall (n=10) saw this as a pressing area for federal support. A few key informants emphasized that many issues in FCR could be resolved through changes to the immigration process – an area where federal jurisdiction is clear.

Labour Mobility

The evidence regarding the success of the ILMI was generally of a positive nature. A large majority of the ILMI projects met their objectives and the incidence of working out-of-province increased in those occupations assisted by the ILMI compared to unassisted occupations. Some labour mobility barriers remain between jurisdictions and occupations although the quantitative study found that barriers are of greater concern among professions than among skilled trades.

Between 2009 and 2013, available data showed that ILMI has funded 44 projects for a total of $5.9 million. The average project amount was $134,680. The evidence available through the file review indicates that the vast majority of ILMI projects have achieved their objectives.

Since implementation of Chapter 7 of the AIT, there has been a general consensus Footnote 10 that labour mobility within Canada as a whole has improved significantly, however, barriers remain and licensing requirements may differ from province to province and occupation to occupation. Chapter 7 was significantly amended in 2009 to establish certificate-to-certificate recognition across jurisdictions, effectively reversing the burden of proof of competencies from workers to regulators. It is now up to regulators to demonstrate why a certified worker from one jurisdiction is not qualified to be licensed in their jurisdiction. Where significant differences in skills, areas of knowledge or abilities exist, Chapter 7 provides a mechanism for public posting of additional requirements. The evaluation found that there is a lack of information available to workers on the labour mobility requirements for each of the occupation.

Documents reviewed indicated that there is a continued lack of information on the requirements for transferring applicants, a continued misalignment of standards across provinces, and a lack of standardization of posting of exceptions to labour mobility under Chapter 7. Footnote 11

Further, according to the 2014 Status of Labour Mobility Research Report: Final Report, some workers in some jurisdictions in the following occupations continue to experience labour mobility barriers: “social workers, nurse practitioners (NPs), psychologists, engineering technicians and technologists, and power engineers”. Footnote 12 Barriers for some workers in these specific occupations are based on several factors, including non-compliance of province with labour mobility obligations for some occupations (e.g. nurse practitioners in British Columbia). There remains variability of entry to practice requirements among social work regulators for example, and four jurisdictions have posted exceptions against other jurisdictions citing scope of practice, requisite knowledge and skill, and significant variation in academic requirements. Workers not having obtained certification in their home province in occupations not requiring certification (e.g. engineering technology) will be not able to obtain certification in other jurisdictions just by virtue of moving. There remain differences in education requirements and scopes of practice in some occupations (social workers), and uncommon occupation classes and certification requirements (e.g., power engineering).

The quantitative study conducted as part of the evaluation found that the number of Canadians who change their province of residence is quite small, with an average annual mobility rate of less than 2% across all occupations for persons ages 25 to 64. It also found evidence that individuals in regulated professions (but not regulated trades) faced (and continue to face) barriers to interprovincial migration, or, “boundary effects”. As well, Canadians in regulated trades occupations were more likely to work outside their province of residence than those in unregulated occupations but individuals in regulated professions were considerably less likely to do so.

It should be noted that from 2005 to 2010, the probability of moving between provinces increased among regulated professionals when compared to individuals in unregulated occupations. The relative change in the regulated trades was actually negative over the same period. In summary, the study concluded that barriers to interprovincial labour mobility are of greater concern among professions than among skilled trades.

The quantitative study also found that, when comparing the changes within regulated occupations only, the incidence of working out-of-province (i.e., not changing the province of residence) increased in those occupations assisted by the ILMI compared to unassisted occupations. This is consistent with the initial research hypothesis that the ILMI has had a positive impact with those occupations that decided to address labour mobility barriers.

Contextual Shifts

There have been important changes in other areas of relevance to FCR matters. In particular, a new Action Plan was released by the FLMM which introduced four priorities. As well, there have been changes to the immigration system. These changes suggest that FCRP could fund projects that consider and complement these and other changes (such as pre-arrival training and/or bridge to work programs, processes to enable licensure in regulated occupations pre-arrival).

As the FCRP looks to the future, there have been two important shifts in Government priorities that could affect the program’s investment strategy.

Action Plan for Better Foreign Qualifications Recognition

Building on the Framework, the 2014 Action Plan for Better Foreign Qualifications Recognition issued by FLMM outlines the following four priorities:

  • Pre-decision information and assessment processes; 1/2
  • Workforce participation and integration supports;
  • Monitoring and evaluation; and
  • Communicating F-P/T actions to stakeholders.

The Framework issued in 2009 included a goal that ITIs would have early access to reliable and accurate information. The 2014 Action Plan acknowledges that more information is available online today, but emphasizes that content gaps and a lack of clear contact points and navigation help remain. As well, the Action Plan indicates that government has a role to inform immigrants about the demand for their skills and credentials before they arrive in Canada.

Beyond providing more information, the Action Plan identified increased access to assessment and recognition processes prior to arrival in Canada as important. This is consistent with the views of representatives from some occupations consulted for interviews and case studies. They see a role for the federal government to support pre-arrival licensure, through the establishment of overseas assessment centres, overseas training centres, and better communications between regulatory bodies and CIC missions.

The second area of the Action Plan focuses on steps to ensure the full utilization of ITI skills and experience. This full utilization would be facilitated by the development and improvement of bridge-to-work programs, occuppation-specific language training, orientation and training on Canadian workplace culture and practice, and mentorship programs.

The third area focuses on improved monitoring of progress, gaps and challenges and measurement of ITI FQR outcomes. The Action Plan suggests that governments will need to work more closely with regulators and stakeholders to identify effective and efficient strategies for data collection within each target occupation while respecting the legislative authorities and requirements in some jurisdictions.

The fourth area of the Action Plan centres around improved communications with stakeholders to raise their level of awareness and utilization of existing and future information, tools and other supports offered by jurisdictions and other key stakeholders.

The FLMM Action Plan is relevant for FCRP since the program funds projects that are directly related to issues addressed by the Action Plan. The Action Plan priority areas offer opportunities for FCRP to ensure its ongoing relevance to federal priorities and ensure consistency with the efforts of other stakeholders involved in FCR and who are impacted by the Framework and the related Action Plan.

Changes to the Immigration System: Express Entry

Some stakeholders interviewed for the evaluation noted that federal policies, particularly those of CIC, do not adequately address the disconnect between qualifying for immigration and qualifying in one’s occupation in Canada.

While a new Express Entry system has been recently introduced by CIC, Footnote 13 some concern was expressed by a few respondents that the new system may further complicate the Canadian FCR environment for ITIs wishing to live and work in Canada. However, others suggested that this new system presents opportunities to better integrate the processes for immigration and credentialing, and to bring the regulators closer to these decision making processes. Some welcomed a federal role in coordinating support for regulated professions, in providing services to support pre-arrival processes (for example, in terms of examination facilities and identity verification).

The new immigration system could be supported by FCRP-funded projects, particularly those that aim to make some or all parts of the FCR process available pre-arrival. The Action Plan also refers to the launch of the Express Entry system as being relevant for FCR-related initiatives.

4. Findings – Efficiency

Effectiveness of Investments

FCRP has funded a total of 142 projects over the last five years. Projects tend to be large, with an average project value of almost $600,000. After the implementation of the Framework, the FCRP adopted a more directive investment approach that prioritizes projects with target regulated occupations and P/Ts (although the Framework also guides FCRP’s investments in other occupations). The evaluation found examples of FCRP-funded projects that interviewees expressed as not resulting in a sustainable solution, and interviewee evidence identified a few instances where there was duplication between the projects funded with FCRP funds.

FCRP’s total budget over the five-year timeframe of the evaluation was $139.5 million, although the program only spent $105.3 million (a lapse of $34.2 million or 25%).

FCRP Planned versus Actual Financial Resources ($ millions)

  • 2009-2010
    • Planned: 26.4
    • Actual: 19.8
    • Variance: 6.6
  • 2010-2011
    • Planned: 29.8
    • Actual: 25.0
    • Variance: 4.91
  • 2011-2012
    • Planned: 29.4
    • Actual: 25.4
    • Variance: 4.0
  • 2012-2013
    • Planned: 26.9
    • Actual: 21.4
    • Variance: 5.6
  • 2013-2014
    • Planned: 26.9
    • Actual: 13.8
    • Variance: 13.1
  • Total
    • Planned: 139.5
    • Actual: 105.3
    • Variance: 34.2

The program had 142 projects underway during the period of the evaluation, with a cost per project of approximately $741,500. The average value of the projects was $592,250. Taking only the case study occupations into account, there is evidence that over the last five years, FCRP has funded multiple projects in each occupation with total values ranging from $1.4 million to $6.1 million (total FCRP funding for all seven occupations was approximately $25 million). All but one occupation was also the subject of a case study in the previous evaluation.

The document review found that, over the five-year period of the evaluation, 21 proposals were not funded. According to documents and internal interviews, FCRP personnel work with applicants to validate their project ideas prior to the submission of a formal proposal. Since FCR rests in P/T jurisdiction, FCRP has only the offer of funding to influence project objectives and the program takes a flexible posture when negotiating projects with possible contribution agreement holders.

When FCRP considers whether to invest or not, the proposed project must meet the program objectives as stipulated within FCRP’s Terms and Conditions. As well, a proposal must indicate a demonstrable need for FCRP funding which is aligned with program priorities, department activities and objectives. The proposal must also include information about the occupation being supported including the current size of the workforce in Canada, demand for professionals in the field, output of graduates from Canadian post-secondary institutions and the number of foreign educated professionals in the field. According to program documentation, FCRP seeks applications that have a return on investment as measured by a reduction in labour market skill shortages.

An undated FCRP document that presents the evolution of the program’s investment strategy indicates that after the implementation of the Framework, the FCRP adopted a more directive investment approach. According to the document, work with targeted regulated occupations and P/Ts was prioritized (although the Framework also guides FCRP’s investments in other occupations). Target occupations were identified using a multi-pronged approach that considered the supply of workers in the occupation, the demand for workers and other qualitative factors such as the willingness/readiness of the occupation for an FCR project, whether other initiatives were already underway, and the existence of mechanisms for inter-provincial dialogue, among others. The file review confirmed that most proposals mention the Framework in their objectives or had objectives that aligned with one of the four principles of the Framework.

However, there does not appear to be a direct linkage with the occupational action plans (most of which were developed by 2011). Program documentation also does not articulate how labour mobility projects with an FQR component are being identified and considered for project funding. Moreover, there is a lack of aggregate data about projects that have been funded and how they link to the objectives of FCRP and the Framework. Thus, the decisions around the selection of projects are not consistently taking into account previous funded projects.

As mentioned earlier, a few external interviewees identified projects that possibly should not have been funded since they were not sustainable (i.e., did not have a plan to implement the project post-FCRP) or the funding recipient had not properly engaged the necessary stakeholders to help ensure project success and sustainability after funding ended.

Funding recipients surveyed offered modest support for the nature of the investments. Keeping in mind that all of these respondents had been selected for an FCRP investment, about a quarter (23%, n=7 of 30) indicated that FCRP funding has been directed to the most effective investments to a great extent. Most respondents (53%, n=16) indicated that this had occurred to some extent and almost a quarter (23%, n=7) said to a little extent.

Evidence suggests that, while rare, there are cases of perceived duplication between projects funded by FCRP. Examples include projects funding different organization representing the same occupation, as well as similar projects funded by FCRP as well as the P/Ts (with FCRP funding). Some case studies also expressed that duplication between projects exists.

The review of the literature indicates that many of the practices identified as being more effective at facilitating the integration of newcomers have been incorporated into the FCR system in Canada. The literature review identified a number of initiatives (such as requiring an Education Credential Assessment as part of screening, programs that require having employment pre-arrival, alternative paths to licensing, online tools and self-assessments, mutual recognition agreements). However, information on the funding sources for most of these projects is not readily available, making it difficult to determine the role of FCRP funding. While areas such as improved screening at entry and pre-employment migration are beyond the mandate of FCRP, the practices of development and implementation of limited licensing and of alternative testing and assessment procedures, do fall within its mandate.

Sharing of Project Results

FCRP has spent considerable funds on projects before and over the timeframe of the evaluation, but does not have a formal mechanism to share project outputs (such as tools, approaches, handbooks, reports), lessons learned or best practices. While there is good sharing of FCRP-funded outputs within occupations, sharing between occupations appears to be limited to those who attend conferences and/or other presentations. Cross-occupational sharing could result in more efficient allocation of resources on projects that are able to build on past successes and learn from past projects that did not fully achieve their objectives.

FCRP does not have a formal mechanism that is available to the public, or to funding recipients, that provides access to FCRP-funded project outputs. However, there are networking opportunities supported by Employment and Social Development (ESDC) and the FQRWG that facilitate the sharing of information, best practices and tools across occupations and jurisdictions among those that attend these networking sessions. According to FQRWP progress reports, the sessions are described as including presentations on various topics (e.g., fair access legislation, engagement strategies) and opportunities to share promising practices with those in attendance.

The document review confirmed that there is good sharing of FCRP-supported processes and tools within occupations. However, information obtained for the evaluation, showed it was difficult to discern whether tools and processes developed with FCRP support were shared with organizations outside the occupation.

The 2010 Summative Evaluation of FCRP recommended an increased sharing and dissemination of existing tools and processes across occupations, sectors, and jurisdictions to increase the likelihood of their usage. According to the Management Response (2010), the recommendation had been partially implemented by establishing contribution agreements for workshops and conferences to disseminate project results. Examples include, through the Alliance of Sector Councils, FCRP delivered an FCR 101 workshop to increase awareness of FCR issues in Canada, particularly among employers in the non-regulated sector, and, through the Canadian Foundation for Economic Education, FCRP hosted a conference to showcase best practices in assessment and recognition of foreign trained individuals with representation from various occupational groups. As well, the Management Response mentioned that FCRP works collaboratively with the Going to Canada Immigration Portal Initiative on enhancements to the Working in Canada Tool that will include FCR-related information and changes Footnote 14 , although this collaboration does not contribute to the sharing of project outputs, lessons learned and best practices with other organizations. Planned activities (from 2010) included the development of an Action Plan by the FQRWP that promotes information sharing and also using the FCRO's Pan-Canadian Information Centre as a platform to share best practices. While FQRWG progress reports reported advancement on information sharing (with FQRWG members and other presenters), it is not certain whether the last of these actions has occurred.

Survey respondents reported they appreciated the sharing of best practices and processes developed for FCR when these opportunities were available, with 69% (n=25 of 37) and 35% (n=13 of 37) indicating these were the most useful aspects of attending an FCR event.

Interview respondents are mainly satisfied with current sharing mechanisms and the level of information dissemination to other FQR stakeholders. Most respondents believe that information is being exchanged but are not aware of the specific mechanisms or to what extent tools and processes are shared. Most respondents said they were aware of FCR-related tools and processes that are available as a result of FCRP efforts and most find these tools and processes useful. However, there was limited awareness of what projects were supported by FCRP funding overall. Note that key informants were selected to participate in the evaluation because of their active involvement in FCR-related matters or are funding recipients. Thus, it is likely that these individuals are often invited to/participate in FCR-related meetings, conferences and roundtables where FCRP-funded projects are discussed.

5. Conclusions

Effectiveness – Achievement of Expected Outcomes

FCRP has been successful in continuing to keep FCR issues in the minds of the key stakeholders and articulating the need for change, through project funding and involvement with the FQRWG. Outreach by FCRP-funded projects was considered to be good although outreach conducted by the program itself on raising awareness of the Framework was more limited. FCRP supports horizontal linkages through various mechanisms; the most commonly cited one being the FQRWG.

FCRP has funded 142 projects over the five-year timeframe of the evaluation, of which 27 occupations were funded to develop standardized processes and/or tools. Of these, 21 occupations reported some degree of progress on the issue. However, due in large part to a complex F-P/T environment, progress is partial and varies among occupations (i.e., covering only some jurisdictions and/or some parts of the process).

Data and/or aggregate reports to assess the results of funded projects were lacking. As well, the evaluation revealed that the program’s performance measurement strategy is inconsistent with more recent program documentation, includes unclear terminology and would benefit from and/or improved/additional data sources.

Relevance – Need for the program

The evaluation findings support a continued need for a federal program that coordinates the efforts of the various players in FCR and provides funding for initiatives and projects that are aligned with the principles of the Framework. The needs of FCR stakeholders are diverse, varied and evolving. Potential for overlap with other funders is an issue (particularly with P/Ts), principally with respect to the development of information resources such as fact sheets. There is a federal role in the coordination of the stakeholders (to address existing or potential overlap and duplication and assist with formulating solutions for continuing and evolving needs. The new FQRWG Action Plan and recent changes to the immigration system will also influence the role of the program and the activities it funds.

While almost all projects funded under the ILMI have achieved their objectives, there remain some barriers to interprovincial labour mobility. These barriers differ between jurisdictions and occupations although they appear to be greater among professions than among skilled trades.

Performance – Efficiency and Economy

FCRP underspent its budget by 25% over the five-year timeframe of the evaluation. Many of the case study occupations are only partially achieving standardized processes and tools and they have received funding for multiple projects from the program with total FCRP funding for all seven occupations totalling approximately $25 million. However, the cost to making systemic pan-Canadian changes to the regulatory environment of an occupation is high due to the complexities associated with effecting system-wide changes.

The FCRP uses a directive investment approach that prioritizes projects with target occupations and P/Ts, guided by the Framework. However, the evaluation found some instances where projects funded were not sustainable in the long-term because of the lack of support and involvement from various stakeholders, or because it was aimed at achieving a longer-term outcome.

Aside from the occasional networking opportunities funded by ESDC or the FQRWG (such as conferences and meetings), there is no formal mechanism to share FCRP project outputs, lessons learned or best practices. The evaluation did find good sharing of FCRP-funded outputs within occupations, but that sharing between occupations is limited to those who attend the networking opportunities. Cross-occupational sharing would allow current and future FCRP projects to build on past successes and learn from past projects that did not fully achieve their objectives.

6. Recommendations

Based on the evidence presented in this evaluation, it is recommended that the Foreign Credential Recognition Program:

  1. Continue to leverage on coordination and consultation efforts of the FLMM and to fund and hold consultation and coordination activities.

    There is a need to improve outreach about the Framework and to continue to raise awareness of the FCR issue and evolving needs. FCRP is well positioned to play a strengthened coordination and consultation role, working with governments, regulatory bodies and immigration stakeholders, to ensure consistency with principles of the Framework.
  2. Further improve awareness of funded projects, their outputs and lessons learned from implementation, and establish a mechanism for information sharing.

    FCRP should enhance the sharing of project results, lessons learned and best practices across occupations, sectors and jurisdiction. This would contribute to the effective development and delivery of projects, ensure that there is no overlap with other projects previously or currently being funded by FCRP, and build on past successes.
  3. Further explore the issues and the areas in need of support around interprovincial labour mobility.

    The evidence regarding the success of the ILMI was generally of a positive nature. A large majority of the ILMI projects met their objectives and the incidence of working out-of-province increased in those occupations assisted by the ILMI compared to unassisted occupations. Some labour mobility barriers that differ between jurisdictions and occupations remain, and FCRP is well positioned to assist occupations in addressing these barriers.
  4. Renew and share an FCRP investment strategy and project selection criteria that align with other federal, FLMM and occupational priorities.

    The way in which projects funded by FCRP are aligned with federal, FLMM and occupational priorities should be better documented. As well, for subsequent projects within the same occupation, they should explicitly build on past successes, best practices and lessons learned and be linked with an overall strategic vision to increase the likelihood of meeting the medium and long-term program outcomes.
  5. Advance performance measurement activities, including collecting and collating performance data and conducting follow-up activities.

    There is an opportunity to renew the performance measurement strategy and to strengthen the plan for the collection and use of performance data. In particular, more could be done to collect and aggregate information related to project successes, linked to the program’s logic model.

Annex A: Logic Model for FCRP

Foreign Credential Recognition Program Logic Model

Logic model
Logic model chart
Text version of Appendix A: Foreign Credential Recognition Program (FCRP) Logic Model
Strategic objective

FCRP seeks to contribute to developing fair, transparent, consistent, and timely foreign qualification assessment and recognition capacity and to enhance labour market outcomes of internationally-trained workers in targeted occupations and sectors.

Components of Work

FCRP has the following components of work:

  • Regulated and Non-regulated Occupations
  • Non-occupation Specific
  • Horizontal Leadership
Activities

FCRP has the Following four activities:

1.1 Develop/enhance the assessment and recognition capacity of partners and stakeholders to assess and recognise foreign qualification and integration issues

1.2 Support the development of specialized information on qualification recognition and assessment

1.3 Policy and strategy development to understand FQR in the broader context of immigrant labour market integration

1.4 Engage all levels of governments and stakeholders

Outputs

FCRP produces the following five outputs

2.1 Contribution agreements with predetermined occupations to understand and address regulated, non-regulated occupations and sector-specific FQR issues

2.2 Contribution agreements with predetermined occupations to understand and address FQR issues instrumental to advance the FQR agenda

2.3 Specialized information and data and internal reports that identify and/or address FQR issues

2.4 Promotion and awareness of the Framework and national coordination among partners and stakeholders on FQR

2.5 Federal and FPT committees and public outreach activities to advance the FQR agenda

Immediate Outcomes

FCRP produces the following three immediate outcomes:

3.1 Understanding, consensus, collaboration and commitment among stakeholders and partners on issues and potential solutions related to FQR

3.2 Promotion, Information sharing and transfer of best practices in developing Pan-Canadian FQR processes

3.3 Partnership among key stakeholders and P/Ts to advance the FQR agenda

Medium-Term Outcomes

FCRP produces the following medium-term outcome:

4.1 Availability of tools and processes to assess and recognize foreign qualifications among organizations

Longer-Term Outcomes

FCRP produces the following two longer-term outcomes:

5.1 Standardization of Pan-Canadian FQR processes and tools in targeted occupations and sectors

5.2 Use of tools and processes by organizations to assess and recognize the qualifications of internationally-trained workers in targeted occupations and sectors

Ultimate-Outcomes

Enhanced labour market outcomes of internationally-trained workers in targeted occupations and sectors

Alignment with ESDC Strategic Outcome

FCRP is aligned with HRSDC Strategic Outcome 1: A skilled, adaptable and inclusive labour force and an efficient labour market

Annex B: Summary of the Methodology

The joint summative evaluation of FCRP and ILMI covering the five-year period of April 2008 to March 2013, was designed to gather information on each of the evaluation issues using a multi-method approach. Where possible, there is a balance applied between the use of quantitative and qualitative methods, with qualitative methods providing further description and explanation for the quantitative information. Data for the evaluation was collected from September 2014 to January 2015, using both primary and secondary data sources. In all, seven methods were implemented.

Primary data collection:

Key informant interviews (KIIs)

  • Interview guides were developed to capture evidence and KIIs’ perceptions, for example, on the need for FCRP funding, and on duplication or complementarity with other similar programs. The analysis noted the extent of corroboration or divergence in opinions for topic across and within the respondent types. Interview results treat each interview as the base for analysis rather than the number of individuals interviewed.
  • Thirty-four interviews of approximately 60 minutes each were conducted with 42 individuals, either by phone or in-person. (Numbers of interviewees are indicated in brackets). Some interviews were attended by two key informants, so that the total number of key informants is actually higher than the total number of interviews completed.
  • Interviewees were representatives of: Federal Departments/Other Government Department (OGDs) (4); Organizations that received FCRP funding (16); Stakeholders (7); Funded Non-Regulated Occupation Projects (4); Funded Non-Occupation Specific Projects (3); Funded Regulated Occupation Projects (4); and ESDC (4).

Survey of FCRP funding recipients

  • The survey was designed to capture evidence and KIIs’ perceptions, for example, on whether the FCRP has supported P/Ts and key stakeholders to develop systemic capacity to implement the Framework across occupations.
  • All organizations that received funding between April 2008 and March 2013 were invited to participate in the survey, except P/Ts since they were interviewed instead. Forty-one of the eligible 70 organizations completed the survey, (resulting in a final response rate of 59%). The survey invitation was issued to 91 unique contacts over 142 FCRP-funded projects, but 21 of these contacts were ineligible, either due to invalid contact information or they were not the correct contact person. Response rates based on the 91 invitations issued would be 45%. More details on the evidence obtained from applying the evaluation methodology are available upon request.
  • Despite the good response rate of 59%, the total number of respondents was still quite low (41), limiting the ability of the analysis to do meaningful comparisons by respondent type.

Occupational case studies

  • The case studies review methodology was designed to capture evidence and KIIs’ perceptions, for example, regarding whether FCRP funding is being directed to the most effective investments and regarding alternative delivery methods.
  • In all, eight case studies were conducted: five in funded regulated occupations, including physicians, engineers, nurses, medical laboratory technologists and medical radiation technologists; two in funded non-regulated occupations, including information and communications technologist and biotechnologist; and one in a non-funded regulated occupation, which was veterinarians.
  • The key criterion for the selection of the occupations to be assessed was continuity with the summative evaluation conducted in 2009. Five of the eight selected occupations assessed through this evaluation were also assessed as case studies in 2009. This provided a baseline upon which to examine progress over time.
  • The case studies were conducted using the following data collection methods:
    • A review of all funded projects targeting the occupation under study;
    • Interviews with key informants; and,
    • A review of relevant documentation and literature on the occupation.
  • A summary for each case study/occupation was developed in conformity to an approved outline.
  • The case studies encountered challenges that limited the number of interviews that were held. As a result, some of the case studies include the perceptions of only a limited number of informants.

Secondary data collection:

  1. File and document review

    • The File and document review was designed to capture evidence, for example, regarding the nature of funded activities and associated outcomes of FCRP-funded projects where the CA holder is a P/T. It included the review of documents produced by the program, the GoC, the FLMM, and electronic file project information.
    • The review was used, in part, to address the relevance, impact, and the cost-effectiveness of FCRP as a Grants and Contributions program and the relevance of ILMI. It also provided FCRP context, including the social environment surrounding the program (Red tape Reduction Program, and focus on outreach, etc.).
    • There was a lack of available information to address some of the indicators, and thus the limitations pertaining to the document review are directly related to the availability of recent information. Much of the documentation available covered the earlier years of the evaluation scope, with fewer documents from more current years
  2. The study entitled « Labour market outcomes and determinants of success for the integration of internationally-educated immigrants in the Canadian labour market: A quantitative study » focused on profiling the socio-demographic characteristics and labour market outcomes of immigrants aged 25-64, holding at least a post-secondary degree obtained outside Canada, and on a comparison of this population with the Canadian-born or educated in Canada. Special attention was paid to changes between 2006 and 2011, to the determinants of labour market success defined as a match between education and job skills education and employment earnings, and to the comparison between the three groups. Two data sets were used for this study: Census 2006 and the 2011 National Household Survey.
  3. The study entitled « The Current State of Labour Mobility in Canada » focused on determining the degree to which interprovincial differences in labour standards represent barriers to mobility (rather than the determinants of that mobility per se), to assess any changes in that mobility that might be attributed to the ILMI. The empirical strategy included a treatment effects approach, impeded migration, and provincial wage differentials. The report uses the master files of Census 2006 and the 2011 National Household Survey to examine mobility patterns and Survey of Labour and Income Dynamics in 2005 and 2010 to examine occupational wage structures across provinces.
  4. The Literature review, an updated version of a review prepared for the 2009 evaluation of the FCRP, collected/analyzed information regarding the relevance and cost-effectiveness of FCRP from both academic and grey literature available online between 2009 and 2013.

Annex C: List of Technical Reports

  1. Document and File Review for the evaluation of the Foreign Credential Recognition Program (Goss Gilroy Inc.)
  2. Literature Review on Foreign Credential Recognition (ESDC)
  3. Labour market outcomes and determinants of success for the integration of internationally-educated immigrants in the Canadian labour market (Quantitative study by Christopher Ferrall – Queens University)
  4. The Current State of Labour Mobility in Canada (Quantitative study by Torben Drewes – Trent University)
  5. Survey of Funding Recipients technical report for the evaluation of the Foreign Credential Recognition Program (Goss Gilroy Inc.)
  6. Key Informant Interviews Technical Report for the evaluation of the Foreign Credential Recognition Program (Goss Gilroy Inc.)
  7. Case Studies Technical Report for the evaluation of the Foreign Credential Recognition Program (Goss Gilroy Inc.)

List of Abbreviations

  • AIT: Agreement on Internal Trade
  • CIC: Citizenship and Immigration Canada
  • ESDC: Employment and Social Development Canada
  • FCRP:Foreign Credential Recognition Program
  • FCR: Foreign credential recognition
  • FCRO: Foreign Credential Referral Office
  • FLMM: Forum of Labour Market Ministers
  • FQR: Foreign qualifications recognition
  • FQRWG: Foreign Qualifications Recognition Working Group
  • ICT: Information and communications technology
  • F-P/T: Federal - Provincial/Territorial
  • IEHPI: Internationally Educated Health Professional Initiative
  • ILMI: Interprovincial Labour Mobility Initiative

Page details

Date modified: