Horizontal Evaluation of the Roadmap for Canada’s Official Languages 2013-2018: Education, Immigration, Communities and Evaluation of the Coordination Program

This evaluation report is part of the Horizontal Coordination of the Roadmap for Canada's official languages 2013-2018: education, immigration, communities

Evaluation Services Directorate
Period: 2013-2014 to 2016-2017

May 29, 2017

Cette publication est également disponible en français.

This publication is available in HTML and PDF on the Internet at Canada.ca/canadian-heritage

© Her Majesty the Queen in Right of Canada, represented by the Minister of Canadian Heritage, 2017

Cat. No. CH7-58/2017E-PDF
ISBN.: 978-0-660-08793-1

Contents

• Acronyms
• Executive summary
• Introduction
• 1. Chapter 1: horizontal evaluation of the Roadmap for Canada's official languages 2013-2018: education, immigration, communities
  • 1.1 Program description
    • 1.1.1 Background and context
    • 1.1.2 Objectives and outcomes
    • 1.1.3 Governance
    • 1.1.4 Program resources
  • 1.2 Evaluation methodology
    • 1.2.1 Document review
    • 1.2.2 Interviews with key stakeholders
    • 1.2.3 Literature review
    • 1.2.4 2016 public opinion survey on official languages
    • 1.2.5 Comparative analysis of existing surveys (2005, 2012 & 2016)
    • 1.2.6 OLMC case studies
    • 1.2.7 Expert panels
    • 1.2.8 Limitations and constraints
  • 1.3 Findings ‒ relevance
    • 1.3.1 Core issue 1: continued need for the program
    • 1.3.2 Core issue 2: alignment with government priorities
    • 1.3.3 Core issue 3: alignment with federal roles and responsibilities
  • 1.4 Findings - performance
    • 1.4.1 Core issue 4: achievement of expected outcomes
    • 1.4.2 Core issue 5: demonstration of efficiency and economy
  • 1.5 Conclusions
  • 1.6 Recommendations and management response
• 2. Chapter 2: evaluation of the Official Languages Coordination Program – part of the horizontal coordination of the Roadmap 2013-2018: education, immigration, communities
  • 2.1 Program description
    • 2.1.1 Background and context
    • 2.1.2 Objectives and outcomes
    • 2.1.3 Governance
    • 2.1.4 Program resources
  • 2.2 Evaluation methodology
    • 2.2.1 Document review
    • 2.2.2 Literature review
    • 2.2.3 Interviews and qualitative questionnaires with key stakeholders
    • 2.2.4 Expert panel
    • 2.2.5 Limitations and constraints
  • 2.3 Findings ‒ relevance
    • 2.3.1 Core issue 1: continued need for the program
    • 2.3.2 Core issue 2: alignment with government priorities
    • 2.3.3 Core issue 3: alignment with federal roles and responsibilities
  • 2.4 Findings ‒ performance
    • 2.4.1 Core issue 4: achievement of expected outcomes
    • 2.4.2 Core issue 5: demonstration of efficiency and economy
  • 2.5 Conclusions
  • 2.6 Recommendations and management response
  • 2.7 Looking forward
• Annex A: evaluation matrices
• Annex B: Roadmap 2013-2018 resources
• Annex C: extract of the public opinion surveys
• Annex D: component of the Official Languages Act
• Footnotes

Acronyms

ACOA

Atlantic Canada Opportunities Agency

ACUFC

Association des collèges et universités de la francophonie canadienne

ADM

Assistant Deputy Minister

CADMOL

Committee of Assistant Deputy Ministers on Official Languages

CAMERA

Communications and Math Employment Readiness Assessment

CanNor

Canadian Northern Economic Development Agency

CCA

Canada Council for the Arts

CCACOL

Crown Corporation Advisory Committee on Official Languages

CCAF

Community Cultural Action Fund

CDMOL

Committee of Deputy Ministers of Official Languages

CEDEC

Community Economic Development and Employability Corporation

CED-QC

Canada Economic Development for Quebec Regions

CEOL-TBS

Centre of Excellence for Official Languages – Treasury Board Secretariat

CHSSN

Community Health and Social Services Network

CIHR

Canadian Institutes of Health Research

CNFS

Consortium national de formation en santé

CWGOLR

Coordinating Working Group on Official Languages Research

DPR

Departmental Performance Report

EAN

Economic Action Network

EDI

Economic Development Initiative

ESD

Evaluation Services Directorate

ESDC

Employment and Social Development Canada

EX-CADMOL

Executive Sub-Committee of the Committee of Assistant Deputy Ministers on Official Languages

FAAFC

Fédération des aînés et aînées francophones du Canada

FCFA

Fédération des communautés francophones et Acadienne du Canada

FedDev

Federal Economic Development Agency for Southern Ontario

FedNor

Federal Economic Development Initiative for Northern Ontario

FMC

Francophone minority communities

FOLS

First official language spoken

FSB

Francophone Significant Benefit

FTE

Full-time equivalent

HC

Health Canada

HMAF

Horizontal Management and Accountability Framework for the Roadmap for Canada's Official Languages 2013-2018

IRCC

Immigration, Refugees and Citizenship Canada

ISEDC

Innovation, Science and Economic Development Canada

JUS

Justice Canada

LES

Literacy and essential skills

LINC

Language Instruction for Newcomers to Canada

MCCF

Ministerial Conference on the Canadian Francophonie

NRCC

National Research Council Canada

NTPBP

National Translation Program for Book Publishing

OCOL

Office of the Commissioner of Official Languages

OL

Official languages

OLA

Official Languages Act

OLAC

Official Languages Advisory Committee

OLACF-OLS

Official Languages Accountability and Coordination Framework – Official Languages Secretariat

OLB

Official Languages Branch

OLHCP

Official Languages Health Contribution Program

OLLS

Official Languages Law Section

OLMC

Official language minority community

OLS

Official Languages Secretariat

OLSPB

Official Languages Support Programs Branch

OLWG

Official Languages Working Group

PAA

Program Alignment Architecture

PCH

Department of Canadian Heritage

PCO

Privy Council Office

PRG

Canadian Heritage Research Group

PSPC

Public Services and Procurement Canada

PWGSC

Public Works and Government Services Canada

QCGN

Quebec Community Groups Network

RDÉE

Réseau de développement économique et d'employabilité

RGMOL

Reference Group of Ministers on Official Languages

RNFJ

Réseau national de formation en justice

Roadmap

Roadmap for Canada's Official Languages 2013-2018

RPP

Report on Plans and Priorities

SSHRC

Social Sciences and Humanities Research Council

TBS

Treasury Board Secretariat

WD

Western Economic Diversification Canada

Executive summary

The purpose of this report is two-fold. It presents the results of the evaluation of two interrelated programs: the Roadmap for Canada's Official Languages 2013-2018: Education, Immigration, Communities (Chapter 1) as well as the Official Languages Coordination Program – part of the Horizontal Coordination of the Roadmap for Canada's Official Languages 2013-2018: Education, Immigration, Communities (Chapter 2).

Objectives of the two evaluations

The horizontal evaluation of the Roadmap was conducted in accordance with the parameters set out in the Treasury Board of Canada Secretariat (TBS) Policy on Evaluation, whereby most of the 28 initiatives in the Roadmap must be the subject of a distinct evaluation. The horizontal evaluation of the Roadmap thus focuses on the overall dimension of the Roadmap and examines its relevance in relation to the needs of Canadians, its alignment with the priorities, roles and responsibilities of the Government of Canada, its overall results and efficiency. To avoid duplication in the gathering and analysis of data, the horizontal evaluation of the Roadmap is based in part on the evaluation products of partners regarding their initiatives, and on the evaluation of the Official Languages Coordination Program – part of the Horizontal Coordination of the Roadmap 2013-2018. It also serves to inform the renewal of the next multi-year official languages plan.

The evaluation of the Official Languages Coordination Program – part of the Horizontal Coordination of the Roadmap 2013-2018 examines the relevance, performance, efficiency and economics of the Program, and also informs the improvement of the next multi-year official languages plan and the horizontal coordination that should support it. It is important to note that the Official Languages Branch (OLB) of Canadian Heritage (PCH) has the mandate of horizontal coordination of the Roadmap and also more general functions of official language coordination and governance. The evaluation only examines the Official Languages Coordination Program – part of the Horizontal Coordination of the Roadmap for Canada's Official Languages 2013-2018, but refers to the federal government's Official Languages Coordination Program, as several functions, responsibilities and resources are interrelated.

Overview of the two programs evaluated

The Roadmap 2013-2018 is a horizontal initiative that represents an investment of $1.1 billion over five years. It supports PCH and 13 other federal departments and agencies, so they can implement 28 official languages initiatives in the fields of education, immigration and community development. The Official Languages Coordination Program – part of the horizontal coordination of the Roadmap 2013-2018, ensures the coordination of work by Roadmap partners.

All interdepartmental, awareness and coordination functions by PCH total $10.5 million over five years for the Official Languages Coordination Program. (It is impossible to estimate the share of those resources for the part of the Horizontal Coordination of the Roadmap alone.) There is no logic model specific to the Official Languages Coordination Program – part of the horizontal coordination of the Roadmap, but instead an overall logic model for the Horizontal Coordination of Official Languages. The expected results include:

• Partners in the Roadmap provide relevant information regarding the implementation and results of their initiatives, including results related to efficiency and economy.
• Decisions by the Minister of Canadian Heritage and senior executives are based on thorough analyses, relevant data and useful information.
• Governance is strengthened for greater efficiency and better coordination.

The achievement of these objectives should contribute toward the final outcome targeted by the Roadmap for Canada's Official Languages 2013-2018: "Canadians live and thrive in both official languages and recognize the importance of French and English for Canada's national identity, development and prosperity."

Evaluation approach and methodology

The horizontal evaluation is based on the following data sources: documentation produced by PCH, partners and stakeholders in the Roadmap, including partners' evaluation products; a documentary review; quantitative data from censuses, inquiries, studies and surveys (2005, 2012 and 2016); the opinions and perceptions of PCH executives working on the horizontal management of the Roadmap, of partner institutions and stakeholders in the Roadmap, gathered during interviews; and the opinions and perceptions of official language minority communities (OLMCs) collected during case studies, and expert opinions gathered from panels.

The evaluation of the Official Languages Coordination Program – part of the Horizontal Coordination of the Roadmap 2013-2018 involved four data sources: a documentary review, texts, interviews with key stakeholders, and two expert panels on the Roadmap that also examined issues related to horizontal coordination.

Limitations and constraints

It is difficult to measure, in concrete terms, the results of multiple interventions under the Roadmap compared with the ultimate outcome, and some factors contributing to that problem: the evaluation is conducted at about the halfway mark, there are delays in submitting initiative evaluation products, and the nature of the funded activities that demonstrate visible long-term results. The evaluation also raises the lack of performance indicators to support the measurement of progress toward shared results by pillar and the ultimate result of the Roadmap, which constitutes a major constraint.

The interviews with key stakeholders and the expert panels provide other elements in response to the questions regarding the expected results of the three pillars of the Roadmap as a whole. However, some stakeholders instead looked at the contribution of one or another of the 28 initiatives, and some mentioned that it is actually too early to give an opinion on the progress toward the expected results, as the current Roadmap has only existed for three years. The analyses of indicators related to the nine OLMCs selected for the case studies are based on census data from 2011. Those data nonetheless allow for a comparison with 2006, and provide a background regarding changes in vitality between 2012 and 2016, which the evaluators were able to explore in the interviews.

A comparative analysis of the results of existing public opinion surveys allowed for some comparison of the appraisal and perception of the two official languages in 2005, 2012 and 2016. That provides a background for the interpretation of findings that emerge from other sources. However, it is impossible to determine the role that government intervention can have on the evolution of those perceptions.

The distinction between the Official Languages Coordination program as a whole and the Official Languages Coordination Program – part of the Horizontal Coordination of the Roadmap for Canada's Official Languages 2013-2018 is a challenge, and affects all data sources in this evaluation. Although partners provide assiduous vertical accountability of their progress toward the Roadmap's expected results, there is no data collected regarding the measures established for the horizontal coordination function of the Roadmap. In addition, there are very few documents specifically about the Official Languages Coordination Program – part of the Horizontal Coordination of the Roadmap. However, the documentary review provides contextual elements and constitutes complementary information to the interviews.

Given the few texts in the fields related to the nature of the Roadmap and the coordination of similar horizontal initiatives, as expected, very few relevant texts were identified. As a result, the literature review instead provides information to complement the two evaluations. Finally, for the two evaluations, the findings from the interviews are based in part on the opinion of people with a direct interest in the Roadmap or its coordination. However, the approach used minimized the potential bias, as the discussions were conducted by qualified evaluators and the respondents had to justify their perceptions or provide supporting examples.

Chapter 1: horizontal evaluation of the Roadmap for Official Languages 2013-2018: education, immigration, communities

Findings

Relevance

Overall, official languages remain a cornerstone of Canadian identity and are generally seen in a positive light by Canadians, and the current Roadmap addresses several ongoing needs of OLMCs. However, it would be necessary to adapt some interventions to better address needs in certain areas, and the needs of English-speaking OLMC's in Quebec in particular. Among other things, greater support is needed in the areas of early childhood and youth and seniors, for the integration and retention of newcomers, and to counter the vulnerability of official language minority media. There is also a lack of detailed data to better identify needs related to health, workforce development, regarding promising sectors in terms of economic development, and the needs of OLMCs in rural or remote areas.

Performance – efficiency

Design and implementation

The Roadmap is seen within the government as a promotional tool and, externally, as a vision statement. Within federal institutions, it contributes to greater awareness of obligations under the OLA, but there is a perception of disengagement regarding their official language obligations for institutions that are not part of the plan. The current Roadmap and the previous five-year plans had a leverage effect in certain areas, as well as a structuring effect. However, the Roadmap contributed to confusion that has lasted for several years between existing programming, which was increased in some cases, and new initiatives or new funds.

Federal partners are divided between those who do not see progress attributable to the horizontal nature of the Roadmap, and those who are of the opinion that the Roadmap has a broader impact than the sum of its parts. The grouping of initiatives by pillar allows for work on shared results, but the pillar structure also fosters work in silos, which goes against the desired collaboration between departments and with community organizations. Moreover, the evaluation finds that the performance measurement strategy in place does not include indicators or data collection specifically to measure progress against shared results by pillar and the ultimate result.

Achievement of expected results

Education pillar

The Roadmap contributes to opportunities to learn the minority official language. However, the simultaneous evaluation of the Official Languages Support Programs (OLSPs) indicate that it remains difficult to measure the intermediate and ultimate results of the funded activities. However, the evaluation indicates that the provinces and territories are advancing the activities set out in their respective agreements. The evaluation also notes the absence of an obligation to offer second-language learning programs, which places particular importance on federal investment. Regarding training opportunities in the first official language and the learning of the second official language, the evaluation notes the significant contribution of Health Canada's Official Languages Health Contribution Program (OLHCP), and the contribution to a similar program in justice. Federal partners also noted the Roadmap's contribution in terms of the development of language technology that supports the learning of a second official language.

Immigration pillar

Federal partners and stakeholders agree that there are specific actions in Francophone immigration, and several stakeholders on the ground have better expertise, which should help the government achieve its objectives. Preliminary results of the evaluation by Immigration, Refugees and Citizenship Canada (IRCC) provide indications of the progress toward the overall result for this pillar, but also indicate that additional efforts will be needed to increase Francophone immigration to OLMCs. For example, the language skills of newcomers in either official language were improved. Francophone immigrants are active in the labour market at rates comparable to other immigrants, and they are volunteering, are involved in groups and have a sense of belonging in Canada at rates comparable to those of Canadians in general. However, their relative proportion remains below set targets: between 2013 and 2016, the proportion of Francophone permanent residents in the immigrant population reached 1.5%, lower than the objective of 4.4%, and 44% of Francophone immigrants who settled in OLMCs were in the economic class, representing approximately 1.1% of all economic immigration outside Quebec, still below the objective of 4%.

Communities pillar

Stakeholders are largely convinced of the contribution of the Roadmap to the vitality of OLMCs, and interviews, documentation (evaluation products, etc.) and case studies provide multiple examples of progress in terms of increased activities and outputs in health, justice, arts, culture and economic development. Given that some initiatives are part of the various five-year plans that preceded the current Roadmap, we note that the Roadmap contributes to ongoing progression of certain results for communities. Regarding progress toward the expected results by pillar, there is little information related directly to the results obtained for several initiatives: the evaluation is conducted at about the halfway mark, there are delays in submitting initiative evaluation products, and the nature of the funded activities gives visible long-term results. As well, overall, the nature and very structure of the Roadmap presents a challenge in terms of attributing the anticipated effects.

Efficiency and economy

Data to demonstrate the efficiency and economy of such a horizontal initiative are limited, but partners' evaluation products conclude in a positive manner that their own initiatives are efficient. Measures have been put in place at PCH for greater efficiency. Several other partners, including Health Canada and Employment and Social Development Canada (ESDC), have revised their program terms to be more efficient, and some (PCH, IRCC, Health Canada and ESDC) have already looked at future opportunities to improve efficiency or economy. Finally, federal stakeholders do not see any other way of achieving the same results as the Roadmap.

Chapter 2: Official Languages Coordination Program – part of the Horizontal Coordination of the Roadmap for Canada's official languages 2013-2018: education, immigration, communities

Findings

Relevance

Horizontal management of official languages at the federal level existed prior to the Roadmap and is still needed in order to pursue shared objectives regarding official languages in a coordinated manner. However, there is some confusion between the horizontality of the Roadmap, the broad leadership and coordination role conferred on PCH, and the "vertical" decision processes within each federal department that are not designed for horizontal collaboration. As noted in the last evaluation, there is room for improvement in the horizontal management of official languages at the federal level and in the horizontal coordination of the Roadmap in order to clarify the roles and responsibilities and better align governance, the chain of accountability and the decision process. The OLB is currently conducting a review of the horizontal governance of official languages within the federal government, including the horizontal coordination of the Roadmap.

Performance – effectiveness

The simplified and coordinated collection of data regarding progress and results from partners seems to be working well. The current strategy assumes an aggregation of information by the OLB based on reports by initiative, by partner. In general, partners are of the opinion that reporting is effective and not too costly, and that the OLB supports them. It increases the information provided to the Minister of Canadian Heritage and to other ministers of partner institutions. However, according to some partners, there is not enough exchange of information regarding results, and there are not enough details in the annual report to the Committee of Assistant Deputy Ministers on Official Languages (CADMOL). It must be noted that regarding information intended for the public, despite initiatives undertaken such as the redesign of the annual report to Canadians, stakeholders such as representatives of OLMCs and the Office of the Commissioner of Official Languages (OCOL) have nonetheless raised a few concerns regarding the lack of sufficient communication with Canadians, particularly regarding the horizontal governance of official languages.

It is difficult to determine the quality and frequency of opinions and advice provided by the OLB, as they are not systematically documented. However, partners seem to be generally satisfied. Some provide supporting examples, such as analyses produced to help answer questions from their minister or questions asked in the House of Commons or by parliamentary committees, and to prepare data for reporting. Finally, the changes to the governance structure since 2013 have been positive; the mandates of committees have been renewed, and the frequency of meetings and participation rate are appropriate. The most influential committee at this time is the CADMOL, but it has no established responsibilities regarding decision-making. Some partners also suggest an additional committee of directors to improve information sharing.

Efficiency and economy

Since the integration of the Official Languages Secretariat (OLS) with the OLB, it is impossible to clearly identify the portion of financial and human resources dedicated specifically to the horizontal coordination of the Roadmap. The data include resources for both coordination of the Roadmap and coordination of official languages. An analysis of the optimization of resources is therefore impossible. There is also no data regarding possible alternatives and possible improvements for the use of resources.

Recommendations and management response

Horizontal evaluation of the Roadmap for Official Languages 2013-2018: education, immigration, communities

Recommendation 1 - Alignment of key initiatives and programs

The evaluation indicates that the current Roadmap addresses several ongoing needs of OLMCs, but it must continue to adjust to the changing demographic, social and economic context. According to the findings of the evaluation, there are emerging needs in certain areas (early childhood and youth, seniors, integration and retention of newcomers, vulnerability of minority media). Considering the context of the renewal of the federal strategy and new priorities regarding official languages:

Recommendation

Given the context of the renewal of the federal strategy and the government's new priorities regarding official languages, it is recommended that the Assistant Deputy Minister, Citizenship, Heritage and Regions:

1. prioritize (in cooperation with partners) the programs and initiatives that can address the emerging needs of OLMCs and the government's priorities; and
2. give preference to programs and initiatives that can best achieve the expected results of the next federal multi-year official languages plan.

Management response

Recommendation 1: accepted.

The Official Languages Branch recognizes that there are emerging needs in certain areas and it is appropriate to ensure that programs and initiatives are prioritized and promoted in order to better meet the emerging needs of OLMCs, government priorities and the expected results of the next multi-year official languages plan.

The OLB supported the Department in its cross-Canada consultations in 2016 to identify the needs and priorities of OLMCs. Several partners also held sectoral consultations. The OLB is working together with federal partners to propose programs and initiatives for the next federal official languages action plan that meet these needs and priorities.

Action plan

Action	Deliverables	Timetable	Program sponsor
1.1 Identify the needs of OLMCs	Cross-Canada official languages consultations 2016	April 2017	Senior Director, Policy and Research
1.2 Identify key programs and initiatives	Next federal official languages plan	April 2018	Senior Director, Policy and Research

Date of full implementation

April 2018

Recommendation 2 - Reinforcement of management by results

The evaluation finds that there are no indicators and no data collection for shared results by pillar or for the ultimate result. There is also little complementarity between initiatives and only a few federal partners are of the opinion that the Roadmap has a broader impact than the sum of its parts. The grouping of initiatives by pillar meant to allow partners to work to achieve shared results, but the current structure is actually more conducive to work in silos.

Recommendation

As part of the next multi-year official languages plan, it is recommended that the Assistant Deputy Minister, Citizenship, Heritage and Regions:

1. better define the shared horizontal results and indicators that will demonstrate that the next multi-year official languages plan has a broader impact than the sum of its parts.

Management response

Recommendation 2: accepted.

The OLB agrees that, although the Roadmap's logic model has horizontal strategic outcomes and a performance measurement framework that supports its effective management, the current results and accountability management structure and the next federal action plan would benefit from a better results framework and more specific and easily measurable performance indicators.

As part of the implementation of "deliverology" and the new Policy on Results, the OLB will ensure that the new departmental results framework is better designed. In addition, the new federal official languages action plan will be developed in accordance with these same requirements so that shared horizontal outcomes and indicators can be better defined.

Action plan

Action	Deliverables	Timetable	Program sponsor
1.1 Revise the departmental results framework for the next federal official languages plan	New federal official languages strategic action plan	April 2018	Senior Director, Policy and Research

Date of full implementation

April 2018

Recommendation 3 - Reinforcement of management by results

The evaluation noted a lack of detailed data to better identify certain needs for intervention (related to workforce development, regarding promising sectors in terms of economic development, the needs of OLMCs in rural or remote areas, and to better identify health needs). It also remains difficult to measure the results of multiple Roadmap interventions in OLMCs over a longer period.

Recommendation

To support the development of public policies and programs, the adaptation of interventions and decision-making based on evidence, it is recommended that the Assistant Deputy Minister, Citizenship, Heritage and Regions:

1. conduct ongoing research for and collection of data to support the initiatives of the official languages plan; and
2. conduct broader research (such as a net impact assessment of investment in the development of OLMCs, or a post-censal survey regarding the vitality of OLMCs).

Management response

Recommendation 3: accepted.

The development of good public policies and programs requires detailed and often multidisciplinary data that the OLB does not always have, which is why it is necessary to conduct a variety of research, studies and collection of data on official languages.

Research is also one of the methods used by the OLB to obtain data, and the Department will continue to support it and make use of it.

Moreover, the Government of Canada's new Policy on Results emphasizes the importance of associating program management and the achievement of results with evidence gathering. The new federal official languages action plan will go forward along the same lines, and its performance measures will be subject to indicators that have been further developed and will require data collection using different research instruments.

The OLB has a long tradition of data collection, sharing and analysis related to the vitality of its official-language communities. The publication of the 2016 Census language data in August 2017 will give us the opportunity to update a series of statistical profiles and tools that will provide better context for government action.

The OLB also plans to conduct a longitudinal impact study of Official Languages Support Programs and their effect on community development and the promotion of linguistic duality.

Action plan

Action	Deliverables	Timetable	Program sponsor
1.1 Integrate language data from the 2016 census in statistical profiles and tools. Share data with federal partners and communities.	Production of statistical profiles	November 2017	Senior Director, Policy and Research
1.2 Take research into account when developing the new federal official languages action plan	New federal official languages action plan	April 2018	Senior Director, Policy and Research
1.3 Conduct a longitudinal impact study of Official Languages Support Programs	Final report of the study	November 2019	Senior Director, Policy and Research

Date of full implementation

Fall 2019

Recommendation 4 - Reinforcement of management by results

The evaluation notes that, over the five-year plans and following the various resulting groupings of initiatives, the Roadmap has contributed to confusion that has lasted for several years between existing programming and new initiatives or new funds.

Recommendation

It is recommended that the Assistant Deputy Minister, Citizenship, Heritage and Regions:

1. implement a monitoring mechanism to distinguish new investments resulting from the next federal official languages strategy in order to link them to results, to better address the government's priorities in terms of results and transparency, and to comply with the 2016 Policy on Results requirements.

Management response

Recommendation 4: accepted.

The OLB is aware that the two previous Roadmaps did not clearly distinguish between new investments and their objectives and the investments and the objectives of existing programming, which made it difficult (if not impossible) to separate the results related to different investments/initiatives.

The new federal official languages action plan will be developed in accordance with the new Policy on Results requirements, and so as to clearly establish objectives and expected results for the new investments resulting from this new plan.

Action plan

Action	Deliverables	Timetable	Program sponsor
1.1 Develop the next federal official languages action plan so as to clearly establish objectives and expected results for new investments resulting from the next federal official languages plan.	New federal official languages action plan	April 2018	Senior Director, Policy and Research

Date of full implementation

April 2018

Official Languages Coordination Program – part of the horizontal coordination of the Roadmap for Canada's official languages 2013-2018: education, immigration, communities

Recommendation 1 - Better align the governance of official languages

Acknowledging that this evaluation and the reflection stemming from PCH's review of the governance of official languages both suggest that there is still a need to clarify roles and responsibilities, as well as ministerial accountability regarding official languages:

Recommendation

It is recommended that the Assistant Deputy Minister, Citizenship, Heritage and Regions (in cooperation with the appropriate departments):

1. take the measures needed to better align actions, mechanisms and horizontal governance tools for official languages at the federal level, including those that are related to the horizontal coordination of the future federal official languages strategy;
2. update the 2003 Accountability and Coordination Framework for Official Languages; and
3. communicate to all stakeholders the measures put forth.

Those measures should help to better align the governance of official languages, particularly by improving the decision-making process and by clarifying ministerial roles and responsibilities.

Management response

Recommendation 1: accepted.

The OLB agrees that there is still some confusion about roles and responsibilities, as well as ministerial accountability regarding official languages. Measures are also needed to better align actions, mechanisms and horizontal governance tools as well as the accountability and coordination framework for official languages.

The Accountability and Coordination Framework of the Roadmap 2013-2018 was made public in fall 2016 to share information about the coordination mechanisms and the roles and responsibilities of various stakeholders.

The overall coordination of official languages will be reviewed within the context of the development of the next federal official languages action plan. The publication of the next plan will provide the opportunity to communicate clearly with all stakeholders in regard to the related accountability and coordination mechanisms.

Action plan

Action	Deliverables	Timetable	Program sponsor
1.1 Coordination measures taken within the framework of the next official languages plan and communication with all stakeholders	New federal official languages action plan	April 2018	Senior Director, Policy and Research

Date of full implementation

April 2018

Recommendation 2 - Improve reporting and communication

Simplified data collection regarding partners' progress and results appears to be working well. However, data to demonstrate efficiency and economics of the horizontal initiative are limited and, according to some partners, there is not enough exchange of information regarding results, at least not detailed information.

Recommendation

To contribute to improved reporting to PCH, to the sharing of information among partners, and ultimately to the demonstration of progress toward the broad results of the federal strategy, it is recommended that the Assistant Deputy Minister, Citizenship, Heritage and Regions:

1. establish measures and tools to improve the capacity of partners to report on their contribution to horizontal results and their efficiency.

Management response

Recommendation 2: accepted.

We agree that the data and exchange of information on outcomes are sometimes limited, and the accountability and information sharing between partners need improvement.

The next federal official languages action plan will be developed in accordance with the new Policy on Results requirements and the OLB will continue to enhance and define measures and tools in order to report on their contribution to horizontal results and their efficiency.

Action plan

Action	Deliverables	Timetable	Program sponsor
1.1 Review the design of the new federal action plan for official languages	New federal official languages action plan	April 2018	Senior Director, Policy and Research

Date of full implementation

April 2018

Recommendation 3 - Improve reporting and communication

The evaluation notes that there is no data on the effectiveness of the coordination function. This aspect is not systematically documented. Moreover, it is impossible to clearly identify the portion of financial and human resources dedicated specifically to the horizontal coordination of the Roadmap compared with the Official Languages Coordination Program. An analysis of the optimization of resources is therefore impossible.

Recommendation

It is recommended that the Assistant Deputy Minister, Citizenship, Heritage and Regions:

1. implement a mechanism for monitoring the human and financial resources dedicated to horizontal coordination for the next federal official languages plan in order to link the expected results and be able to meet the requirements of the 2016 Policy on Results.

Management response

Recommendation 3: accepted.

The OLB recognizes that it is difficult to clearly identify the portion of financial and human resources dedicated specifically to the part of the Horizontal Coordination of the Roadmap compared with the Official Languages Coordination Program.

The OLB has responsibilities in regard to the development of policies and horizontal coordination that involve both the Department of Canadian Heritage under Part VII of the Act, and the coordination of the horizontal strategy for official languages. These are two components of the Official Languages Coordination Program.

Since the merger of the Official Languages Support Programs Branch and the Official Languages Secretariat (as part of the Deficit Reduction Action Plan - DRAP), the two coordination functions have been combined: the same teams, people, accountability mechanisms and budgets are used for the two components. This measure has resulted in improved accountability and economy, as well as significant gains in efficiency.

In the event that additional financial and human resources are provided for this purpose, the OLB will see to the implementation of an accountability mechanism that clearly separates the resources related to the horizontal coordination of the future federal action plan for official languages and those dedicated to the coordination of official languages. Without these resources, the OLB will continue with the current practice.

Action plan

Action	Deliverables	Timetable	Program sponsor
1.1 In the eventuality that additional financial and human resources are allocated for this purpose, the OLB would establish an accountability mechanism that clearly separates resources for horizontal coordination from the future federal action plan in official languages and those devoted to the coordination of official languages. In the absence of such resources, OLB will continue the current practice.	N/A	N/A	N/A

Date of full implementation

N/A

Recommendation 4 - Improve reporting and communication

Stakeholders such as OLMC representatives and the Commissioner of Official Languages perceive a lack of sufficient communication aimed at Canadians:

Recommendation

It is recommended that the Assistant Deputy Minister, Citizenship, Heritage and Regions:

1. adopt other appropriate means of communicating more detailed information on expected results in order to address the government's priorities regarding results and transparency and the information needs of numerous parties interested in the results and impact of the future federal official languages strategy.

Management response

Recommendation 4: accepted.

The OLB notes a perceived lack of sufficient communication aimed at Canadians and sees that this needs to be addressed.

The OLB already communicates information about results and achievements of the Roadmap 2013-2018 in the Annual Report on Official Languages and in the Canadian Heritage Departmental Performance Report. These reports will be subject to continuous improvement in view of the next federal official languages plan and the new departmental results framework.

Action plan

Action	Deliverables	Timetable	Program sponsor
1.1 Improve the information in the online table of the new federal action plan for official languages	Online table of the Canadian Heritage Departmental Performance Report 2018‑2019	April 2019	Director, Interdepartmental Relations and Accountability
1.2 Improve accountability of the new federal action plan for official languages in the Canadian Heritage Official Languages Annual Report	Canadian Heritage Official Languages Annual Report 2018-2019	March 2020	Director, Interdepartmental Relations and Accountability

Date of full implementation

Spring 2020

Introduction

This report serves two purposes. It presents the results of two interrelated programs, namely the Roadmap for Canada's Official Languages 2013-2018: Education, Immigration, Communities, and the Official Languages Coordination Program – part of the Horizontal Coordination of the Roadmap for Canada's Official Languages 2013-2018: Education, Immigration, Communities .

Structure of the report

Chapter 1 covers the horizontal evaluation of the Roadmap for Canada's Official Languages 2013-2018, while Chapter 2 covers the evaluation of the Official Languages Coordination Program – part of the Horizontal Coordination of the Roadmap for Canada's Official Languages 2013-2018: Education, Immigration, Communities.

Each of these main sections contains sub-sections that provide:

• the scope and objectives of the evaluation;
• a brief description of the program (its structure, objectives and outcomes, governance and financial resources);
• the methodology for the evaluation and its limitations;
• the main findings of the evaluation, namely those on the relevance, performance/effectiveness, and efficiency and economy;
• conclusions; and
• recommendations and management response.

The report concludes with a looking forward section.

1. Chapter 1: horizontal evaluation of the Roadmap for Canada's official languages 2013-2018: education, immigration, communities

This chapter presents the results of the Horizontal Evaluation of the Roadmap for Canada's Official Languages 2013-2018 (hereinafter called the Roadmap). The evaluation covers the period from 2013-2014 to 2015-2016.

The Roadmap is a horizontal initiative representing an investment of $1.12 billion over five years. Of this investment, $886.9 million is permanent funding and $237.1 million is renewed and/or new funding. The funds are intended to support Canadian Heritage (PCH) and 13 other federal agencies and departments in the implementation of 28 official languages initiatives in the fields of education, immigration and community development. The Official Languages Coordination Program – part of the Horizontal Coordination of the Roadmap, which coordinates the work of the Roadmap partners, was also evaluated and is the subject of a separate chapter.

Evaluation objectives

The Horizontal Evaluation of the Roadmap was undertaken within the parameters established by the Policy on Results of the Treasury Board of Canada Secretariat (TBS)^{Footnote a}. In accordance with this policy, most of the 28 Roadmap initiatives must be the subject of a specific evaluation.^{Footnote b} This horizontal evaluation, therefore, focuses on the comprehensiveness of the Roadmap and addresses its relevance in relation to the needs of Canadians, its alignment with the priorities, roles and responsibilities of the Government of Canada, the achievement of its overall outcomes and its efficiency. To avoid duplication in the collection and analysis of data, the horizontal evaluation of the Roadmap is based in part on partner evaluation products and on the evaluation of the Coordination Program – part of the Horizontal Coordination of the Roadmap. It will also serve to inform the renewal of the next multi-year official languages plan.

1.1 Program description

1.1.1 Background and context

The Roadmap 2013-2018 is the third edition of the Government of Canada's horizontal initiative on official languages. It brings together 14 partner federal institutions, including PCH, that are implementing 28 initiatives grouped into three pillars that target three general outcomes, which all contribute to the achievement of an ultimate outcome.

The federal partners in the Roadmap are:

• Western Economic Diversification Canada (WD)
• Canadian Northern Economic Development Agency (CanNor)
• Atlantic Canada Opportunities Agency (ACOA)
• Federal Economic Development Agency of Southern Ontario (FedDev)
• Immigration, Refugees and Citizenship Canada (IRCC)
• Canada Council for the Arts (CCA)
• National Research Council Canada (NRCC)
• Canada Economic Development for Quebec Regions (CED-QC)
• Innovation, Science and Economic Development Canada (ISEDC) & Federal Economic Development Initiative for Northern Ontario (FedNor)
• Justice Canada (JUS)
• Canadian Heritage (PCH)
• Employment and Social Development Canada (ESDC)
• Health Canada (HC)
• Public Services and Procurement Canada (PSPC)

The diagram below (Figure 1) illustrates this horizontal initiative.

Figure 1: architecture of the roadmap 2013-2018

1.1.2 Objectives and outcomes

As indicated above (Figure 1), the ultimate outcome of the Roadmap is to help "Canadians live and thrive in both official languages and recognize the importance of French and English for Canada's national identity, development and prosperity".

To achieve the ultimate outcome described above, the Roadmap is structured around three pillars: education, immigration and communities. Each pillar aims to achieve a general outcome through a series of initiatives.

• Education: Canadians benefit from education and training opportunities in their first official language, from learning the other official language of the country and from access to technological tools, taking advantage of the many economic, cultural and national identity advantages resulting from these.

Table 1: education pillar initiatives

Initiative	Institution	Deliverables
Support for minority-language education	PCH	Offer of provincial and territorial programs and activities to provide education in the language of official language minority communities (OLMCs).
Support for second-language learning	PCH	Offer of provincial and territorial programs and activities related to learning English and French as second official languages.
Summer language bursaries	PCH	Offer of summer language bursaries.
Official language monitors	PCH	Offer of language monitors.
Exchanges Canada	PCH	Young participants enhance their knowledge and understanding of Canada. Young participants connect and create linkages with one another. Young participants enhance their appreciation of the diversity and shared aspects of the Canadian experience.
Language Portal of Canada	PSPC	Canadians access the Language Portal of Canada and language resources in both official languages.
Strengthening the language industry and technologies	NRCC	Contribute to the growth and competitiveness of the Canadian language industry and other Canadian industries through research and development.
Training, networks and access to justice services (Education component)	JUS	Justice system stakeholders are able to respond to the queries of offenders in their official language, where allowed by law.
Training, networks and access to health services (Education component)	HC	Increased presence of bilingual health professionals across Canada. Enhancement of the expertise of bilingual health professionals, especially in areas with the greatest need. Health networks are maintained, enhanced, and engaged in building capacity to effect change in the health care system to improve access to health services within their communities. Bilingual health professionals are encouraged to practice in communities of greatest need and are engaged in providing services in the second official language at first line points of service in the health care system.

Source: Canadian Heritage. (2015). Horizontal Evaluation of the Roadmap for Canada's Official Languages 2013-2018: Education, Immigration, Communities. Framework. October 27, 2015.

• Immigration: Newcomers' language skills in either official language are strengthened and enable them to further contribute to Canada's economic, social and cultural development.

Table 2: immigration pillar initiatives

Initiative	Institution	Deliverables
Immigration to OLMCs	IRCC	A high proportion of French-speaking economic immigrants settle in Francophone minority communities (FMC). (Contribution to Strategic Outcome #1 of the IRCC's Program Alignment Architecture [PAA]) Clients (French-speaking immigrants) make informed decisions about life in Canada, enjoy their rights and assume their responsibilities in Canadian society. Clients (French-speaking immigrants) make use of their knowledge of official languages to function and participate in Canadian society. Clients (French-speaking immigrants) participate in networks, the local labour market and the wider community.
Language training for economic immigrants	IRCC	Clients (economic immigrants) make use of their knowledge of the official languages to function and participate in Canadian society.

Source: Canadian Heritage. (2015). Horizontal Evaluation of the Roadmap for Canada's Official Languages 2013-2018: Education, Immigration, Communities. Framework. October 27, 2015.

• Communities: The vitality of both official languages and the communities that embody them is increased, enabling them to contribute fully to Canadian society, and to Canada's history, national identity, development and prosperity.

Table 3: communities pillar initiatives

Initiative	Institution	Deliverables
Intergovernmental cooperation	PCH	Offer of provincial, territorial and municipal government services in the minority language in fields other than education.
Support to OLMCs	PCH	Offer of activities and services to OLMCs by community organizations.
National Translation Program for Book Publishing	PCH	Help Canadian publishers translate the works of Canadian authors from one official language to another. Foster new collaboration between English- and French-language publishers in Canada. Continue to offer better access to the cultural wealth of the country's Anglophone and Francophone communities.
Community Cultural Action Fund	PCH	Provision of activities and services to strengthen and share the cultural, artistic and heritage activities of OLMCs.
Music Showcases	PCH	Provide OLMCs with better access to music showcases in their language; and give artists from these communities an opportunity for exposure to a wider audience. Aim to maintain the number of music showcase presentations at 400 annually and the number of artists supported at 200.
Market Access Strategies	CCA	Facilitate greater circulation of artists from OLMCs in various regions of Canada and abroad, and increase the number of distribution channels available to them. Promote greater access to the wealth and variety of OLMC arts and culture by Canadians across the country. Support the development of new markets for artists and art organizations in OLMCs. Ensure wider visibility of the works, artists and art organizations of OLMCs in a range of artistic disciplines such as dance, integrated arts, media arts, music, theatre, visual arts and writing and publishing.
Contraventions Act Fund	JUS	Canadians have increased access to judicial and extrajudicial services related to federal offences in the official language of their choice.
Training, networks and access to justice services (Communities pillar)	JUS	Canadians from official language minority communities have access to legal information services in their official language of choice.
Training, networks and access to health services (Communities pillar)	HC	Health networks: The networks continue to operate like community entities. OLMCs are actively building their capacity to influence change in the health care system in order to improve access to health services in their communities. The networks are optimized as part of target budgets, in line with provincial/territorial priorities and community needs. Access to health and retention services: Bilingual health professionals are encouraged to practise their profession in communities with the greatest need. Health care providers ensure that they offer services in the second official language at service points such as pharmacies, doctor's offices and clinics. OLMCs support the human resources retention strategies at their health facilities. Focus is on resolving health concerns specific to Anglophone and Francophone OLMC demographics (e.g., seniors, mental health).
Social Partnership Initiative in OLMCs	ESDC	Activities by OLMCs to address complex social issues such as early childhood development or the living conditions of poor families. A greater number and diversity of intersectoral partnerships that provide value for money and expand the scope of the initiative through the additional resources brought in by the new stakeholders. Improved dissemination and sharing of best practices, knowledge and research.
OLMC Literacy and Essential Skills Initiative (LES)	ESDC	Partners, stakeholders and employers who have a better understanding of the possibilities and benefits related to LES programs. Greater capacity of partners, stakeholders and employers. Improved dissemination, transfer and application of knowledge and information among partners, stakeholders and employers.
Enabling Fund for OLMCs	ESDC	Communities have plans, projects and services that are consistent with their priorities and have the capacity to achieve identified goals. Effective collaboration across/among groups. Resources are mobilized for responsive initiatives that align with OLMC priorities. OLMCs are empowered to self-assess progress and make informed decisions.
Economic Development Initiative (EDI)	ISEDC (including FedNor) CED ACOA FedDev WD CanNor	Business and community development. Capacities, new expertise and partnerships developed. Better understanding of the economic issues of OLMCs.

Source: Canadian Heritage. (2015). Horizontal Evaluation of the Roadmap for Canada's Official Languages 2013-2018: Education, Immigration, Communities Framework. October 27, 2015.

1.1.3 Governance

Roadmap governance is ultimately the responsibility of the Minister responsible for Official Languages^{Footnote c} (currently a Canadian Heritage responsibility). The Minister also receives support for the Roadmap from three governing bodies with a mandate involving federal horizontal governance of official languages:

• The Committee of Assistant Deputy Ministers on Official Languages (CADMOL) is an executive forum for discussing and preparing advice that affects the Roadmap's strategic directions, implementation, accountability, performance and risk management, and the evaluation of progress and results achieved by partners, to gain an understanding of the federal strategy overall. CADMOL reports to the Deputy Minister of PCH.
• The CADMOL Executive Sub-Committee (Ex-CADMOL) acts as a governance council on the strategic direction of official languages issues, including the Roadmap. It enables the sharing or development of strategies for dealing with official languages issues that come up within government (horizontal issues) and strengthening collective leadership.
• The Official Languages Directors General Forum is responsible for reviewing, supporting, approving and providing leadership on the development of strategic policy and for Roadmap-related issues, in addition to ensuring horizontal coordination of official languages initiatives. It reports to the Ex-CADMOL.

Horizontal coordination of the Roadmap is the responsibility of the PCH Official Languages Branch (OLB). In particular, the OLB has a government-wide official languages coordination mandate for Part IV of the OLA that encompasses strategic policy and activities such as planning, partnership building, awareness-raising and accountability of federal institutions. The Horizontal Coordination of the Roadmap implementation was added to this government-wide mandate in 2013; previously it was under the responsibility of Official Languages Secretariat (OLS). It was merged with the Official Languages Support Programs (OLSP) Branch, which became the OLB. Previously, the OLS was located within the Privy Council Office.

1.1.4 Program resources

The Roadmap 2013-2018 represents an investment of $1.1 billion over five years. As previously indicated, of this investment, $886.9 million is permanent funding and $237.1 million is renewed and/or new funding (see Annex B, Table2). The envelope includes $29.9 million earmarked for official languages coordination and governance duties^{Footnote d} (PCH: $10.4 million [Annex B, Table 1]; TBS: $17 million; JUS: $2.5 million), as well as the $1.1 billion invested directly in the 28 initiatives (Annex B, Table 2). Actual spending for the first two years of the Roadmap are presented by initiative in Annex B, Table 3 and by pillar in Table 4.

1.2 Evaluation methodology

This section presents the methodology used for this evaluation, which included multiple lines of evidence. Annex A contains the evaluation matrix structured by evaluation question and indicators, as well as the proposed data sources.

The horizontal evaluation was based on the following data sources:

• The documentation produced by PCH and Roadmap partners and stakeholders, including partners' evaluation products;
• Quantitative data from censuses, surveys, studies or record analysis;
• Independent literature on Roadmap-related issues;
• A comparative analysis of past surveys (2005, 2012 and 2016);
• The opinions and perceptions of PCH managers working on the horizontal management of the Roadmap, collected during interviews;
• The opinions and perceptions of Roadmap partner institutions collected during interviews;
• The opinions and perceptions of Roadmap stakeholders collected during interviews (other federal institutions, Commissioner of Official Languages, provincial and territorial representatives, OLMC spokespersons, linguistic duality groups, the language industry, representatives of official language majorities);
• Expert opinions collected within the framework of panels;
• Public opinion gathered through a survey administered to Canadians or some population segments in 2016; and a comparative analysis of official languages surveys in 2005, 2012 and 2016;
• The opinions and perceptions of official language minority communities (OLMCs) gathered from case studies in some communities.

1.2.1 Document review

The document review collected and analyzed documentation relevant to the Roadmap evaluation according to the established evaluation matrix. Document types included:

• Speeches from the Throne, Budget announcements and other official documentation;
• A report of consultations with the Minister of Official Languages in 2012;
• Internal strategic documents;
• Reports on Plans and Priorities (RPP) and Departmental Performance Reports (DPR), and the input provided by partner departments and agencies for this purpose;
• PCH annual reports on official languages;
• Parliamentary reports on official languages;
• Reports and studies from the Office of the Commissioner of Official Languages (OCOL);
• Previous audit reports and follow-up on the recommendations of the previous evaluation;
• The minutes of interdepartmental governance committees meetings;
• Program budgets and expenditures;
• Documents and administrative data of the three PCH Roadmap initiatives that are not part of the Official Languages Support Programs (OLSPs) (Exchanges Canada, Music Showcases, National Translation Program for Book Publishing);
• The evaluation products of Roadmap partners received during the period covered by the document review.

The software program NVivo was selected for the document analysis. Coding was used to organize relevant information according to the indicators in the evaluation matrix.

1.2.2 Interviews with key stakeholders

The 33 interviews conducted with key stakeholders collected opinions and perceptions about the relevance, design and implementation, as well as the effectiveness and efficiency of the Roadmap. The interviews also introduced factual elements that facilitated a more in-depth review of certain issues using other data sources. Key stakeholders included Roadmap coordinators and managers (n=3), federal partner institutions (n=20) and other official languages stakeholders (n=10).

The NVivo software was also useful in the analysis of interviews.

1.2.3 Literature review

The literature review conducted and drafted by the Canadian Heritage Research Group (PRG) contributed to identify:

• Lessons learned from public policy using horizontal approaches;
• The nature of Canadians' needs and OLMC priorities;
• The relevance of the various official languages programs/initiatives with respect to these needs and priorities; and
• The issues and outcomes in:
  • education and training in the minority language;
  • minority-language learning;
  • immigration and official language learning among immigrants;
  • strengthening the vitality of OLMCs; and
  • recognition of linguistic duality by the linguistic majorities.

The review was supported by findings collected from other data sources.

 1.2.4 2016 public opinion survey on official languages

A public opinion survey to better understand the perceptions and attitudes of Canadians towards both official languages was administered across the Canadian population (n=1,501) from March 26 to April 27, 2016 by an external firm on behalf of the Department of Canadian Heritage. The results of this survey have been incorporated into the data triangulation.

1.2.5 Comparative analysis of existing surveys (2005, 2012 & 2016)

The analysis of existing surveys sought to extract data that provides insight on perceptions of linguistic duality, language vitality or language learning, official languages policies and programs and actions by the community or private sector in these areas. The results of the comparative analysis of Canadian public opinion trends with respect to official languages according to previous surveys have been incorporated into the data triangulation.

1.2.6 OLMC case studies

The case studies made it possible to gauge the vitality of certain OLMCs that are representative of OLMC diversity in Canada. When evaluating the previous Roadmap in 2012, the case study method was selected to evaluate vitality, and it was used again in 2016, except that there were no field visits. Nine OLMCs were selected from across Canada for the case studies: Surrey (BC) Gravelbourg (SK), Timmins (ON), London (ON), Pontiac (QC), Beaconsfield (QC) New Carlisle (QC), Bathurst (NB) and Summerside (PEI).

Each case study included telephone interviews with an array of key community stakeholders (on average, seven) from the community (representatives of target groups, service providers, and reputable local observers), and a review of relevant documents in order to address as many vitality indicators as possible developed in 2012 and updated in 2016 according to the most recent literature on the subject. Each stakeholder addressed the topics that were most relevant to them from the following:

• Linguistic and Socio-demographic Profile
• Government Services
• Education
• Health and Social Services
• Justice
• Economy
• Immigration
• Arts, Culture, Communications and Heritage
• Linguistic Landscape
• Governance and Community Participation
• Relationship with the Majority
• Openness
• Future.

1.2.7 Expert panels

Two expert panels composed of individuals from outside Roadmap partner and beneficiary circles were held. The first was conducted on an exploratory basis at the beginning of the evaluation period on the design and implementation of the Roadmap, to support framework development. The second was conducted at the end of the evaluation period on the Roadmap overall. Regarding the Roadmap overall, the experts examined trends that evaluators had observed in the data, in addition to correlations and benchmarks that raised preliminary findings related to the Roadmap's mandate. They also examined potential solutions for resolving some of the difficulties observed. This exercise helped to further analyze and interpret the results of the evaluation in order to draw conclusions.

1.2.8 Limitations and constraints

It is difficult to measure, in concrete terms, the results of multiple interventions under the Roadmap compared with the ultimate outcome, and some factors contribute to that problem: the evaluation is conducted at about the halfway mark, there are delays in submitting initiative evaluation products, and the nature of the funded activities that give visible long-term results. The evaluation is being conducted almost halfway through the five-year horizontal initiative, which limits the available data set. In general, the available documentation underscores the activities carried out as well as partner initiative outputs up to 2014-2015; it does not cover the 2015-2016 year. In most cases, the documentation does not refer to advancement towards or achievement of the expected outcomes. Moreover, some of the partners' evaluation products will not be available until later in the Roadmap cycle, whereas those which became available during the evaluation were included, even if they contained only preliminary findings. The evaluation also points to the lack of performance indicators to support the measurement of progress towards common outcomes per pillar and the ultimate outcome of the Roadmap which is another major constraint.

Interviews with key stakeholders and expert panels provided further potential answers to questions about the expected outcomes of the three pillars of the Roadmap in general. However, some stakeholders limited themselves to contributing to one or another specific initiative out of the 28, and some indicated that it is indeed too early to comment on the progress towards expected outcomes, since the current Roadmap has only been around for three years. Furthermore, findings from the interviews are based in part on the opinions of people with a direct interest in the Roadmap or one of the 28 initiatives, and may therefore be biased in favour of the Roadmap. To a lesser extent, this also applies to the expert panels. However, the adopted approach minimizes potential bias because the interviews and panels were conducted by qualified evaluators, and respondents were asked to justify their perceptions and provide examples, with supporting study results, and so on.

The analysis of indicators relating to the nine OLMCs selected for the case studies is based on census data from 2011. These data are still comparable with 2006 data, and set the backdrop for the change in vitality between 2012 and 2016 that the evaluators were able to explore during the interviews.

The comparative analysis of the results of existing public opinion survey allowed for some comparison of the appreciation and perception of both official languages in 2005, 2012 and 2016. This provided background for the interpretation of the findings that emerged from other sources, in particular with respect to relevance, and more particularly the need and interest vis-a-vis the type of interventions selected (for example, with regard to learning the second official language). However, it is not possible to determine the role that government intervention may have had on the change in these perceptions.

Due to the low volume of literature in fields related to the nature of the Roadmap, as expected, very little new relevant literature has been identified in the four years since the exhaustive literature review for the evaluation of the previous Roadmap. Consequently, the review offers few new elements but still provides more information.

1.3 Findings ‒ relevance

This section presents the main findings on the relevance of the Roadmap, namely its alignment with the needs of Canadians, and alignment with federal priorities, roles and responsibilities.

Summary

Overall, the evaluation indicates that the current Roadmap addresses several ongoing OLMC needs, but it must continue to adapt to the demographic, social and economic context. To better address these needs, increased support is required in the areas of early childhood and youth, seniors, integration and retention of newcomers, and minority media.

There is also a lack of detailed data that would help to better determine health care needs, workforce development needs, promising sectors for economic development, and the needs of OLMCs in rural or remote areas.

Section 42 of Part VII of the OLA gives the PCH minister, in consultation with the other federal ministers, the mission of initiating and encouraging coordination of the implementation by federal institutions of section 41 duties. The Roadmap 2013-2018 is aligned with these roles and responsibilities. In general, managers, partners and stakeholders believe that the Roadmap is indeed in alignment with federal roles and responsibilities under the OLA. However, they highlight that it focuses on the responsibilities under Part VII of the OLA, rather than Parts IV, V and VI.

The majority of partners indicate that the Roadmap is in alignment with the priorities of their department or agency, or at least that their initiative aligns well with those priorities.

1.3.1 Core Issue 1: continued need for the program

Demonstrable, well-established needs

The historical evolution over time of the horizontal five-year plans for official languages is a testament to the continued need for emphasis and a special focus on Canada's linguistic duality and OLMC development. The focus areas of the three five-year plans have changed slightly, most recently with the addition of immigration as its own focus area.

The first five-year plan, the Action Plan for Official Languages 2003-2008, comprised two components: an initial Accountability and Coordination Framework to increase federal institutional awareness of the provisions of the Official Languages Act (OLA) (see Annex D) and coordinate government official language processes; and a new $751.4 million investment over five years targeting four areas: education, community development, public services and language industries.^Footnote1

The Roadmap 2008-2013 was a $1.1 billion investment over five years aimed at improving and expanding government action^{Footnote e} to multiply the benefits of linguistic duality and make them available to all Canadians. This plan was designed around two pillars: promotion of linguistic duality and support for OLMC development.^Footnote2 In 2012, the study of the Roadmap 2008-2013 by the House Standing Committee on Official Languages confirmed the continued need for and success of this initiative,^Footnote3 and the evaluation of the Roadmap 2008-2013 emphasized that it is "recognized today for the public brand it provides for the Government of Canada's language policy" and clarified that "this image was built in the sequence that included the Action Plan 2003-2008 and the Roadmap 2008-2013".^Footnote4 The image of the current Roadmap was built in consequence of the Action Plan of 2003-2008 and the Roadmap 2008-2013.

Several consultations have been held since 2012 to continue monitoring the evolving official language needs and aspirations of Canadians. The consultations also contributed to the design of the Roadmap 2013-2018, which served as the foundation for the design of the three pillars of the current Roadmap: Education, Immigration and Communities, which bring together the "arts and culture" and "economic development" components.^Footnote5 The CADMOL 2014 and 2015 annual meetings with stakeholders contributed to identifying the change in needs since the implementation of the Roadmap 2013-2018. According to the CADMOL consultations in 2014 and 2015, FMC representatives underscored that there is an ongoing need for adjustment to the demographic, social and economic context: aging population, youth out-migration, integration and retention of newcomers, and sense of belonging.^Footnote6

More recently, during the last wave of OLMC consultations on official languages (2016) for the next multi-year official languages plan, PCH identified four priority areas: immigration; community and school infrastructure; OLMC media in the digital age; and the continuum of educational opportunities in the minority language, from preschool to the post-secondary level.^Footnote7

The survey on the perceptions and attitudes of Canadians towards official languages conducted in 2016 indicates that almost three quarters of the respondents perceive that the Government of Canada is effective in protecting both of Canada's official languages (71% of respondents agree), and little more than half perceive that its policy on official languages reinforces national unity (67% agree).^Footnote8 A comparative analysis of trends in Canadian public opinion of official languages also demonstrated that between 2005 and 2016, Canadians' opinion of official languages fluctuated very little for the majority of the topics covered (see Figure 3, Annex C). During this period, the percentage that agreed with the statement "The fact that there are two official languages is an important part of what it means to be Canadian" rose slightly, from an average of 7.1% in 2005 to 8% in 2016 among Francophones, and 5.5% to 6.7% among Anglophones.^Footnote9 (see Figure 3, Annex C). In short, official languages remain one of the foundations of Canadian identity and are in general favourably perceived.

The priority areas identified during the various consultations, therefore, are in addition to and support the findings from other data sources for the purposes of this evaluation. Given the relatively short amount of time since the 2012 consultations that served as the foundation for the design of the current Roadmap, it is not surprising that there was no significant change in the needs of the minority and of the majority. Furthermore, Roadmap managers estimate that, overall, it addresses these needs and that, in general, needs are ongoing. Besides the continuing needs, there are those related to larger social phenomena such as the pervasiveness of predominantly Anglophone social media, the rapidly aging population and an increase in the proportion of Canadians whose mother tongue is neither English nor French. In the areas of youth, health and justice, the partners confirm that the Roadmap continues to address a demonstrable, well-established need.

Better address needs

According to many of the partners and stakeholders interviewed for the evaluation, to better address needs, increased support is required in the areas of early childhood and youth, seniors, and in the integration and retention of newcomers. Special attention must also be paid to the needs of OLMCs in Quebec. There is also a lack of detailed data that would help to better determine health care needs, workforce development needs in promising sectors for economic development and the needs of OLMCs in rural or remote areas. The evaluation of the previous Roadmap also underscored the need for support for research to better inform interventions. Furthermore, there is a need for increased support to counteract the vulnerability of community media / minority official language media, which was revealed in various recent consultations indicated above. The following subsections summarize the positions of various stakeholders and observers with respect to needs in each of the seven areas where greater support would be desirable to meet emerging needs.

Early childhood

The FMCs have been working hard in recent years to raise the profile of the family support sector.^Footnote10 The 2015 Federation of Francophone and Acadian Communities of Canada (FCFA) report on the Community Strategic Plan 2007-2017 points out that the Roadmap 2008-2013 had earmarked approximately $4 million over five years for early childhood services, but that the Roadmap 2013-2018 has earmarked next to nothing, even though this is an important sector.^Footnote11 Experts agree that support for early childhood is low, sometimes absent altogether, highlighting the importance of this service for language development, culture and identity. Within the framework of the parliamentary report on the new Action Plan for Official Languages, several witnesses are also of the opinion that "early childhood is the foundation for the future of Canada's Francophone communities," stating that "it is the stage at which a sense of identity is formed and a feeling of belonging to the Francophone community develops".^Footnote12 A report by the COL recently published on this topic also shares this opinion. In the report, the COL recommends that the federal government support and set aside sufficient funds for early childhood initiatives, which includes educator training, raising awareness among parents and service providers and improving infrastructure.^Footnote13 In its recent report, the House of Commons Standing Committee on Official Languages maintains that funds allocated by the government to these services in OLMCs should be incorporated in the next tripartite memorandum of understanding on minority-language education.^Footnote14

Seniors / aging population

Stakeholders, case study participants and experts note that there is a total absence or lack of interventions that focus on the needs of seniors, both in terms of health and OLMC vitality. According to experts, seniors are an important resource across the community, since they are often more active in their communities. The need for interventions adapted to seniors is also connected to the lack of detailed data required to better determine health care needs. The Standing Committee on Official Languages emphasized that "data on the health of FMCs that is needed for research and informed decision-making are non-existent or unavailable,"^Footnote15 and that as indicated by FMCs, given that the Francophone population is aging twice as fast as the Anglophone majority population, there is a growing need for services offered in the minority language.^Footnote16

Integration and retention of newcomers

Given that FMCs are facing rural out-migration and low birth rates, there is a growing need for the settlement of French-speaking immigrants in these communities to fill jobs and contribute to community vitality.^Footnote17 According to the partners interviewed, existing immigration mechanisms can contribute to meet the need of some OLMCs for growing the population. However, workforce development needs are not always properly identified due to a lack of detailed data. Furthermore, one of the major constraints mentioned by the experts is support for the integration of newcomers in small FMCs as well as interprovincial mobility and recognition of certain diplomas. The Standing Committee on Official Languages also recently identified associated challenges such as how to make Canadian employers aware of the advantages of hiring Francophone immigrants, preparing immigrants for the realities of the Canadian job market, facilitating international and interprovincial mobility of qualified workers, and supporting temporary workers.^Footnote18 According to a 2015 study by the same Committee, the recognition of foreign diplomas is still an obstacle to the establishment of equivalencies.^Footnote19 Another study on immigration conducted by the COL supports this finding and adds that for French-speaking newcomers who settle in an OLMC outside Quebec, English is an additional challenge in terms of access to and integration into the job market.^Footnote20

Economic and workforce development

According to experts and partners, certain sectors of the OLMC economy have become relatively promising in recent years, and it is important to ensure that efforts continue to be made in fields that have a good chance of remaining promising, even though these fields differ from one OLMC to another. To that end, experts have noted that the manner in which economic development support is structured in the Roadmap presents a challenge to the identification of local priorities. A similar perspective was also shared by OLMCs during consultations for a parliamentary report on economic development in OLMCs. OLMCs generally would like to see the government work with community agencies to establish programs adapted to their specific realities. This would help them to recognize and achieve their full economic potential.^Footnote21 The Economic Development Initiative (EDI) is part of the programs based on economic development and is already working on a regional delivery model.

Support for research

A majority of Canadians (73% to 88%) consider initiatives that protect access to services in both languages, even in a minority community, to be important, indicating that this is a priority need.^Footnote22 However, according to experts, there is a lack of data on official language minorities; there are demographic data, but there is a problem with data on community vitality and access to services in the minority official language, as well as the overall impact of the Roadmap. Furthermore, partners and stakeholders interviewed indicated that there is a need for support for research in specific areas so that interventions can be targeted—in workforce development, economic development, OLMC needs in rural or remote areas, and to better identify needs in health and immigration (see section 1.4.1, Design and implementation)— and more broadly in research to measure the effect of Roadmap investments.

Media

According to experts and representatives of OLMCs, media support should be one of the priority areas for action, especially in light of the controversy surrounding recent service cuts at Radio-Canada and the impact on access to programming in French outside Quebec. As noted by the Standing Committee on Official Languages, "A significant part of federal support for OLMC media is in the form of buying advertising. However, in recent years, the federal government has reduced its advertising spending in traditional media (television, radio and print newspapers) in favour of social media,"^Footnote23 and "this choice has an impact on the survival of community media and on communities' ability to access local and regional information".^Footnote24 It is in this light that OLMC media are requesting support to make the digital shift also mentioned in the Minister's consultations for the next multi-year plan. The FCFA notes, however, that this raises concerns for OLMCs in some localities that have no access to digital media or where this type of service is too expensive.^Footnote25

Needs of english-speaking communities in Quebec

The evaluation indicates that some actions will have to be tweaked to better meet the specific needs of English-speaking communities in Quebec. The expert panel underscored a perceived imbalance between the funds allocated to Anglophone versus Francophone OLMCs. According to some of the stakeholders interviewed and experts consulted, these OLMCs are primarily in rural or remote areas and seem fated to slowly disappear, and the Roadmap seems to not take their needs into account as it does for OLMCs elsewhere in Canada, particularly because immigration is not a solution to the loss of demographic weight, as is the case elsewhere in Canada. The CLO is in agreement, adding that despite the existence of organizations with the capacity to support the integration of English-speaking immigrants, the province did not officially recognize them.^Footnote26 The stakeholders interviewed were, however, divided on the Roadmap's alignment with the needs of English-speaking communities in Quebec. While some stakeholders believe that the Roadmap shows a good understanding of the asymmetrical needs of Francophone and Anglophone minorities, another indicates that the pressing needs of English-speaking communities in Quebec to counter the loss of demographic weight and of institutions such as health facilities are not fully taken into account by the current Roadmap or the previous five-year plans.

1.3.2 Core issue 2: alignment with government priorities

In principle, Roadmap is aligned with the strategic outcome of the federal government that aims to achieve "a diverse society that promotes linguistic duality and social inclusion"^Footnote27 and is clearly aligned with the third strategic priority of PCH, which aims to "connect Canadians through language and culture" and the second PCH strategic outcome, aimed at ensuring that "Canadians share, express and appreciate their Canadian identity."^Footnote28 It should be noted that the Contraventions Act Fund, one of Roadmap initiatives, is an exception and is not related to these strategic outcomes.

The Government of Canada had also renewed its commitment to the Economic Action Plan 2013. The renewal of the Roadmap reaffirmed and solidified the federal government's commitment to official languages, and has been described as an "ongoing commitment to enhance the vitality of official language minority communities in Canada".^Footnote29 The recent Speeches from the Throne also show that official languages remain a priority. This commitment was recognized in 2013 in the context of national identity, in that the government spoke of "a federation in which our two national languages position us uniquely in the world where Francophones thrive and celebrate their unique culture in solidarity with their fellow Canadians".^Footnote30 More recently in 2015, the current government confirmed its commitment to official languages in a context where "diversity is Canada's strength".^Footnote31

The majority of partners also indicate that the Roadmap is well aligned with the priorities of their department or agency, or at least of their initiative. Some indicated that this was not the case or that they were unable to comment on alignment at this point, as they are focusing on their own initiatives.

1.3.3 Core issue 3: alignment with federal roles and responsibilities

First recall that the Government of Canada has special responsibilities under the OLA, (see Annex D) namely to:

• ensure respect for English and French as the official languages of Canada and ensure equality of status and equal rights and privileges as to their use in all federal institutions, in particular with respect to their use in parliamentary proceedings, in legislative and other instruments, in the administration of justice, in communicating with or providing services to the public and in carrying out the work of federal institutions (Parts I to V);
• support the development of English and French linguistic minority communities and in general advance the equality of status and use of the English and French languages within Canadian society (Parts VI and VII);
• set out the powers, duties and functions of federal institutions with respect to the official languages of Canada (Part VIII).^Footnote32

The Roadmap for 2013-2018 falls within some of these objectives, and is also the result of the federal government's intention to keep up its efforts in the framework of the 2008-2013: Acting for the Future. Under Part VII of the OLA, and in particular section 43, the Government of Canada renewed its commitment to promote both official languages and the vitality of OLMCs through the Roadmap for 2013-2018.^Footnote33 As set out in the OLA, all federal institutions have official language obligations. Under section 42 of Part VII, the OLA gives the PCH minister, in consultation with the other federal ministers, the mission of initiating and encouraging coordination of the implementation by federal institutions of their section 41 obligations.^Footnote34 As regards the current Roadmap for 2013-2018, according to the evaluation products available, all initiatives that are related to Part VII comply with section 41, which commits the federal government to "enhancing the vitality of English and French linguistic minority communities in Canada and supporting their development," by taking positive measures. Only one initiative, the Contraventions Act Fund is actually linked to Part IV.

In general, managers, partners and stakeholders believe that the Roadmap does align with the roles and responsibilities of the federal government under the OLA. However, they pointed out that it focuses actually on the responsibilities under Part VII of the OLA, and not on Parts IV, V and VI. Moreover, some stakeholders noted that in recent years, actions in the communities have focused less on the capacity to take charge and more on isolated actions or the provision of services considered priorities by the government, while according to Part VII of the OLA relating to community development, the focus should in fact be on building the capacity of OLMCs to take charge, and to identify their priorities and the appropriate mechanisms.

In addition to coordinating the implementation of the Part VII obligations (promotion of French and English and community development), in practice PCH also assists the Minister with the responsibilities of coordinating official languages for the federal government (inherited following the transfer from OLS to PCH).^Footnote35 However, as explained earlier, ministers are responsible for different parts of the OLA, but none is responsible for all parts of the OLA and the position of Minister responsible for Official Languages, which has not existed since 2015, was without legal basis. Furthermore, the Accountability and Coordination Framework goes back to 2003, and thus certain aspects of the role of coordinating all federal activities on official languages have evolved since then^{Footnote f}, which can be confusing.^Footnote36 This is discussed further in the context of the evaluation of the Coordination Program – part of the Horizontal Coordination of the Roadmap 2013-2018.

1.4 Findings - performance

1.4.1 Core issue 4: achievement of expected outcomes

This section presents the main findings relating first to evaluation questions about the logic of the Roadmap design, and its specific contribution and added value, followed by the achievement of expected outcomes for each of the three pillars and the overall outcome of the Roadmap.

Summary 

Design and implementation

• Despite the clarity of the Roadmap's mandate on paper, it is not so clear in practice, especially with regard to governance and horizontal coordination. (This is discussed in greater detail in the context of the Evaluation of the Coordination Program – part of the Horizontal Coordination of the Roadmap for Canada 2013-2018.)
• Although PCH managers and several federal partners believe that the pillar structure and the sequence of the expected outcomes are logical, some federal partners and external stakeholders believe that the pillar structure instead promotes working in silos. Some experts agree.
• Through the five-year plans and the different grouping of initiatives, the Roadmap has caused some persistent confusion between existing programming, which in some cases was enhanced, and new initiatives or new funds.
• Despite the desire for complementarity in the design of the Roadmap, very few federal partners mentioned a real complementarity between initiatives; most have indicated that they are focusing on their respective initiatives.
• Partners are sharply divided between those who are convinced that they would have achieved the same results without the Roadmap and those who believe that the horizontal approach contributes to a greater impact than the sum of its parts. Stakeholders and experts see the benefits (for example, it provides a window to promote governmental intervention).

Outcomes

Education pillar: The Roadmap continues to contribute to learning opportunities in the minority official language; however, the OLSP evaluation indicates that it remains difficult to measure intermediate and final outcomes arising from the activities funded. The evaluation also highlights the absence of an obligation to offer second-language learning programs, which places special importance on federal investment. With respect to training opportunities in the first official language and second official language learning, the evaluation highlights the significant contribution of Health Canada's Official Languages Health Contribution Program (OLHCP), and the contribution of a similar program in justice. Federal partners also highlighted the Roadmap's contribution to the development of language technologies that support second official language learning.

Immigration pillar: The available documentation primarily covers the activities undertaken, and the level of outcome achievement is difficult to measure. However, the preliminary results of the IRCC's evaluation of the Immigration to OLMCs Initiative provides some indication of progress towards the overall outcome of this pillar. Moreover, federal immigration partners and stakeholders interviewed agree that there are an increased number of specific actions in Francophone immigration, and many stakeholders on the ground have better expertise, which should help the government to achieve its objectives. The preliminary results of the IRCC Immigration to OLMCs initiative evaluation indicates; however, that additional efforts will be required to increase Francophone immigration in OLMCs. While Francophone economic immigrants have settled in FMCs, their relative proportion remains below targets: between 2003 and 2015, the proportion of Francophone permanent residents within the immigrant population averaged 1.5%, which is below the target of 4.4%; and 44% of Francophone immigrants who settled in FMCs were from the economic class, representing about 1.1% of all economic immigration outside Quebec and remains below the target of 4% set by the IRCC- FMCs committee.

Communities pillar: Stakeholders are mostly convinced of the Roadmap's contribution to OLMC vitality, and they provide many examples in health, justice, arts and culture, and economic development. Furthermore, the evaluation products of partners and annual performance measures indicate progress towards the expected outcomes. However, it is difficult to attribute the effects to interventions as part of the Roadmap.

It remains difficult to actually measure the outcomes of many Roadmap interventions compared with the ultimate broad outcome, and certain factors contribute to this problem: the evaluation is taking place at about the halfway point, there are delays in the submission of initiative evaluation products, and the nature of the activities funded that lead to long-term visible outcomes.

Clarity of the mandate

Recall that the Roadmap 2013-2018 comprises a total of 28 initiatives being carried out by 14 different federal institutions including PCH. While they are responsible for the management of their respective initiatives, PCH is responsible for overall horizontal coordination^Footnote37 through the PCH's Official Languages Branch (OLB). The OLB has a pan-governmental mandate of coordinating official languages, which involves strategic policy and planning activities, creation of partnerships, awareness-raising and accountability of federal institutions.

Overall, despite the clarity of the Roadmap's mandate on paper, it is not so clear in practice, especially with regard to its governance and horizontal coordination. Horizontal coordination is one of the primary mechanisms for ensuring that these institutions understand their obligations under the OLA and ensuring their joint commitment to achieving concrete and tangible results for Canadians. Confusion is not uncommon, because the OLB's role relates to the implementation of Part VII of the OLA and not only the Roadmap, even though the Roadmap does have its own governance structure. (This is discussed in greater detail in the context of the Evaluation of the Official Languages Coordination Program – part of the Horizontal Coordination of the Roadmap for Canada's official Languages 2013-2018: education, immigration, communities.)

It is important to note that the horizontal evaluation of the Roadmap in 2012 had recommended that PCH clarify the roles and responsibilities of federal departments and agencies with a mandate involving enforcement of the OLA. In response to this recommendation, PCH organized a governance review in 2013, and it is ongoing. Furthermore, an internal audit in 2015 found that despite best practices in the documentation of discussions, recommendations and actions, nothing specifically demonstrated that CADMOL and Ex-CADMOL were making decisions regarding the Roadmap 2013-2018, as expected in the governance structure.^Footnote38 PCH responded to this finding by indicating that at the time of the audit "a thorough review of official languages governance and coordination, including the Roadmap 2013-2018, was underway in order to simplify and better align the roles of key stakeholders, thereby eliminating any duplication," which would mitigate the perceived risk.^Footnote39 The findings of the review to date are analyzed as part of the Evaluation of the Official Languages Coordination Program – part of the Horizontal Coordination of the Roadmap for Canada's Official languages 2013-2018.

Specific contribution

Any discussion on the design of the current Roadmap naturally makes reference to its similarities and differences with previous plans. As indicated in the brief history above (section 1.1.1), the areas emphasized by the three five-year plans have changed slightly, more recently with the addition of immigration as its own priority focus area. The number of federal partners directly involved grew (from 8 to 15), and the content of the priority areas (or pillars) was also expanded to include more programs (especially existing and some new — up to 32, and recently 22).

In summary, the first Action Plan for Official Languages 2003-2008 involved eight departments and agencies that targeted four areas: education, community development, services to the public, and language industries. As for the Roadmap 2008-2013, it had the commitment of 15 federal departments and agencies working under two pillars: promoting linguistic duality and supporting OLMC development, which included the focus areas of health, justice, immigration, economic development, and arts and culture.^Footnote40 Later, the 2012 consultations for the current Roadmap confirmed the same priority areas, but were grouped into three pillars with a slightly different emphasis: education, immigration and community support, the latter also including the components of "art and culture" and "economic development".^Footnote41 More specifically, to improve the logic of the Roadmap, its architecture is deliberately less detailed than its predecessor. Drawing heavily on the Program Alignment Architecture (PAA) model, it is an attempt to more clearly reflect government priorities and goals in the three main areas of action: education, immigration and community support, and to better demonstrate that the outcomes of various initiatives contribute to achieving the objectives of the pillars, and that partner institutions all have a responsibility in the achievement of the strategic outcome of the Roadmap.^Footnote42

Managers and partners believe that the Roadmap serves as a showcase or a promotional tool for government action on official languages and provides an overall logic, and that for stakeholders (outside government), it represents the government's vision and stresses that official languages is a priority for the government. Some partners also indicated that the Roadmap provides a specific platform for official languages, enables the expansion of programs already in place, that it is an effort by government to prioritize, and increases government interaction with minority communities. Some experts agree, emphasizing awareness of the federal government's official languages efforts because of the Roadmap and the previous five-year plans, which has—in some regions more than others—increased dialogue with provincial and territorial governments and with community organizations.

Only four of the twenty partners interviewed expressed their common understanding of the role of the Roadmap as it relates to federal partners. They indicate that it is not necessarily clear to the partners, but that its general objectives are, and its role seems clear, on the ground at least, among stakeholders and beneficiaries in the areas of economic development, employment and health.

Just one partner perceives no specific contribution to the Roadmap, indicating that it is largely a combination of interventions that federal institutions were already carrying out. Furthermore, according to some stakeholders and partners, through the five-year plans and a different grouping of initiatives, the Roadmap has caused some persistent confusion for several years between existing programming, which in some cases was enhanced, and new initiatives or new funds.

According to some partners, the Roadmap also helps to highlight the official languages file, and increase awareness of obligations under Part VII of the Act. However, according to some managers, partners and external stakeholders, the emphasis that the Roadmap places on certain federal partners or some of their programs also might have had a negative effect as well because the institutions or programs not directly participating do not feel involved or, worse, feel that they have no obligations under Part VII of the OLA. This perception of disengagement was also highlighted in the evaluation of the Roadmap 2008-2013, and led to the more systematic inclusion of key federal institutions in the governance structure for OL coordination, which goes beyond Roadmap governance (see the findings of the evaluation of the Official Languages Coordination Program – part of the Horizontal Coordination of the Roadmap for Canada's official Languages 2013-2018: education, immigration, communities in this regard).

As indicated in question 4 above, although the actions under each pillar are varied, in theory and according to the documentation consulted, they all contribute to the achievement of the same strategic outcome and are mutually supportive.^Footnote43 PCH also provides a detailed explanation in the 2013 Roadmap itself and in other documentation. For example, the education pillar comprises initiatives that are varied but complementary, and which aim to support the language industry, which focus on minority-language education and second- language learning. The immigration component comprises two initiatives: language training and immigration to OLMCs, which intersect with the objectives of the other two pillars. The last pillar, community, comprises a wide range of initiatives to enhance vitality and, although varied in nature, together they contribute to stronger communities. Although the initiatives are grouped under one of the three pillars based on the nature of their activities, the Coordination Framework underscores the assumption that many of them "also enhance the pursuit of the objectives of the other pillars".^Footnote44

The logic is, therefore, explained in theory. The managers and several partners believe that the pillar structure is logical. Many partners believe that the sequencing of the deliverables is also logical—more specifically those in the areas of health, economic development, employment, and arts and culture. The representatives of the immigration partners are divided on this issue. All the same, in the absence of indicators to measure the progression toward shared results by pillar and the ultimate outcome, the evaluation is not able to measure if this is, effectively, the case.

Very few of the partners interviewed indicated any factual complementarity between initiatives. Furthermore, some of the partners indicated that they cannot speak to the complementarity among Roadmap initiatives because they are focusing on their respective initiatives. Only three were able to give concrete examples of complementarity among initiatives (e.g., second official language learning initiatives such as bursaries and exchanges). In immigration, the representatives of partners in this area indicate that although there are connections between the different initiatives, this complementarity is not represented in the logic of the Roadmap's expected results.

Two partners in the areas of immigration and employment believe instead that the pillar structure is detrimental, as it actually promotes working in silos. One partner indicated that the pillars themselves promote silos instead of the planning of broader collective actions. Another said that there is some imbalance in Roadmap's design because it comprises very specific initiatives such as the Contraventions Act Fund and the Community Cultural Action Fund, and other very broad initiatives such as the recruitment and integration of Francophone immigrants in OLMCs.

Added value of grouping

Partners are sharply divided between those who see no progress attributable to the horizontal nature of the Roadmap and its grouping of initiatives, and who are convinced that they would have achieved the same results without the horizontal approach; and those who believe that the Roadmap contributes to a greater impact than the sum of its parts because of:

• the increased commitment of PCH and its partners;
• the emphasis on OLMC vitality; or
• the tools developed by the Roadmap.

One of the stakeholders emphasized that the intervention's impact on the population would have to be measured, otherwise the true added value of the Roadmap cannot be determined. Another suspects that there is no added value to the horizontal structure, explaining that, in fact, it promotes work in silos (related to the responses to question 5 above).

The experts agree that in theory, grouping initiatives into pillars is beneficial in terms of interdepartmental collaboration and that it promotes working towards common outcomes. However, they did point out that, in reality, there is work that is performed in silos. This hinders creativity and interdepartmental cooperation, as well as flexibility and local adaptation of interventions.^{Footnote g}

Nature of collaboration

The managers interviewed underscore that the Roadmap encourages collaboration among federal partners. Only three partners provided examples of collaboration: ISEDC, ACOA and ESDC^{Footnote h}. Two stakeholders from PCH and IRCC did, however, highlight increased collaboration within their department. A few other partners have a negative opinion of the degree of collaboration under the Roadmap:

• Collaboration is insufficient, even "accidental" – they sometimes discover partners and their programs through recipients.
• There is no mechanism in place to foster collaboration.
• The degree of collaboration would be the same without the Roadmap.
• When there is collaboration, it is limited to the bilateral exchange of information on programming or via interdepartmental governance mechanisms, or referral of potential beneficiaries.

External stakeholders are more positive and believe that the Roadmap encourages collaboration between departments and partner agencies, and also with other key stakeholders in the official languages area, including key stakeholders in OLMCs.

The case studies of a sample of OLMCs confirm that many representatives of OLMC agencies do not understand the difference between the Roadmap and other federal approaches. Naturally, they suggest that many, if not all, federal programs should be coordinated through the Roadmap, or at least grouped under PCH, insofar as it provides support to various facets of OLMC vitality and the delivery of services to the public in the minority official language. For their part, many of the partners, stakeholders and experts who participated in the evaluation identified specific actions that, in their view, seemed to be missing from the Roadmap but that should be included or that require further support to address needs.

Support for early childhood services was mentioned several times, in particular from the viewpoint of a health determinant (children's development), the continuum of education in the minority official language, and development of identity and a sense of belonging, which ultimately affect OLMC vitality. As indicated, the FCFA believes that the current Roadmap devotes almost nothing to it, even though it is a significant sector.^Footnote45 One expert pointed out that, to their knowledge, early childhood services in English receive no financial support whatsoever in Quebec and that, as a result, there is an even greater need.

During the annual CADMOL consultation in 2013-2014 and during the interviews conducted as part of the evaluation, representatives of Canadian linguistic duality promotion highlighted the need to establish a common framework of reference for languages in order to better measure the level of second-language skills nationally,^Footnote46 which did emerge during the previous Roadmap. They also suggested the possibility of developing a Canada-wide linguistic duality strategy.

According to the federal partners, other initiatives that could be improved or enhanced include:

• Increased effort to support the integration of immigrants and support for temporary residents and foreign students of the minority official language; and
• In addition to current efforts, more targeted support for the economic development of OLMCs in rural or remote areas, as well as support for innovation, science and technology in OLMCs.

According to partners and external stakeholders, the Roadmap could include some programs in health that have at least one official language component, or research on the health of OLMCs, such as programs from the Public Health Agency of Canada, Canadian Institutes of Health Research, and the Canadian Food Inspection Agency. Furthermore, a CIHR initiative to allocate funds for research on the health of OLMCs was cancelled in 2014. In 2016, following a complaint, CIHR reviewed the COL's recommendation to ensure that the agency was in fact supporting research on the health of OLMCs through existing programs or new measures.

The findings of the annual CADMOL consultation in 2013-2014 and of evaluation interviews indicate that other federal portfolios could be included to increase awareness of bilingualism and better address the concerns of OLMCs, such as National Defence and Public Services and Procurement Canada (PSPC) (in particular the Translation Bureau).

Moreover, representatives of English-speaking community organizations stated that they would like to see federal support to establish a "designated place"^{Footnote i} for considering issues specific to OLMCs of English-speaking communities in Quebec, such as identity, mobilization of youth, the health sector and farming communities.^Footnote47

Finally, the experts shared one concern, emphasizing that broadening the scope of the Roadmap could result in a reduction in the various envelopes, and that there would be interest in maintaining the current level of intervention and status quo regarding the initiatives included.

Education in the minority-language and second-language learning

PCH representatives are of the opinion that the Roadmap continues to contribute to OLSP and thus continues to contribute to learning opportunities in the minority language by increasing the number of minority-language schools and school-community centres, and to the continued rise in the number of second official language immersion programs, and the increase in the number of English-French bilingual Canadians, particularly through the increase in capacity of exchange programs. The OLSP evaluation provides data on enrolment.

However, the OLSP evaluation indicates that it remains difficult to measure intermediate and final outcomes arising from the activities funded, particularly regarding education in the minority language and second-language learning. The problem is exacerbated by the time frames involved in the submission of reports or the longer term nature of the funded activities. However, available information indicates that the various OLSP components are generally implemented as planned and that they support the implementation of activities that contribute to the expected outcomes.

For the education in the minority-language and second-language learning components, there is little direct information on the outcomes achieved as a result of funding since the provincial and territorial governments submit their reports a few years after the end of a fiscal year. The evaluation indicates that the provinces/territories are carrying out the planned activities in their respective agreements.

The OLSP evaluation also found that the absence of the obligation to offer second-language learning programs places special importance on federal investment offered through OLSPs. Young Canadians have access to second-language learning programs, but it is still difficult for many provinces to meet the entire demand for French immersion programs. Some stakeholders also believe that the Roadmap has contributed to improve the strategies used by second-language teachers in many provinces or territories and has emphasized retention in immersion programs up to graduation from high school. However, it is not possible to differentiate between the Roadmap's contributions to these results compared with the possible contribution of the other OLSPs (which are not in the Roadmap). Some of the stakeholders interviewed indicated that it is also complex to measure progress in bilingualism levels more specifically in relation to job market requirements.

Finally, some stakeholder representatives indicated that since the Roadmap contributes additional funds to existing agreements with provincial/territorial governments that target needs in minority-language education and second official language learning, it automatically contributes to advancement towards the deliverables of this pillar.

Certain other activities supported by the Roadmap also contributed to the achievement of deliverables in minority-language education, in particular the activities of the Tripartite Education Committee, the Education Summit and the National Table on Education, which bring together several community partners and where the emphasis is placed on promoting the transition from secondary to postsecondary education in French.

Bursaries and language monitors, promotion of linguistic duality

According to the OLSP evaluation, even though the level of bilingualism of the Canadian population is still relatively limited, there is a desire to encourage official language learning and to facilitate language exchanges, especially among young people.

According to 2006 and 2011 census data, the absolute number of people who report that they can hold a conversation in both official languages increased by 350,000, for a total of 5.8 million people. Over a longer period of time, it is noted that the percentage of Canadians who can hold a conversation in both official languages rose from 13.4% of the population in 1971 to 17.5% in 2011.^Footnote48 In examining the data specifically for youths aged 15 to 19 years, the census data indicate that among young Anglophones outside Quebec, the level of bilingualism dropped during the 15 years between 1996 and 2011, from 15.2% to 11.2%. The most recent survey conducted in 2016 also indicates that the majority of respondents is of the opinion that the relationship between Anglophones and Francophones has improved over the last decade (54% of respondents agreed or strongly agreed), that official languages reinforce national unity (67% agreed or strongly agreed), that linguistic duality is a source of cultural enrichment (65% agreed or strongly agreed), and that it is important to invest in exchange programs in Canada (78% agreed or strongly agreed).^Footnote49

Also according to the OLSP evaluation, many stakeholders would like to see greater awareness among Canadians of the benefits of official languages and linguistic duality.

Also related to learning and use of the second official language, the Exchanges Canada program is a two-pronged program that includes Youth Exchanges Canada. Through contribution agreements, this program is offered by four Canada-wide non-profit agencies including Experiences Canada, which organizes second official language exchanges. Experiences Canada has received $11.3 million via the Roadmap (part of the funds allocated to the organization). The Exchanges Canada program (through Experiences Canada) has therefore supported a growing number of exchanges, but only reports overall outcomes. Therefore, it is impossible to determine the progress achieved through the funds allocated under the Roadmap, which are permanent, and that make up a small part of Exchanges Canada's total budget.

Finally, PCH representatives and stakeholders indicate that the Roadmap has facilitated a push and increase in the capacity to meet the growing demand for exchange programs, which are already successful.

Post-secondary and vocational training

With respect to opportunities for training in the first official language and second official language learning, Roadmap partners pointed out the significant contribution of Health Canada's OLHCP—for many years one of the Roadmap's flagship programs, according to them. To that end, the OLHCP evaluation report (2012-2013 to 2014-2015) concluded that the program did in fact achieve its first immediate outcome: the supported activities contributed to a rise in the number of bilingual health professionals by supporting post-secondary institutions' offer of health programs and language training activities in the minority official language.^Footnote50 However, from the available data it is impossible to confirm that this led to an increase in access as such. The department recognizes that it does not have the necessary data to make this connection.

From the first year (2013-2014) of the Networks, training and access to health services initiative (education component), 13 contribution agreements were extended and then renewed for the years 2014-2015 to 2017-2018, and 12 new agreements were signed for the 2013-2014 to 2017-2018 fiscal years to train health professionals in OLMCs. From the time the Roadmap was renewed in 2013 until the end of the 2014-2015 fiscal year,1,452 health professionals in Quebec completed a language training program (1,302 in 2013, and 150 in 2014) and 1,586 Francophone students outside Quebec earned their diploma in French in a health related program (693 in 2013, and 870 in 2014, out of 11 institutions outside Quebec).^{Footnote51Footnote52}

Similar to the network created with the CNFS in health, Justice Canada contributed to the creation of the Réseau national de formation en justice (RNFJ) in 2013-2014—a group of 14 members from government, community and post-secondary establishments, whose secretariat is provided by the Association of Colleges and Universities of the Canadian Francophonie (ACUFC). For the period of 2014 to 2018, the RNFJ is tasked with ensuring better cooperation and consultation on legal training in order to maximize investments and generate savings while advocating greater use of technology and the elimination of duplication^Footnote53 . In addition, the French language training component of Access to Justice in the Both Official Languages Support Fund supports the justice system's stakeholders in the provision of services to Canadians in the official language of their choice by financing professional development of a cross section of stakeholders (provincial crown attorneys, provincial clerks, provincial officers and provincially appointed judges. Moreover, in 2014-2015, the Department of Justice funded a total of 21 projects in the "Education" pillar, four more projects than in the 2013-2014 fiscal year, and leading to a significant increase in the number of professionals who were trained (630) in 2014-2015 compared with the previous year (390).^{Footnote54,Footnote55} Notably, there have approximately 230 judges who have participated in the judicial linguistic training for provincial judges since 2013^Footnote56.

Language technologies

Federal partners also highlighted the Roadmap's contribution to the development of language technologies that support second official language learning. To that end, the NRCC is focusing on research and development to advance high-tech automated natural language processing systems, such as machine translation systems. In fact, according to the NRCC, this initiative contributes to the overall outcome of both the education and community support pillars.^Footnote57 That said, machine translation tool Portage, designed in partnership with the Translation Bureau, was one of the main topics discussed in the Study of the Translation Bureau. Furthermore, a parliamentary report published by the Standing Committee on Official Languages in June 2016 raised several concerns over this type of tool. During the 2015-2016 fiscal year, the NRCC signed a new licence giving over 350,000 public service workstations access to Portage.^Footnote58 While some see many advantages to the tool, there are still questions regarding the quality of translation it produces, and that the tool could compromise official language equality under the OLA. Despite its imperfections, many of those interviewed admitted that the tool is useful nonetheless, especially in terms of comprehension and use of both official languages and in increased productivity. In light of these findings, "the Committee [Standing Committee on Official Languages] firmly believes that Portage should be referred to as a language comprehension tool rather than a translation tool".^Footnote59 Furthermore, while the long-term vision of a machine translation system implies tangible impacts on access to education in any language, within the framework of the evaluation of the NRCC initiative, stakeholders noted that in the short and medium term, the NRCC's language technologies are primarily useful for increasing the productivity of human translators. To that end, the evaluation of the initiative recommended that the NRCC better determine how the activities and expected outcomes of its initiative align with the architecture of the Roadmap's outcomes, and specifically how it aligns with the expected outcome of the education pillar.^Footnote60

For its part, the evaluation of the Language Portal of Canada concluded that through the funding provided via the Roadmap for Canada's Official Languages 2013-2018, "the Language Portal supports Canadian linguistic duality and expertise in the official languages of Canada by providing free access to Canadian language resources". Furthermore, according to PSPC, "the volume of resources provided and the usage of those resources demonstrates a continuing need"^Footnote61, ^Footnotej. There is no documentation on the connection between the outcomes of this initiative and the overall deliverable of the education pillar.

On the whole, the federal immigration partners and stakeholders interviewed agree that there are an increase number of specific actions targeting Francophone immigration, and many stakeholders on the ground have better expertise; this should help the government achieve its objectives.

The available documentation mainly highlights the activities carried out as part of initiatives related to this pillar. In some cases, the level of outcome achievement is difficult to measure or demonstrate. However, the preliminary results of the evaluation of the IRCC's Immigration to OLMCs Initiative provides some indication of progress towards the overall outcome of this pillar, but also that additional efforts will be required to increase Francophone immigration in OLMCs.

Newcomers' language skills in either official language

The Roadmap provided that IRCC would "invest $120 million over five years to train thousands of newcomers in terms of language learning" so that they develop the English or French first or minority-language skills "that will enable them to overcome the challenges of daily life and enter the workforce."^Footnote62 The annual amount spent for this purpose is $24 million, as expected, and according to the Report on Progress and Results,^Footnote63 "[i]n 2014-2015, more than 46,000 economic immigrants (17,000 more that in 2013-2014) enrolled for language classes, funded by IRCC, to develop the proficiency they need to function in Canadian society and contribute to the economy"—namely Language Instruction for Newcomers to Canada (LINC) and Cours de langue pour les immigrants au Canada (CLIC). In addition, job market-related language training in English and French offers workplace programs (especially at high proficiency levels); occupational-focused language training also helps newcomers with an intermediate level of language; and language training is offered full time or part time, in the classroom, online or in hybrid mode.^Footnote64 However, it is impossible to identify the proportion of language training for economic immigrants that is related to the Roadmap, since it represents only a small part of the IRCC investment for language training of economic immigrants (which was $188.5 million in 2013-2014).^Footnote65

The preliminary results of the IRCC evaluation (from 2013-2014 to 2015-2016) indicate that:

• 37% of the 8,202 French-speaking settlement services clients included in the study had received language training funded by IRCC; 95% had received training in English only, 4% in French and only, 1% in English and French.
• French-speaking newcomers residing in FMCs use both of Canada's official languages in their day-to-day lives. While they appreciate the opportunity to use French and to have access to services and resources in French, knowledge of English is also required to function and participate in Canadian society outside Quebec.
• Most respondents who were asked about their use and preferences in official languages said it was important for their children (or future children) to speak French (95%) and English (98%). Seventy-six percent (76%) of respondents with children attending school in Canada had sent some of them to minority French language schools.
• Most French-speaking settlement services clients who participated in the IRCC settlement services survey reported having at least an intermediate level of proficiency in English, and many reported using this language at least about half of the time outside the home.

Contribution to economic, social and cultural development

The Roadmap has also earmarked "$29.5 million over five years to support the arrival of an increasing number of Francophone immigrants in minority communities,"^Footnote66 their settlement and their integration. Again, this is only a small part of the IRCC investment in recruitment and integration, but note that the department has:

• funded the creation of the National Community Table on Francophone Immigration in 2013, and the former steering committee was replaced by the IRCC-FMC committee;^Footnote67
• established an Interdepartmental Working Group on Labour Needs in FMC in 2014, which met twice in 2014 and 2015 to advance key strategic projects identified by the IRCC-FMC committee;
• put in place a research program in 2013 to support research on immigration in OLMC, that facilitated the implementation of numerous research projects and data development, and initiated mobilization and knowledge dissemination activities such as a meeting on Francophone immigration research in March 2015;
• according to interviews, the funding included support for research to define what is a Francophone immigrant, measure their numbers and examine their contribution;
• developed institutional programming policies tailored to the needs of Francophone minority communities (FMCs);
• financially supported the FCFA in assuming national coordination of the francophone immigration file and led initiatives of a national scope on francophone immigration;
• financially supported projects from over 40 Francophone organizations outside Quebec from its settlement program, thereby allowing them to offer settlement services in French or to coordinate the regional or local Francophone immigration file;^Footnote68 and finally
• conducted several consultations with the FMCs in support of the development of program policies with the support of the FCFA and the Francophone Immigration Networks.

Federal immigration partners indicate that responsibilities related to immigration to FMCs are now shared between IRCC and community partners and the private sector. Partners are asked to contribute to the supply of services and they are more engaged, in particular the provinces, territories and FMCs (Réseaux en immigration francophone, Réseau de développement économique et d'employabilité du Canada [RDÉE Canada], FCFA and all Francophone grassroots settlement agencies). Moreover, a survey administered to members of the Community Leaders Forum (under the aegis of the FCFA) in 2012 showed that half of these agencies had ties to civil society in immigration, while in 2014 a second survey showed that 85% of the agencies include diversity and immigration in their programming.^Footnote69

With respect to the very tangible results, according to the preliminary results of IRCC evaluation, Francophone immigrants participate in the job market at rates comparable to other immigrants in FMCs. The use of social assistance is higher among Francophone immigrants, but the difference is getting smaller over time. On average, the employment rate of Francophone immigrants in Canada between 2003 and 2013 was 68% compared with 66% for other immigrants. The average employment income of Francophone immigrants has increased over time and was comparable to that of other immigrants. The Francophone clients of the settlement services surveyed also reported levels of volunteering, group membership and a sense of belonging to Canada comparable to those of Canadians in general. Furthermore, a higher level of English proficiency was associated with volunteering, group membership and having friends outside one's community.

Contribution to the demographic vitality of OLMCs

Federal immigration partners and many of the stakeholders interviewed agree that the 4% target settled by the IRCC and CFSM committee for economic immigrants in FMCs by 2018 will not be reached. According to the IRCC evaluation, although Francophone economic immigrants have settled in FMCs, their relative proportion within the population of economic immigrants outside Quebec actually falls short of the target, specifically, between 2003 and 2016:

• Outside Quebec, the proportion of French-speaking permanent residents in the total immigrant population averaged 1.5%, reaching a peak of 1.8% in 2012 and 2016, which is below the target 4.4%.
• 44% of Francophone immigrants who settled in FMCs were from the economic class, representing about 1.1% of all economic immigration outside Quebec during this period, and still below the target of 4% established for Francophone economic migration in these communities.^Footnote70

Measures to promote Francophone immigration to FMCs have focused primarily on promotion and recruitment efforts in Canada and abroad, such as job fairs and mechanisms to facilitate temporary stays. In particular, a new component of the International Mobility program, Mobilité Francophone was introduced in June 2016. This labor market impact study exemption replaces the Francophone Significant Benefit program, abolished in 2014.

Federal immigration partners indicate that Roadmap funding has also helped to highlight the importance of Francophone immigration to FMCs to ensure that communities understand the importance of Francophone immigration and do not see it as a panacea, and to ensure that employers are more aware of opportunities to hire Francophone immigrants. However, the interviews and documentary review conducted as part of the IRCC evaluation identified a need for enhanced measures to increase immigration to FMCs, such as employer engagement, awareness of FMCs abroad, and mechanisms or programs to facilitate permanent residency.^Footnote71

Again, the Roadmap's contribution to these programs is only a small part of the IRCC investment under immigration to OLMCs, and the evaluators have no evidence of the effect that this may have had to date on newcomers' contribution to Canada's economic, social or cultural needs, or to the vitality of Francophone minority communities specifically.

Finally, in terms of immigration within Quebec's English-speaking community, according to the 2012 report of the Standing Committee on Official Languages and interviews with stakeholders, community organizations do not see this as a priority in Quebec, and stressed that English-speaking communities in Quebec have not benefitted from immigrant recruitment and integration initiatives to the same degree as the Francophone communities (the Roadmap only targets the recruitment of French-speaking immigrants to FMCs). The documentation confirms that the Canada-Quebec Agreement imposes limits on the involvement of IRCC in the Anglophone immigration file in Quebec since, under this agreement, immigrant selection and integration is the exclusive jurisdiction of the province.

Key participants (partners and stakeholders) are mostly convinced of the Roadmap's contribution to OLMC vitality, and they provide many examples in health, justice, arts and culture, and economic development. The representatives of OLMC organizations that have been the subject of case studies also highlighted examples of the federal government's contribution (beyond the Roadmap) to the continuum of education in the minority official language, to the heritage sector (included in arts and culture) and to the terms of general economic development.

Some managers, federal partners and external stakeholders believe that concrete results are difficult to demonstrate or measure, especially at the halfway point of a five-year plan, because it is difficult to attribute the effects to Roadmap interventions. Moreover, insofar as the five-year plans have addressed several aspects of community vitality for many years, the current Roadmap contributes to continued progress towards community deliverables that precede the current five-year horizon. Moreover, the challenge of pinpointing the effects of interventions over the years was evident in conversations held in the context of the case studies.

Composite indicators of vitality for the sample of nine OLMCs, according to 2011 census data compared to 1996 (Table 8 below), indicate:

• A demographic weight relative to the surrounding region which increases in or near major urban areas (Pontiac) but decreases sharply elsewhere, especially in the most isolated OLMCs (Gravelbourg);
• A high proportion of the population in the area around the OLMC speak the minority official language: from one time to more than five times the size of the OLMC, by region, despite a bilingualism rate among the majority that remains low in many areas (Summerside, London, Gravelbourg, Surrey);
• An immigrant population that remains high and increases in or near large urban areas but remains low elsewhere.

Although this picture does not yet include Census 2016 data, according to interviews conducted in these OLMCs, these trends do not seem likely to change. The challenge remains knowing how to distinguish the effects of broader social and economic phenomena from those stemming from government interventions.

Table 8: composite indicators of OLMC vitality, 2011 and changes since 1996

OLMC	Region	Geospatial complexity		Demographic vitality		Demo-linguistic vitality			Sociocultural diversity
		Population in OLMCs		Proportion of OLMCs		Bilingualism among the majority		Ratio of speakers of the minority OL versus the OLMC, 2011	Born in another Canadian province		Immigrants
		2011 size	1996-2011 change	2011 proportion	1996-2011 change	2011 rate	1996-2011 change		2011 rate	1996-2011 change	2011 rate	1996-2011 change
Summerside	Prince (PEI)	3,038	0.77	6.9%	0.78	9.0%	1.21	2.16	4.6%	0.45	0.0%	-
Bathurst	Gloucester (NB)	64,780	0.89	84.6%	1.01	46.7%	1.04	1.08	2.7%	1.16	0.3%	1.20
New Carlisle	Bonaventure (QC)	2,725	0.95	15.3%	1.04	27.3%	1.12	2.52	6.2%	1.68	2.5%	2.00
Beaconsfield	Montreal (QC)	611,005	1.09	32.8%	1.02	57.2%	1.10	2.12	12.4%	1.15	40.4%	1.10
Pontiac	Les Collines-de-l'Outaouais (QC)	12,108	1.21	26.2%	0.88	63.2%	1.02	2.79	9.2%	0.55	11.3%	1.19
London	Middlesex (ON)	6,210	1.29	1.4%	1.14	5.5%	1.00	4.74	11.5%	0.89	21.2%	1.17
Timmins	Cochrane (ON)	35,750	0.82	44.6%	0.95	25.2%	1.01	1.30	1.0%	0.60	0.4%	0.71
Gravelbourg	Division No. 3 (SK)	1,013	0.59	8.1%	0.76	5.1%	0.89	1.49	2.3%	0.78	2.2%	1.29
Surrey	Greater Vancouver (BC)	31,970	1.15	1.4%	0.91	6.3%	1.01	5.19	19.9%	0.66	35.4%	1.14

Source: PCH. (2016). Data prepared by the OLB Research Group.

The progress of initiatives in the community pillar in terms of expected outcomes is described below for the arts and culture, justice, health, economic development and employability sectors.^{Footnote k}

Arts and culture

Two cultural programs managed by PCH have provisions specifically for OLMCs and offer additional support via the Roadmap:

• Music Showcases for OLMC artists (additional $5.8 million)
• National Translation Program for Book Publishing ($4 million)

A third is under PCH responsibility, but it is managed by the CCA:

• Market Access Strategy for Artists from OLMCs of the CCA (additional $2.7 million)

According to the documentation, the three programs have achieved their targets or have demonstrated a change in their activities and outputs as a result of the additional funding, supporting a growing number of artists, producers, arts organizations and works from OLMCs. According to the documentation and to partners and stakeholders, the programs have contributed to the growth of OLMC artists and markets in the arts, improving communities' ability to retain their artists and expand their outreach (e.g., translation, extension of tours to regional and national markets, which have brought OLMCs closer to the majority language, and exploratory missions to meet broadcasters in Canada and abroad).

During the first two years of the Roadmap, Music Showcases for OLMC artists funded nearly 400 artists from OLMCs in more than 1,700 music showcases, while the annual target was 400 showcases. As indicated by PCH, "since the launch of this initiative in 2008, artists from OLMCs have been exposed to a larger audience while OLMCs have had access to music showcases in their language (regional and national showcases). Specifically, an average of 200 artists and 400 showcases are supported annually compared to only 40 artists and 85 showcases before 2008. As such, there has been an increased access (e.g. consumption) to music of OLMC artists in many formats (live performance, on-line access, album sales, etc.)".^{Footnote72,Footnote73,Footnote l}

In 2014-2015, the National Translation Program for Book Publishing (NTPBP) funded 68 translations of Canadian-authored books in both official languages, and the translation grant program of the Council for the Arts supported 48 translations, for a combined total of 116 books—an increase of almost 10% over 2013-2014—and in 2014-15, the program is at 83% of the target of 35 publishers who receive translation subsidies annually. Furthermore, the program has made significant progress towards its two intermediate objectives in terms of new business collaborations and new book collaborations.^Footnote74 As the evaluation of the previous Roadmap pointed out the lower-than-projected level of grant applications for this program, measures were taken to eliminate potential barriers. Finally, this program is not evaluated separately.

The evaluation of the CCA Market Access Strategy for Artists from OLMCs (from 2013-2014 to 2015-2016) indicates that most of the stakeholders are satisfied with the scope of support for the Strategy and that "recipients unanimously and emphatically stated that their projects and learnings would not have advanced without support from the Strategy."^Footnote75, ^Footnotem In the first two years of the Roadmap, the funds helped to support a total of 71 projects. Recipients argued that the objectives of the Strategy are helpful in overcoming their market access problems. In addition, the evaluation confirmed that a significant priority for many recipients is to break into the domestic market and break through expanding regional circuits before aiming for national or international targets.^Footnote76

Still in the field of arts and culture, OLSPs also provide financial support to approximately 340 community organizations, enabling them to operate and implement their programs and individual projects (festivals, historical celebrations, artistic creations, gatherings, mobilization, etc.). According to the findings of the OLSP evaluation, although there is strong support for this component, there are also significant challenges, including the difficulty of expanding the circle of funded organizations or that the financial investment has remained at the same level for several years, limiting the action of community organizations. There is widespread interest in the Community Cultural Action Fund (CCAF), which specifically targets OLMC development through arts and culture. CCAF participation, however, is uneven across the country.

Finally, it is difficult to attribute the effects of these cultural programs specifically to OLMC vitality.

Justice

Since 2013, Justice Canada has continued its activities ($21.2 million), funding the creation of five justice information hubs in OLMCs in Nova Scotia, Ontario, Manitoba, Saskatchewan and Alberta, and the Canadian legal information web portal, www.cliquezjustice.ca. In all, the Networks and Access to Justice Services initiative supported 21 projects in 2014-2015 (versus 18 projects in 2013-2014) aimed at raising awareness among the legal community of justice issues in both official languages.^Footnote77 In addition to the hubs, roughly 15 projects targeted awareness-raising and the provision of legal information in lay terms to promote access to justice in both official languages, and one aimed to encourage bilingual students interested in pursuing a career in law.^{Footnote78Footnote79} The preliminary findings of the evaluation of the Initiative in Support of Access to Justice in Both Official Languages (from 2013-2014 to 2015-2016) stress that in terms of training legal professionals, there is indeed a high level of satisfaction among professionals, and that the initiative has helped develop unexpected partnerships beyond expectations between community organizations and stakeholders, multiple networking opportunities and information sharing. In short, the evaluation confirms the success of the initiative and stresses the continued need for such an initiative, although it emphasizes the importance of addressing the systematic and quantitative measurement of impacts, which is not the case at present.^Footnote80

The Contraventions Act Fund exists because of a court decision and is essentially an initiative to enforce legislation. According to the current evaluation (from 2013-2014 to 2015-2016), it has therefore a link to the logic of the Roadmap outcomes (essentially focused on Part VII of the OLA). The final findings of the evaluation of this Fund are not yet available; however, the preliminary findings indicates that the provinces that signed an agreement have put in place the necessary measures to ensure that Canadians have access to judicial and extrajudicial services in the official language of their choice.

Health

Between 2013 and 2015, Health Canada continued its efforts with the Official Languages Health Contribution Program (OLHCP) to improve access to health care in Anglophone and Francophone OLMCs ($67.8 million). A total of 38 partnerships have been maintained since the renewal in 2013 to increase health care services in OLMCs; and contribution agreements were signed with seven new recipients for the 2014-2015 to 2016-2017 fiscal years "to support innovative projects that aim to increase access to a range of health care services" in OLMCs.^Footnote81

Some of the federal partners and stakeholders interviewed for the horizontal evaluation believe that the Roadmap makes a significant contribution to OLMC vitality through increased numbers of health care professionals who can serve people in their language, and increased number and capacity of community networks. Furthermore, some of the interviewees called OLHCP a "flagship initiative" of the official language five-year plans. Moreover, the evaluation of the OLHCP (2012-2015) highlighted the following:

• There is a significant link between training and benefits to the community;
• The role of the Société Santé en français and of the Community Health and Social Services Network (CHSSN), including their local community networks (supported by the OLHCP), which also contribute to increased access to and quality and safety of health services for OLMCs; and
• It concludes that the OLHCP has contributed to the revitalization and empowerment of OLMCs in Canada and to the awareness of stakeholders outside OLMCs about various issues related to access to and quality and safety of health care services for these communities.^Footnote82

Economic development and employability

The partners in economic development and in employment (ISED and regional economic development agencies) and in employment (ESDC) interviewed indicated that, through their respective initiatives, the Roadmap has contributed to community vitality by providing support for:

• exploring other economic development opportunities in recent years (tourism, construction, etc.);
• supporting economic development and employability networks;
• acquiring essential skills;
• workforce training; and
• increasing capacity in specific areas such as translation, development of entrepreneurship, and business incubators.

Some external stakeholders share the same view in terms of the initiatives contribution in the area of economic development and employment to the vitality of communities.

At the time of the evaluation, EDI was used to support various projects (109 in the first two years) fostering innovation, economic diversification and business development. According to ISED, this aligns with the Roadmap's objective of supporting economic development in OLMCs.^Footnote83 A separate evaluation of EDI will be conducted by EDI's partners. ESDC's Enabling Fund for OLMCs has supported initiatives that facilitate the integration of young professionals in the job market and business development, improve the bilingual tourism offer, and contribute to the development of jobs in the green economy sector and to the economic revitalization of some rural communities. According to ESDC, OLMCs are better informed, more competent and have more resources thanks to the multiple initiatives undertaken through the Fund.^{Footnote84,Footnote85} An ongoing evaluation provides some preliminary findings, notably that funds, despite some challenges in terms of levels of funding and external factors to the program, has indeed contributed to the development and vitality of LOMCs. For its part, the evaluation of the OLMC Literary and Essential Skills Initiative demonstrated that it has contributed to the ability of Canadians to live and work in OLMCs. The translation and adaptation of the Communications and Math Employment Readiness Assessment (CAMERA) tool was also mentioned several times as an example of the initiative's impact on the lives of Canadians.^Footnote86 ESDC has not yet paid out any funds for the new Social Development Initiative in OLMCs, but in place since 2013, the initiative has made it through the first stage by having helped to establish solid partnerships with key organizations in OLMCs through the Federation of Francophone and Acadian communities of Canada (FCFA) and Québec Community Groups Network (QCGN).^Footnote87

As a backdrop (and as indicated in relation to question 1 on relevance), the survey conducted in 2016 indicates that a majority of Canadians agree that Canada's linguistic duality is tied to Canadian identity (70%), and that bilingualism is an asset for finding employment (80%), and of cultural enrichment (65%). Furthermore, respondents perceived that the government of Canada is effective in protecting both of Canada's official; (71% of respondents agree) and 67% of respondents agree (of which 46% strongly agree) that its policy on official languages reinforces national unity^Footnote88 (see Figure 2, Annex C). Data on demographic and linguistic indicators for the sample of nine OLMCs in the case studies also found:

• an increase in the bilingualism rate between 1996 and 2011 among the majority official language population surrounding these OLMCs in all but one case, i.e. the region around Gravelbourg, Saskatchewan; and
• a high index of intergenerational transmission of bilingualism in 2011 in the region encompassing these OLMCs, indicating a higher rate of bilingualism among those aged 25-44 than those aged 45-64 years (with the exception of the region of Collines-de-l'Outaouais, where the proportion is virtually the same for both age groups).

However, the evaluation underscores just how difficult it is to actually measure the outcomes of multiple Roadmap interventions against such a large-scale ultimate outcome, which encompasses both the concepts of living and thriving in both official languages and recognizing the importance of both official languages for the identity, development and prosperity of the country. The fact that the horizontal evaluation is taking place at about the midway point, the time frame related to the submission of initiative evaluation products or the nature of the activities funded (e.g., the long start-up period of the social partnership initiative) contribute to this problem. The information available indicates, however, that the various initiatives are implemented largely as expected, and many of them contribute to some advancement towards the expected outcomes of each pillar (see the answers to questions 8, 9 and 10 above). This is, however, not the case for the ultimate outcome. Virtually none of the stakeholders interviewed could comment on any progress towards the ultimate outcome of the Roadmap, limiting themselves to the logic of this progress in very general terms and underlining the difficulty of measuring it.

More specifically, some partners believe that the Roadmap generally facilitates progress towards the ultimate outcome, that the interventions in economic development, and in arts and culture necessarily contribute to identity and development in general, and that the growth of Francophone immigration helps to stabilize the demographics of OLMCs. They emphasized that such contribution is measured over a much longer period than the Roadmap. For their part, stakeholders indicated that the Roadmap has in fact contributed to the ultimate outcome, but they were unable to specify that contribution, stating that it is difficult to discern a direct effect and measure the contribution of the Roadmap to progress towards an outcome so broad in scope. Moreover, there is no documentation establishing a link to progress towards the ultimate outcome of the Roadmap.

One suggestion from experts to overcome this challenge is to redo a post-census survey such as the one in 2006 on OLMC vitality, to measure certain general but regular indicators of vitality, which would include assessing the impact of the Roadmap. Another suggestion is instead to adopt a participatory research approach and to measure certain indicators related to the ultimate outcome regularly in different regions (for calibration or other), involving the community in the monitoring process. While these suggestions may contribute to the challenge of demonstrating progress towards the ultimate outcome, the problem of attribution must also be considered.

While some partners were not aware of any or found no unexpected impacts, many did cite some relatively significant unexpected impacts. Positive impacts included:

• Some regional economic development initiatives became priorities and are tied to pan-Canadian objectives to which more resources are allocated; and
• There are some health projects funded under the auspices of the Roadmap that bring French and English OLMCs together to collaborate, particularly in a mutual knowledge transfer.

The experts agree that the Roadmap has had positive effects, although not entirely unexpected, stressing in particular its ripple effect. For example, there is a continued interest on the part of Roadmap beneficiaries in community development, as a result of the five-year plans that have succeeded, and in the networking that the Roadmap encourages regionally. There has also been a ripple effect for some groups that do not belong to or do not benefit directly from the Roadmap.

Moreover, according to the stakeholders, progress in the promotion of and access to education in the second language through the Roadmap and more particularly OLSPs contributed to increased demand and helped to show how the demand is outstripping the supply. This is seen as a positive impact, as it underscores the scale of the shortfall of qualified teachers and encourages further reflection on the solutions that could be implemented to address this situation, while acknowledging the sphere of provincial and territorial jurisdiction.

However, there are negative impacts:

• As indicated in Question 4, managers, as well as some partners and external stakeholders, believe that the emphasis that the Roadmap places on certain departments or agencies has a negative effect, since federal institutions not directly participating do not feel involved.
• Some partners see the Roadmap's emphasis on some of their programs as an extension of the department's mandate or consider that the significant injection of funds can push federal intervention beyond the capacity and expertise of other key stakeholders, such as community organizations. This was the case in some regions in economic development, and more generally in immigration.

Moreover, according to some partners, providing repeated financial support to certain recipients can create a dependence. Some stakeholders explained it as a tendency to self-justify and protect one's share of the budget over the five-year plans. Partners and stakeholders also stressed that the emphasis on project funding at the expense of the organizations' core funding has weakened many of them and reduced their capacity. Furthermore, the approval or renewal process for a horizontal initiative of this magnitude and complexity can be lengthy and delay implementation in the first year of the five-year plan. As was the case in 2013, it can create an "interruption" in funding and lead to layoffs, which some organizations can handle, but not all.

1.4.2 Core Issue 5: demonstration of efficiency and economy

This section presents the key findings on demonstration of efficiency and economy.

Summary

There are limited data to demonstrate the efficiency and economy for this type of horizontal initiative; however, the evaluation products of partners come to a positive conclusion on the efficiency of their own initiatives.

Measures have been put in place within PCH to increase efficiency, for example, PCH's administration budget of the Roadmap has decreased. Several other partners, including Health Canada and ESDC have revised their program procedures to be more efficient.

PCH managers and federal partners see no other way to achieve the same results as the Roadmap. Some argue that they do not see alternatives because additional support from the Roadmap is largely provided through the terms of conditions of existing programs. The issue of efficiency comes into play, rather, in each of the 28 initiatives that were selected.

There is no information available that makes it possible to answer this question directly.

There are limited data to demonstrate the efficiency and economy for such a horizontal initiative, but the partners' available evaluation products come to a positive conclusion on the efficiency of their various initiatives, which was corroborated during the interviews.

Representatives of PCH and of several federal partners provided examples of measures introduced to increase the administrative efficiency of the Roadmap and of their respective programs or interventions. Within PCH:

• PCH's budget for Roadmap administration was cut by $7 million per year, or $35 million over five years compared to the previous Roadmap.
• Within the framework of the department-wide Grants and Contributions Modernization Initiative, PCH has implemented innovative measures to simplify, standardize and streamline business processes surrounding the awarding of grants and contributions, and process applications more quickly.^Footnote89
  • Regarding PCH initiatives under the aegis of the Roadmap, applications for support under $75,000 are now subject to the approval of a director general or regional director, speeding up the decision and awarding of funds.
  • The proportion of funding in the form of grants rather than contributions increased significantly, saving time and money for the OLB and organizations that make support requests (contract and cheque, year-end report rather than a detailed agreement and more frequent progress reports).

• In 2013-2014, PCH conducted a review of its support for communities in order to strengthen the effectiveness of community networks and intervene in key sectors for OLMC vitality.^Footnote90

National round tables (National committees) on economic development and employability conducted by ESDC were replaced with a new governance structure called the Economic Action Network (EAN), composed of RDÉE Canada, in collaboration with Quebec Community Economic Development and Employability Corporation (CEDEC) in partnership with ESDC. This mechanism offers a platform of choice for bringing together key actors serving linguistic minorities in community settings and federal government. In addition, two regional economic development agencies have also managed to reduce their operating costs with respect to the EDI:

• Under the EDI, FedDev Ontario has chosen to support large multi-year projects and a third-party implementation model that is more efficient.
• ACOA has created a committee of Official Languages Coordinators in each of its regional offices to discuss and approve projects throughout the Atlantic, thereby reducing travel costs.^Footnote91 This ensures adequate monitoring of projects funded by the agency.

Health Canada encourages the transfer of knowledge among its OLHCP beneficiaries, sharing tools and training programs, cooperation between Francophone and Anglophone OLMCs, and cloning of exemplary projects between OLMCs. For its part, IRCC regularly reviews how its activities meet the needs of communities through consultations that have helped it to be more efficient in developing policies and programs. Efforts have also been made by partners to make better use of available funds. For example:

• The book publishing program found another source of support for the annual publishers' fair, and uses that portion of the funds to further increase the number of book translations; it has also cut administrative expenses by 10%.
• The NRCC was able to achieve savings by moving all of its employees to the same building.^Footnote92

Moreover, the financial leverage hypothesis is an integral part of some initiatives under the Roadmap in the area of economic development, employment, and development of language technologies. According to these partners and available documentation, there is a ratio of 1:1 to 2:1 depending on the initiative. For example:

• ESDC indicates that recipients of the Enabling Fund "will leverage $23 million in additional funding in 2015-16 to support community economic and human resources development"^Footnote93 for a leverage effect of nearly two dollars per dollar invested.
• Meanwhile, the Social Partnership Initiative in OLMCs predicted that by 2014-2015 "a network of intermediaries (partners) will be identified to lever federal funds, over a multi-year period" and that in the coming years they "will further distribute combined federal and partner resources to local organizations for community-based activities that address local social challenges".^Footnote94
• Regional economic development agencies systematically quantify the leverage of their initiatives under the aegis of the EDI: FedNor, leverage of 96%; ACOA, 151%; FedDev Ontario, 61%; WD, 131%; CED-Quebec, 146%; and CanNor, 73%.^Footnote95 .

Some partners added that in other areas (e.g., health and justice), there are positive effects within the individual initiatives of various partners due to the sharing of innovative ideas and best practices that spread, rather than because of leveraging in the strict sense. PCH and several partners also anticipate future opportunities to improve efficiency or economy:

• OLB managers indicated that the digital transformation underway at PCH will likely realize additional savings in the processing of applications for support and reporting.
• Health Canada is considering including interpreters as a tool to address the low number of bilingual health professionals in certain OLMCs;^{Footnote n}
• IRCC is considering a coordinated strategy to promote FMCs abroad, for example, with the Department of Global Affairs or the Canadian International Development Agency; and finally,
• ESDC is proposing to simplify community consultations and incorporate an "official language minority" lens into all the initiatives, rather than adapting the initiatives developed to the majority.

PCH managers and federal partners see no other way to achieve the same results as the Roadmap. Some argue that this is because additional support from the Roadmap is largely provided through the terms and conditions of existing programs. The issue of efficiency, therefore, comes into play rather in each of the 28 initiatives that were selected. Stakeholders have suggested ways to improve Roadmap efficiency:

• A deeper exploration of the Roadmap change theory;
• Developing a more comprehensive horizontal management framework to better clarify the roles and responsibilities of federal partners; and
• Exploring alternative models of intersectoral collaboration among federal partners and OLMC agencies.

Furthermore, the experts suggested a logic focused on themes that cut across more sectors (than the current pillar structure does) and strategic outcomes for each of these themes.

The literature review examined four initiatives relatively similar in nature to the Roadmap in Canada and Australia. They are not identical, but are similar in some respects and offer no alternatives. However, some best practices in education and awareness of the general public, and civil society engagement were noted.

1.5 Conclusions

Relevance

Official languages remain a cornerstone of Canadian identity and are generally viewed favourably by Canadians, according to the 2016 survey to assess Canadians' attitude towards and perception of official languages. Overall, the evaluation found that the current Roadmap addresses several continued needs in OLMCs, but it would be necessary to adapt some interventions to better address needs in some areas, and the needs of OLMCs of English-speaking communities in Quebec in particular.

More specifically, the Roadmap must continue to adapt to the demographic, social and economic context: aging of the population, youth out-migration, integration and retention of newcomers, and sense of belonging. Consequently, to better address these needs, increased support would be required in the areas of early childhood and youth, seniors, integration and retention of newcomers, and to counteract the vulnerability of the official language minority media.

There is also a lack of detailed data that would help to better determine workforce development needs, promising sectors for economic development, the needs of OLMCs in rural or remote areas, and health care needs. One suggestion is to compensate for this lack of data by dedicating a share of the funds to research organizations that already have some expertise in the minority reality; for example in the health sector, the government could seek alliances with the Social Sciences and Humanities Research Council (SSHRC), the Canadian Institutes of Health Research (CIHR), or other organizations outside government It already collaborates with such as the Consortium national de formation en santé (CNFS) or McGill University. The evaluation of the previous Roadmap also underscored the need to support research to better inform actions.

Design and implementation

Within government, the Roadmap is seen as a promotional tool; externally, it is seen as a vision statement. Among federal institutions, it contributes to an increased awareness of obligations under Part VII of the OLA. (Except for the Contraventions Act Fund, with as previously explained contributes to Part IV of the OLA) Furthermore, the current Roadmap and previous five-year plans have had a leveraging and growth-generating effect in some areas. There is, however, a perception that institutions that are not part of the plan are disengaged from their official languages obligations. This was also pointed out in the evaluation of the previous Roadmap. Furthermore, throughout the five-year plans and the change in grouping of initiatives, the Roadmap caused some confusion over the years between existing programming, which in some cases was enhanced, versus new initiatives or new funds.

Federal partners are sharply divided between those who see no progress attributable to the horizontal nature of the Roadmap and those who believe that the Roadmap contributes to a greater impact than the sum of its parts. Grouping initiatives into pillars allows organizations to work towards common outcomes, but the pillar structure also encourages working in silos, which goes against the desire for collaboration among departments and community organizations. In addition, the evaluation finds that the performance measurement strategy in place does not include specific indicators and data collection to measure progress toward common outcomes by pillar and ultimate outcome.

The role of the horizontal management of official languages within the federal government versus horizontal coordination of the Roadmap is confusing. This is discussed in greater detail in the context of the evaluation of the Coordination Program – part of the Horizontal Coordination of the Roadmap 2013-2018 for Canada's Official Languages 2013-2018.

Achievement of expected outcomes

Education pillar

The Roadmap contributed to learning opportunities in the minority official language; however, the OLSP evaluation indicates that it remains difficult to measure intermediate and ultimate outcomes arising from the activities funded. The evaluation finds, however, that the provinces/territories are offering the planned activities in their respective agreements. The evaluation also underscores the absence of an obligation to offer second-language learning programs, which places special importance on federal investment.

With respect to training opportunities in the first official language and second official language learning, the evaluation highlights the significant contribution of Health Canada's OLHCP, and the contribution of a similar program in justice. Some federal partners also highlighted the Roadmap's contribution to the development of language technologies that support second official language learning, but the connection between the Roadmap's contribution and second official language learning with respect to the Language Portal and the Exchanges Canada program is not as clear.

Immigration pillar

Federal partners and stakeholders agree that there are specific actions targeting Francophone immigration, and many stakeholders on the ground have better expertise, which should help the government achieve its objectives.

The available documentation primarily covers the activities undertaken, and the level of outcome achievement is difficult to measure. However, the preliminary findings of IRCC's evaluation of the Immigration to OLMCs Initiative provides indication of progress towards the overall outcome of this pillar, but also indicate that additional efforts will be required to increase Francophone immigration in OLMCs.

For example, newcomers' language skills in either official language have been strengthened. However, it is not possible to establish a link between settlement services funded by the IRCC and newcomers' knowledge of official languages. The preliminary findings of IRCC evaluation also found that Francophone immigrants participate in the job market at rates comparable to other immigrants, and that their average employment income has increased over time and was comparable to that of other immigrants. The evaluation also noted levels of volunteering, group membership and a sense of belonging to Canada comparable to those of Canadians in general. IRCC also noted that funding under the Roadmap contributed to numerous international activities, such as job fairs, and this increased mobilization allowed stakeholders in the economic community to gain better knowledge and understanding of the tools and programs that are offered. Again, this is only a small part of the IRCC investment in recruitment and integration.

However, although Francophone economic immigrants were settled in FMCs, their relative proportion falls short of the established targets: between 2003 and 2016, the proportion of Francophone permanent residents within the immigrant population averaged 1.5%, below the target of 4.4%; and 44% of Francophone immigrants who settled in FMCs were from the economic class, representing about 1.1% of all economic immigration outside Quebec and still below the target of 4%.

Communities pillar

Stakeholders are for the most part convinced of the Roadmap's contribution to OLMC vitality, and the interviews, documentation (evaluation products, etc.) and case studies provide many examples of progress in terms of increased activities and outputs in health, justice, arts and culture, and economic development. Given that some initiatives fall under the various five-year plans prior to the current Roadmap, it is noted that the Roadmap contributes to continuity in the advancement of certain outcomes for communities. However, concrete outcomes are difficult to measure in some respects because it is difficult to attribute the effects to interventions under the Roadmap.

Finally, in terms of progress in terms of the expected outcomes of the Roadmap by pillar, little information is found specifically about the outcomes obtained for several initiatives: the evaluation is taking place at about the halfway point, there are delays in the submission of initiative evaluation products, and the nature of the activities funded lead to long-term visible outcomes. Furthermore, overall, the nature and very structure of the Roadmap makes it difficult to attribute expected outcomes. This also poses a challenge in relation to the objectives and outcomes of the Treasury Board Secretariat's 2016 Policy on Results, in particular requirements "3.1.2 Enhance the understanding of the results government seeks to achieve, does achieve, and the resources used to achieve them" and "3.2.4 Parliamentarians and the public receive transparent, clear and useful information on the results that departments have achieved and the resources used to do so".^Footnote96

Efficiency and economy

There are limited data to demonstrate the efficiency and economy for such a horizontal initiative, but the evaluation products of partners come to a positive conclusion on the efficiency of their own initiatives. Measures were put in place within PCH to increase efficiency. Several other partners, including Health Canada and ESDC, have revised their program processes to be more efficient, and some (PCH, IRCC, Health Canada and ESDC) already anticipate future opportunities to improve efficiency and economy. Moreover, federal stakeholders see no other way to achieve the same outcomes as the Roadmap.

One of the key suggestions for improving Roadmap effectiveness is to explore alternative models of intersectoral collaboration among federal partners and OLMC organizations, following the example of the Community Health and Social Services Network in Quebec (a network of both community organizations and public institutions).

1.6 Recommendations and management response

Recommendation 1 - Alignment of key initiatives and programs

The evaluation indicates that the current Roadmap addresses several ongoing needs of OLMCs, but it must continue to adjust to the changing demographic, social and economic context. According to the findings of the evaluation, there are emerging needs in certain areas (early childhood and youth, seniors, integration and retention of newcomers, vulnerability of minority media). Considering the context of the renewal of the federal strategy and new priorities regarding official languages:

Recommendation

Given the context of the renewal of the federal strategy and the government's new priorities regarding official languages, it is recommended that the Assistant Deputy Minister, Citizenship, Heritage and Regions:

1. prioritize (in cooperation with partners) the programs and initiatives that can address the emerging needs of OLMCs and the government's priorities; and
2. give preference to programs and initiatives that can best achieve the expected outcomes of the next federal multi-year official languages plan.

Management response

Recommendation 1: accepted.

The Official Languages Branch recognizes that there are emerging needs in certain areas and it is appropriate to ensure that programs and initiatives are prioritized and promoted in order to better meet the emerging needs of OLMCs, government priorities and the expected outcomes of the next multi-year official languages plan.

The OLB supported the Department in its cross-Canada consultations in 2016 to identify the needs and priorities of OLMCs. Several partners also held sectoral consultations. The OLB is working together with federal partners to propose programs and initiatives for the next federal official languages action plan that meet these needs and priorities.

Action plan

Action	Deliverables	Timetable	Program sponsor
1.1 Identify the needs of OLMCs	Cross-Canada official languages consultations 2016	April 2017	Senior Director, Policy and Research
1.2 Identify key programs and initiatives	Next federal official languages plan	April 2018	Senior Director, Policy and Research

Date of full implementation

April 2018

Recommendation 2 - Renforcement of results management

The evaluation finds that there are no indicators and no data collection for shared results by pillar or for the ultimate result. There is also little complementarity between initiatives and only a few federal partners are of the opinion that the Roadmap has a broader impact than the sum of its parts. The grouping of initiatives by pillar meant to allow partners to work to achieve shared results, but the current structure is actually more conducive to work in silos.

Recommendation

As part of the next multi-year official languages plan, it is recommended that the Assistant Deputy Minister, Citizenship, Heritage and Regions:

1. better define the shared horizontal results and indicators that will demonstrate that the next multi-year official languages plan has a broader impact than the sum of its parts.

Management response

Recommendation 2: accepted.

The OLB agrees that, although the Roadmap's logic model has horizontal strategic outcomes and a performance measurement framework that supports its effective management, the current results and accountability management structure and the next federal action plan would benefit from a better results framework and more specific and easily measurable performance indicators.

As part of the implementation of "deliverology" and the new Policy on Results, the OLB will ensure that the new departmental results framework is better designed. In addition, the new federal official languages action plan will be developed in accordance with these same requirements so that shared horizontal outcomes and indicators can be better defined.

Action plan

Action	Deliverables	Timetable	Program sponsor
1.1 Revise the departmental results framework for the next federal official languages plan	New federal official languages strategic action plan	April 2018	Senior Director, Policy and Research

Date of full implementation

April 2018

Recommendation 3 - Renforcement of results management

The evaluation noted a lack of detailed data to better identify certain needs for intervention (related to workforce development, regarding promising sectors in terms of economic development, the needs of OLMCs in rural or remote areas, and to better identify health needs). It also remains difficult to measure the results of multiple Roadmap interventions in OLMCs over a longer period.

Recommendation

To support the development of public policies and programs, the adaptation of interventions and decision-making based on evidence, it is recommended that the Assistant Deputy Minister, Citizenship, Heritage and Regions:

1. conduct ongoing research for and collection of data to support the initiatives of the official languages plan; and
2. conduct broader research (such as a net impact assessment of investment in the development of OLMCs, or a post-censal survey regarding the vitality of OLMCs).

Management response

Recommendation 3: accepted.

The development of good public policies and programs requires detailed and often multidisciplinary data that the OLB does not always have, which is why it is necessary to conduct a variety of research, studies and collection of data on official languages.

Research is also one of the methods used by the OLB to obtain data, and the Department will continue to support it and make use of it.

Moreover, the Government of Canada's new Policy on Results emphasizes the importance of associating program management and the achievement of results with evidence gathering. The new federal official languages action plan will go forward along the same lines, and its performance measures will be subject to indicators that have been further developed and will require data collection using different research instruments.

The OLB has a long tradition of data collection, sharing and analysis related to the vitality of its official-language communities. The publication of the 2016 Census language data in August 2017 will give us the opportunity to update a series of statistical profiles and tools that will provide better context for government action.

The OLB also plans to conduct a longitudinal impact study of Official Languages Support Programs and their effect on community development and the promotion of linguistic duality.

Action plan

Action	Deliverables	Timetable	Program sponsor
1.1 lntegrate language data from the 2016 census in statistical profiles and tools. Share data with federal partners and communities.	Production of statistical profiles	November 2017	Senior Director, Policy and Research
1.2 Take research into account when developing the new federal official languages action plan	New federal official languages action plan	April 2018	Senior Director, Policy and Research
1.3 Conduct a longitudinal impact study of Official Languages Support Programs	Final report of the study	November 2019	Senior Director, Policy and Research

Date of full implementation

Fall 2019

Recommendation 4 - Renforcement of results management

The evaluation notes that, over the five-year plans and following the various resulting groupings of initiatives, the Roadmap has contributed to confusion that has lasted for several years between existing programming and new initiatives or new funds.

Recommendation

It is recommended that the Assistant Deputy Minister, Citizenship, Heritage and Regions:

1. implement a monitoring mechanism to distinguish new investments resulting from the next federal official languages strategy in order to link them to results, to better address the government's priorities in terms of results and transparency, and to comply with the 2016 Policy on Results requirements.

Management response

Recommendation 4: accepted.

The OLB is aware that the two previous Roadmaps did not clearly distinguish between new investments and their objectives and the investments and the objectives of existing programming, which made it difficult (if not impossible) to separate the results related to different investments/initiatives.

The new federal official languages action plan will be developed in accordance with the new Policy on Results requirements, and so as to clearly establish objectives and expected results for the new investments resulting from this new plan.

Action plan

Action	Deliverables	Timetable	Program sponsor
1.1 Develop the next federal official languages action plan so as to clearly establish objectives and expected results for new investments resulting from the next federal official languages plan.	New federal official languages action plan	April 2018	Senior Director, Policy and Research

Date of full implementation

April 2018

2. Chapter 2: evaluation of the Official Languages Coordination Program – part of the horizontal coordination of the Roadmap 2013-2018: education, immigration, communities

This chapter presents the results of the evaluation of the Official Languages Coordination Program – part of the Horizontal Coordination of the Roadmap for Canada's Official Languages 2013-2018 (hereinafter called the Official Languages Coordination Program- part of Roadmap or Roadmap Coordination Program). The evaluation examines the relevance, performance, efficiency and economy of the Roadmap Coordination Program, and also serves to inform the horizontal evaluation of the Roadmap 2013-2018 (above), seeking to improve the next multi-year plan on official languages and its accompanying coordination program.

Evaluation objectives

The evaluation covers the four fiscal years from 2013-2014 to 2016-2017, or since the last evaluation in 2012. In addition to managing the administration and delivery of Official Languages Support Programs (OLSP), the majority of their components or sub-components which are grouped and reflected in the Roadmap, the Canadian Heritage (PCH) Official Languages Branch (OLB) also has a mandate of providing the horizontal coordination of the Roadmap as well as the more general functions of coordination and governance of official languages (hereinafter the Official Languages Coordination Program). The Official Languages Coordination Program includes:

• Horizontal coordination of the Roadmap, which coordinates the work of the Roadmap partners;
• Interdepartmental coordination under section 42 of Part VII of the Official Languages Act (OLA); and
• Part of the horizontal governance of official languages at the federal level (including official languages issues that are not in the Roadmap, which are broader than the interdepartmental coordination of section 42 or which relate to other parts of the OLA).

This current evaluation covers only the Roadmap Coordination Program, but will often refer to the federal government Official Languages Coordination Program, since the various tools and documents—for example the logic model and the performance measurement strategy as well as many functions, responsibilities and resources—are interrelated.

2.1 Program description

2.1.1 Background and context

Many decisions have had an impact on the horizontal governance of official languages across the federal government. The various instruments, tools and mechanisms that emerged from this change have also shaped the current state of official languages governance and horizontal coordination, including but going well beyond coordination of the Roadmap.

The development of departmental responsibility for the official languages file had an impact on the administrative structure of the horizontal coordination of official languages. Following the creation of the position of Minister responsible for Official Languages, the Official Languages Secretariat (OLS) was established within the Intergovernmental Affairs Branch of the Privy Council Office (PCO) to support the Minister in the horizontal governance of official languages in the public service and in the implementation of the Action Plan for Official Languages 2003-2008. The OLS was then transferred to PCH in February 2006 but was a separate entity until 2010, when it was integrated with the Official Languages Support Programs Branch (OLSPB) as part of a transformation initiative within the department.^Footnote97 Note that it was not until 2007-2008 that the function of Minister of Official Languages was assigned to the PCH Minister. In this regard, it should be noted that since 2015, the position of Minister of Official Languages is no longer officially connected to PCH or any other department or agency. Moreover, this title was never part of the OLA or described in any reference text.^Footnote98 Finally, in 2013, administrative structures supporting the OLS and OLSPB were merged under one entity: the OLB. It comprises three directorates: Operations and Regional Coordination, Policy and Research, and Interdepartmental Relations and Accountability.^Footnote99

Horizontal governance mechanisms for official languages have evolved within the context described above. The first mechanisms included the Committee of Deputy Ministers of Official Languages (CDMOL) established in 1999, followed by the establishment of a Ministerial Reference Group on Official Languages (MRGOL) in 2001. CDMOL was created to provide integrated, high-level government leadership in official languages, and in particular to ensure "monitoring of official languages (the work of Parliamentary committees, reports from the Office of the Commissioner of Official Languages, dispute management, government priorities for official languages, etc.);strategic thinking on the Action Plan (strategy for review and renewal); (…) and on current official-language issues".^Footnote100 The MRGOL, meanwhile, was established to coordinate horizontal issues relating to official languages. The group included the Treasury Board Secretariat (TBS), Justice Canada and PCH. Both were dissolved in 2006. Therefore, there is no longer any department-level body responsible for the governance and coordination of the three key institutions^Footnote101 nor a Tier I committee (deputy minister) for official language issues. At present, the most influential committees and networks are the Council of the Network of Official Languages Champions and CADMOL, which are both Tier II (assistant deputy ministers or equivalent).^Footnote102 Furthermore, CADMOL was established after the CDMOL was dissolved. Since 2006, CADMOL has expanded its members to include federal institutions other than partners in the Action Plan, and subsequently in the Roadmap, and its mandate was also expanded in order to deal with federal official language issues beyond the Roadmap's purview.^Footnote103

Moreover, with the creation of the Champion Network in 1998, official languages champions were appointed in most of the partner federal institutions and in some Crown corporations. It is important to note, however, that its influencing role is limited to the extent that the Champion Network is not actually accountable.^Footnote104 Other mechanisms that have supported horizontal governance included the implementation of an Accountability Framework for Part VII of the OLA in 1994, and later, the Act to amend the Official Languages Act in 2005. The Accountability Framework for Part VII was developed to implement sections 41 and 42 of the OLA. More specifically, "section 42 of the Act mandates the Minister of Canadian Heritage, in consultation with other Ministers of the Crown, to encourage and promote a coordinated approach to the implementation by federal institutions of the commitments set out in section 41".^Footnote105 A few years later, shortcomings were identified in the Accountability Framework for Part VII. Therefore, in 1997, the PCH Minister and the TBS President signed a memorandum of understanding to increase the accountability of institutions. The Memoradum of Understanding (MOU) entered into force in 2000 and was replaced in 2003 by the Official Languages Accountability and Coordination Framework, a document that was annexed to the first Action Plan for Official Languages 2003-2008. This new Official Languages Accountability and Coordination Framework evolved to include:

• Enforcement procedures for the obligations under Parts I to V of the OLA;
• Commitments under Parts VI and VII of the OLA;
• The responsibilities of each federal institution;
• The definition of the policy and news coordination mechanisms; measures included in the Action Plan for Official Languages;
• A common government-wide communication strategy for all activities;
• A section dedicated to horizontal coordination.^Footnote106

It is also within the Official Languages Accountability and Coordination Framework that the concept of "key departments" (TBS, PCH and Justice Canada) was introduced. More specifically, the Official Languages Accountability and Coordination Framework indicates that these institutions must align "their efforts to ensure enhanced information-sharing and compliance of government documents, policies, programs, and initiatives" with respect to official languages. According to TBS, the Official Languages Accountability and Coordination Framework is still in force but has not been officially updated since it was established in 2003, despite the developments and changes since.^Footnote107

The Act to amend the Official Languages Act (2005) came into force to increase the accountability of federal institutions under Part VII of the OLA. Since 2005, institutions have had an obligation to take positive measures to "enhance the vitality of official language minority communities and promote the use of English and French in Canadian society".^Footnote108 It was followed by the integration of a new interdepartmental coordination approach in 2011 (following that of 2003-2004), which added to this obligation by requiring all federal institutions to submit annual reports of their section 41 implementation activities and outcomes to PCH. The Official Languages Accountability and Coordination Framework involves PCH in the implementation of sections 41 and 42 of the OLA, and the obligations of PCH under these sections are the responsibility of the Interdepartmental Relations and Accountability Directorate (formerly interdepartmental coordination), a component that is not part of this evaluation.^Footnote109 Finally, in light of these developments, the current horizontal governance structure does not only deal with the issues of the Roadmap, which often leads to confusion in the official languages world.

2.1.2 Objectives and outcomes

The Roadmap 2013-2018 is the third version of the Government of Canada's horizontal initiative on official languages. As mentioned in the introduction, the 2013-2018 version brings together 14 partner federal institutions which, along with PCH, will implement 28 initiatives grouped into three pillars: education, immigration and communities. The Roadmap Coordination Program coordinates the work of the partners. The target populations and key stakeholders are shown below.

Table 1: target population and stakeholders

Target population	Minister of Canadian Heritage, Minister responsible for Official Languages (not designated since 2015), senior management, the Roadmap partners, provinces and territories, stakeholders in Canada's French-language minority communities, stakeholders in Quebec's English-language minority communities and linguistic duality stakeholders.
Interested parties	The groups identified as the target population as well as the Office of the Commissioner of Official Languages (OCOL), Parliamentary committees (House of Commons and Senate) and stakeholders in key areas such as education, health and justice.
Implementing partners	The federal partners responsible for the implementation of Roadmap initiatives: PCH Justice Canada (JUS) Public Services and Procurement Canada (PSPC) Employment and Social Development Canada (ESDC) Health Canada (HC) Immigration, Refugees and Citizenship Canada (IRCC) National Research Council Canada (NRCC) Canada Council for the Arts (CCA) Innovation, Science and Economic Development Canada (ISEDC) & Federal Economic Development Initiative for Northern Ontario (FedNor) Federal Economic Development Agency of Southern Ontario (FedDev) Canada Economic Development for Quebec Regions (CED-Quebec) Western Economic Diversification Canada (WD) Canadian Northern Economic Development Agency (CanNor) Atlantic Canada Opportunities Agency (ACOA) The three federal institutions responsible for official languages coordination and accountability activities: PCH (horizontal coordination of official languages) Official Languages Law Group within Justice Canada (OLLG-JUS) Official Languages Centre of Excellence within the Treasury Board Secretariat (OLCE-TBS) Official languages governance and support mechanisms: Committee of Assistant Deputy Ministers on Official Languages (CADMOL) Executive Committee of Assistant Deputy Ministers on Official Languages (Ex-CADMOL) Departmental Advisory Committee on Official Languages (DACOL) Crown Corporations Advisory Committee on Official Languages (CCACOL) Coordinating Committee on Official Languages Research (CCOLR) Official Languages Directors General Forum Official Languages Best Practices Forum Language Rights Practice Group Official Languages Champions Network Network of contacts for the implementation of section 42

Given that, pursuant to section 42 of Part VII of the OLA, the OLB at PCH is responsible for interdepartmental accountability and coordination activities in support of the Minister of PCH and senior officials of federal departments and agencies in the Government of Canada's official languages activities that relate to section 41, the Roadmap Coordination Program is directly connected to the Government of Canada's official languages obligations under Part VII. It is also connected to the PCH Program Alignment Architecture (PAA) for 2015-2016, where it is represented by the sub-activity "Official Languages Coordination Program" and contributes to the "Official Languages" program activity, which contributes to Strategic Outcome 2, "Canadians share, express and appreciate their Canadian identity".

There is no specific logic model for the Roadmap Coordination Program, but rather a common logic model for the Roadmap and for the horizontal coordination of official languages (Figure 1), the deliverables of which are:

• Roadmap partners provide relevant information on the implementation and outcomes of their initiatives, including results in terms of efficiency and economy.
• The decisions of the Minister of Canadian Heritage and senior management are informed by thorough analyses, relevant data and useful information.
• Governance is enhanced for greater efficiency and better coordination.

These results lead to the following ultimate outcome: "Canadians live and thrive in both official languages and recognize the importance of French and English for Canada's national identity, development and prosperity". (Canadian Heritage, 2015, p.3).

Figure 1: logic model
Official Languages Coordination Program (PCH) logic model

Ultimate Outcome	Canadians live and thrive both official languages and recognize the importance of French and English for Canada's national identity, development and prosperity.
Immediate outcome	Roadmap partners provide relevant information on the implementation and results of their initiatives, including efficiency and economy results Decision making by the Minister of Official Languages and senior executives is well informed because they receive thorough analyses, relevant data and useful information Governance is strengthened for greater efficiency and better coordination
Outputs	Policy statements, Memorandums to Cabinet, government responses Submission to TB Interdepartmental committees Improved horizontal management, accountability and governance framework Strategic input and advice Internal and external communications strategy New system for managing information on OL performance
Activities	Plan and coordinate the governance, accountability and reporting of Roadmap partners (interdepartmental collaboration) Support the Minister of Canadian Heritage and Official Languages senior executives and partners with strategic input and advice Develop strategic governmental-wide official languages policies
2013-2018 Roadmap initiative	Horizontal official languages coordination
-	PCH Program Activity Architecture (PAA) Official Languages Coordination Program Official Languages Branch (OLB)

The evaluation notes; however, that the logic model above does not reflect all the roles and responsibilities for the OLB at this time. The evaluation also notes that there is no logic model for the horizontal governance of the official languages at the federal level, reflecting all obligations and intended outcomes of the federal interventions, including strategies such as Roadmap.

2.1.3 Governance

As noted in section 2.1.1, the horizontal governance of official languages mechanisms have evolved. The official languages governance structure currently comprises three bodies. Their roles with respect to Roadmap coordination and, more generally, with respect to official languages governance are described below:

1. CADMOL is composed of the assistant deputy ministers of the Roadmap partner departments and agencies, official language coordination representatives at the same level from OLCE-TBS; the general counsel and director of OLLG-JUS and representatives from other key departments or agencies, as required. CADMOL reports to the Deputy Minister of Canadian Heritage. In addition to providing policy direction on official languages, the committee is responsible for supporting the Minister responsible for Official Languages or, since 2015, the Minister of Canadian Heritage, in Official Language file, including the Roadmap. It meets once a year to approve the planning and policy direction of the Roadmap.
2. The CADMOL Executive Subcommittee (Ex-CADMOL) is composed of the assistant deputy ministers of the federal institutions that have a key role in the implementation of the Roadmap, official language coordination representatives at the same level from OLCE-TBS and OLLG-JUS representatives. It reports to CADMOL and meets on a quarterly basis. This committee acts as a policy direction governance council with respect to horizontal official languages issues across government (including the Roadmap) and is responsible for:
   • sharing and developing strategies to deal with the challenges faced by all government in official languages (horizontal issues) and to strengthen collective leadership;
   • clearly defining the responsibilities of federal institutions and encouraging partnerships.
3. The Official Languages Directors General Forum is composed of DGs or directors from Roadmap partner federal institutions, the departments responsible for the coordination of official languages (OLCE-TBS and OLLG-JUS), the Office of Privy Council and Statistics Canada. It reports to Ex-CADMOL. The Forum's responsibilities are to:
   • participate in the coordination of Roadmap;
   • develop official language strategies while taking into account the results of relevant research;
   • seek to establish consistency among the priorities, programs and reports;
   • oversee the implementation of accountability and reporting mechanisms in accordance with the rules; and
   • coordinate consultation activities organized by Committee members.^Footnote110

2.1.4 Program resources

For the period 2013-2018, all of the PCH's interdepartmental, accountability and coordination functions totalled $10.5 million for the Official Languages Coordination Program. As noted in section 1.4.2, question 10, it is not possible to estimate the share of these resources allocated to the Roadmap Coordination Program only.

Table 2: program for the Official Languages Coordination Program (vote 1)

Year	Ongoing Funds	New Funds	Total
2013-2014	$390,000	$1,700,000	$2,090,000
2014-2015	$390,000	$1,700,000	$2,090,000
2015-2016	$390,000	$1,700,000	$2,090,000
2016-2017	$390,000	$1,700,000	$2,090,000
2017-2018	$390,000	$1,700,000	$2,090,000
Total	$1,950,000	$8,500,000	$10,450,000

Source: According to the Treasury Board submission in 2013, according to financial information provided by PCH on December 5, 2016.

2.2 Evaluation methodology

This section presents the methodology used for this evaluation. It comprises four data sources: a review of documents, literature, interviews with key stakeholders, and two expert panels that focused on the Roadmap but that also addressed issues related to horizontal coordination. Annex A contains the assessment matrix structured by evaluation question and indicators, as well as the proposed data sources.

2.2.1 Document review

The document review consisted of collecting and analyzing documentation relevant to the evaluation of the Roadmap Coordination Program according to the established evaluation matrix in order to formulate an overview of the initiative and identify information gaps that can potentially be met by other data sources. This documentation includes:

• Speeches from the Throne, budget announcements, Treasury Board submissions;
• Strategic internal documents and action plans;
• Reports on Plans and Priorities (RPP) and Departmental Performance Reports (DPR);
• Operational directives and management tools supporting the Roadmap;
• The Performance Measurement, Evaluation and Risk Management Strategy of the Official Languages Coordination Program;
• Minutes of meetings of the governance committees;
• Annual reports on official languages from Canadian Heritage;
• Parliamentary reports on official languages;
• The evaluation of coordination functions previously called the Accountability and Coordination Framework of the Roadmap for Canada's Linguistic Duality 2008-2013 (created in 2012), and documents monitoring the recommendations thereof;
• Program budgets and other financial documents, and documents on staff;
• Files on applications received and processed, or data compilations related to them; and
• Other relevant documents identified during the evaluation.

2.2.2 Literature review

The literature review conducted and written by the PCH Research Group helped to understand the current environment and to answer questions regarding relevance and efficiency, in particular by relying on surveys conducted outside federal institutions. It contributed to identifying the nature of the needs of Canadians and OLMC priorities, as well as the relevance of the various official languages programs/initiatives with respect to these needs and priorities.

2.2.3 Interviews and qualitative questionnaires with key stakeholders

Interviews were conducted with key stakeholders such as managers at PCH and at other partner federal departments and agencies (8), and with members of CADMOL and the Directors' Forum (4). The interviews gathered opinions and informed views about the relevance, design and implementation, as well as the effectiveness and efficiency of the Roadmap Coordination Program. Moreover, Roadmap partner institutions (12) also expressed opinions via a short questionnaire rather than in interviews. In light of the limited documentation on the program specifically, interviews introduced facts that enabled a deeper discussion on certain issues.

2.2.4 Expert panel

Two expert panels were held: one exploratory panel on the design of the Roadmap early in the evaluation period, and a second on the preliminary findings of the Roadmap evaluation at the end of the evaluation period. The panels were held within the framework of the horizontal evaluation of the Roadmap 2013-2018, but many of the issues discussed helped to inform the evaluation of the Roadmap Coordination Program. Each panel was made up of individuals from outside the circle of Roadmap partners and recipients recognized for their impartiality. The experts discussed trends observed in the data, the correlations and points of comparison between the findings and other documented research results. They also discussed the problems raised in the findings related to the Roadmap's mandate and on possible ways to resolve the difficulties observed.

2.2.5 Limitations and constraints

The evaluation is being conducted almost halfway through the five-year horizontal initiative, which limits the available data set. Moreover, the distinction between the Program for the Horizontal Coordination of Official Languages in general and the Roadmap Coordination Program specifically remains a challenge, and it affects all sources of data for this evaluation.

From the interviews with partners and the answers to the questionnaire they were given, it became clear that the Roadmap is so interwoven with other official languages initiatives that it is difficult to distinguish one from the other in terms of performance measurement, including performance of the Roadmap coordination function. The strategy for the performance measurement of coordination also reflects this complexity and confusion. Furthermore, findings from the interviews are based in part on the opinions of people with a direct interest in the Roadmap and its coordination. However, the approach adopted minimizes potential bias because the interviews were conducted by qualified evaluators, and respondents were asked to justify their perceptions or provide supporting examples. Although partners do carry out diligent vertical reporting of their progress towards the expected outcomes of the Roadmap, no data is collected about the established measures of horizontal coordination of the Roadmap function.

Moreover, there are very few documents relating specifically to the Roadmap Coordination Program Still, the document review provides context and additional information for the interviews, primarily with respect to what has emerged to date from the review of the horizontal governance of official languages undertaken by PCH since 2014, which includes the governance of the current Roadmap.

Finally, due to the low volume of documents in areas related to the nature of the Roadmap and coordination of similar horizontal initiatives, as expected, very few relevant documents were identified. Consequently, the literature review merely provides additional information.

2.3 Findings ‒ relevance

This section presents the main findings on the relevance of the Roadmap Coordination Program, namely its alignment with the needs of partners, and alignment with PCH and federal priorities, roles and responsibilities.

Summary

Section 42 of Part VII of the OLA states that the Minister of Canadian Heritage, in consultation with other Ministers of the Crown, shall encourage and promote a coordinated approach to the implementation of the commitments set out in section 41. Since most of its initiatives and outcomes are aligned with Part VII of the OLA, the Roadmap 2013-2018 is aligned with this legislative framework, as is the horizontal coordination function of the current Roadmap.

Furthermore, the horizontal management of official languages existed at the federal level before the Roadmap, and it is still necessary for the coordinated pursuit of common official language objectives. However, in light of the evolving horizontal coordination of official languages at the federal level and its accompanying governance structure, as mentioned in the previous evaluation, there will always be a need to clarify roles and responsibilities and to better align horizontal coordination actions, mechanisms and instruments for official languages at the federal level. The OLB is also conducting a review of the horizontal governance of official languages within the federal government, including horizontal coordination of the Roadmap.

2.3.1 Core issue 1: continued need for the program

All data sources attest to the continued need for the Roadmap Coordination Program. While horizontal management of official languages existed well before the implementation of the Roadmap, horizontal coordination of the Roadmap nevertheless is essential. Based on the interviews and short questionnaires, according to Roadmap managers and partners, there is a continued need for horizontal coordination of the Roadmap because of the whole-of-government nature of this initiative. The interviews and the results of the short questionnaire indicate that partners see the horizontal coordination of official languages (they do not distinguish between coordination of the Roadmap and the rest) as an important mechanism for maintaining emphasis on official languages, especially in departments where this is not a core function, and it remains an important function to ensure a coordinated government approach to official languages. However, while many believe that this mechanism was created to encourage partnership work, some believe that in practice, the work is still often done in silos. Some explain that there are currently no mechanisms in place to know who does what specifically in the context of the Roadmap. As indicated by the horizontal evaluation of the Roadmap, the partners seem more engaged in their own initiatives than in a whole-of-government initiative. Some partners also perceive gaps in the level of communication, and information exchange and sharing, which would have a negative impact on their ability to work together on joint initiatives.

Managers of the OLB have instead described horizontal coordination as an accountability tool for aligning and then measuring the outcomes of the various official languages initiatives against those of the Roadmap, and in particular to help partners measure their outcomes. While there is a clear need for coordination to measure outcomes, there is still some question as to whether the mechanisms are adequate.

And since there is a continued need for coordination, the Roadmap Performance Measurement, Evaluation and Risk Management Strategy (2013) identified the progress and actions taken to mitigate the risk of a lack of coordination in the implementation of the Roadmap. These are also related to certain recommendations that came out of the last evaluation, including:

• Clarification and communication of the roles and responsibilities of departments that have a specific responsibility under the OLA (PCH, Treasury Board Secretariat, Justice Canada);
• Supporting the governance structure as well as departments and agencies partners by sharing data analysis and research results to enable their use in strategic decision making related to policy development and programs on official languages;
• In co-operation with partner departments and agencies, identification of the mechanisms that will enable the ACFOL-OLS to better support the implementation of their initiatives, in particular strengthening the complementarity of initiatives and reducing duplication.^Footnote111

The measures implemented included: a review of the horizontal governance of official languages (including horizontal governance of the Roadmap) conducted by the OLB,^Footnote112 regular updates to the mandates of interdepartmental committees,^Footnote113 the establishment of meeting agendas to deal with files of interest to all committee members, including Roadmap partners,^Footnote114 presentation of an annual internal report for CADMOL approval on the progress and outcomes of the Roadmap^Footnote115 and inclusion on committees of federal representatives other than Roadmap partners.^Footnote116 Moreover, although it had ceased operations in 2012, the mandate of the Coordinating Committee on Official Languages Research (CCOLR) was revised in 2013, and meetings resumed in 2014. The aim of re-establishing the committee was to share information related to research on official languages, to create partnerships, and to inform decision-making^Footnote117 (also see question 6).

Despite the measures in place, according to the governance review conducted by PCH, federal governance of official languages continues to be criticized by both OLMCs and OCOL.^Footnote118 Moreover, OLMCs (through their representative organizations) have indicated that the government has no overall vision as to how it discharges its official languages responsibilities and, consequently, is neither able to present it to Canadians nor take corrective action when deficiencies are identified.^Footnote119 Furthermore, as regards horizontal coordination of the Roadmap, OCOL had recommended in 2013 that:

"as part of their respective responsibilities, the Minister of Canadian Heritage and Official Languages and the President of the Treasury Board: develop a new horizontal management and accountability framework for the Roadmap for Canada's Official Languages 2013–2018: Education, Immigration, Communities by October 31, 2014; ensure rigorous accountability and coordination of the 2013–2018 Roadmap; and continue to have an open dialogue with groups targeted by the investments in the 2013–2018 Roadmap and inform Canadians of the results".^Footnote120

While a new horizontal results-based management and accountability framework (HRMAF) for the Roadmap 2013-2018 was implemented, the Commissioner of Official Languages maintains that evaluating the federal government's official languages performance is difficult since it is not communicated to the general public. As indicated in the Standing Committee on Official Languages report on the new Action Plan for Official Languages: "[w]ithout access to the HRMAF, it is hard to understand how the Roadmap functions and how it works with other components of the Government of Canada's Official Languages Program".^Footnote121 Moreover, as pointed out during the governance review, in addition to annual reports on official languages, few documents are intended to communicate to Canadians the efforts and actions made by all of government in the horizontal governance of official languages and the Roadmap.^Footnote122

As mentioned above, some Roadmap managers and partners believe that the coordination function exists specifically for accountability obligations, while in reality it is much broader. Meanwhile, the public servants responsible for the implementation of section 41 of the OLA within the institutions have reported some confusion as to the respective mandates and responsibilities of networks and committees, and the ensuing difficulty of addressing the right forum to determine an issue.^Footnote123 In light of the development of federal horizontal coordination of official languages, the accompanying governance structure and the implementation of the Roadmap, the decision-making chain does not seem clear enough. As noted in the previous evaluation, there is therefore still a need to clarify the roles and responsibilities of various stakeholders, including key departments, in order to better align federal actions, mechanisms and instruments for the horizontal governance of official languages.

Finally, according to managers and partners, CADMOL, Ex-CADMOL and the DG Forum are essential for sharing information and best practices. However, as demonstrated by the governance review thus far, even with numerous mechanisms in place and clearer mandates for the various bodies, work is not always coordinated.

2.3.2 Core issue 2: alignment with government priorities

The Roadmap Coordination Program does align with federal priorities owing to the nature of its ultimate outcome of ensuring that "Canadians live and thrive in both official languages and recognize the importance of French and English for Canada's national identity, development and prosperity". Recall that section 42 of Part VII of the OLA states that the Minister of Canadian Heritage, in consultation with other Ministers of the Crown, shall encourage and promote a coordinated approach to the implementation by federal institutions of the commitments set out in section 41.^Footnote124

Furthermore, the Roadmap Coordination Program clearly aligns with the PCH program architecture. It is a component of the official languages coordination sub-program of the Official Languages Program (OLP). According to the objectives of the OLP, PCH plays "a coordination and support role among federal institutions in the implementation of their commitment to the development of official-language minority communities".^Footnote125 The Roadmap Coordination Program is therefore in line with the commitments under the OLA and with the objectives of the OLP, and even with PCH policy directions. Finally, the Roadmap 2013-2018 architecture is structured such that the outcomes of each of the 28 initiatives contribute to the achievement of the three pillars and, collectively, to achievement of the strategic outcome. The activities that coordinate the work of Roadmap partners are also aligned with the government-wide framework and with the outcome of a "diverse society that promotes linguistic duality and social inclusion".^Footnote126

To ensure that the Coordination Program continues to align with priorities and policy directions, following the recommendations from the last evaluation, the OLS—now part of the OLB—had committed to establish a working committee comprising representatives of the Roadmap partners, with a mandate to review the Accountability and Coordination Framework for Official Languages (ACFOL) of 2010. It was submitted to TBS as part of the presentation of the Roadmap 2013-2018 and was approved in November 2013; it became the horizontal results-based management and accountability framework for the Roadmap for official languages 2013-2018 (HRMAF). The HRMAF was adopted by the evaluation chiefs of federal partner institutions and by TBS. The OLB had also committed to undertake a review of government-wide governance over a period of two years. A working group composed of representatives from PCH, Justice Canada and TBS was therefore created. The group began its work in January 2014, and the review is still ongoing. As noted above in connection with Question 1, a preliminary discussion following the review suggests that there is still a need to clarify departmental responsibilities and accountability as they pertain to official languages. Some Roadmap managers and partners interviewed for this evaluation share this opinion, and some of them believe that the role of PCH in coordinating the Roadmap should be clarified with other federal institutions.

2.3.3 Core issue 3: alignment with federal roles and responsibilities

Among the Roadmap partners who spoke on this subject, it is clear that the role of horizontal coordination of this Roadmap is aligned with the OLA commitments of the government and PCH. More specifically, under the OLA, PCH is responsible for coordinating the implementation of the Part VII commitments (promotion of French and English and community development).

However, in light of the interviews and the literature reviewed, it was noted that following the transfer of the OLS from a central agency (PCO) to PCH, the PCH Minister and the OLB in practice inherited the roles and responsibilities pertaining to the horizontal coordination of official languages within the public service. Moreover, the evaluation found that since December 2015, there is no longer a designated minister responsible for official languages, even though the PCH Minister assumes the responsibilities in practice anyway. However, no minister has whole responsibility of all the different parts of the OLA, and the former title of Minister responsible for Official Languages was without legal basis, as the associated roles and responsibilities for horizontal management were not set out in the OLA or any other document.^Footnote127

According to the governance review to date, this contributes to a scarcity of interventions by other key departments such as the TBS and Justice Canada, which also have coordination responsibilities. Specifically, the OLA sets out PCH coordination responsibilities under Part VII, TBS under Parts IV, V and VI (communication with and services to the public, language of work, participation of English-speaking and French-speaking Canadians) and Justice Canada under the Department of Justice Act (providing legal services on behalf of the government).^Footnote128 Roadmap partners interviewed confirmed that the role of horizontal coordination of the current Roadmap logically falls to PCH due to its focus and expertise in supporting OLMCs and linguistic duality, which are found in Part VII of the OLA, but some acknowledge that over time, the PCH Minister has become automatically responsible for all matters pertaining to official languages, and that the role should be clarified.

Furthermore, in 2016, as part of the parliamentary report on "a new action plan for official languages", the Standing Committee on Official Languages stated that all federal institutions have obligations under the OLA, but that the Roadmap is limited to the initiatives of only 14 institutions. According to the Standing Committee, this has the negative effect of overshadowing the official languages work of other federal institutions that are not part of the Roadmap.^Footnote129 Recall that the Official Languages Accountability and Coordination Framework has not been updated since 2003, and that the responsibilities specified therein no longer reflect reality.^Footnote130 As noted above in connection with Issue 1 and 2, a preliminary discussion following the review on the horizontal governance of official languages suggests that there is still a need to clarify departmental responsibilities and accountability with respect to official languages, specifically because of the absence of a statement of roles and responsibilities related to the position of Minister responsible for Official Languages (not appointed since 2015).

2.4 Findings ‒ performance

2.4.1 Core issue 4: achievement of expected outcomes

This section presents the main findings of the evaluation on the achievement of the expected outcomes of the Roadmap Coordination Program. Question 5.1 on support to partners in the implementation of their initiatives is discussed first, then second, questions 4 and 5.2 of the matrix on data collection and improving accountability are discussed together. Questions 6 and 7 on information at the departmental level and effectiveness of the governance structure follow.

Summary: The simplified collection of data on progress and outcomes coordinated among partners seems to be working. In general, partners believe that reporting is effective, not too expensive, and that the OLB supports them. It contributes to the information provided to the PCH Minister and other ministers of the partner institutions. However, according to some partners, there is not enough information exchange on outcomes, and the level of detail in the annual report to CADMOL is insufficient.

There is no evidence on the impact of the merger of the OLS with the OLSPB. Nor is there any data on the effectiveness of the coordination function, although this is part of the performance measurement strategy; however, the partners generally seem to be satisfied.

Changes to the Roadmap governance structure since 2013 have been positive; committee mandates have been renewed, and the frequency of meetings and participation rate are appropriate.

The last evaluation detailed the concern of some stakeholders as to the potential negative impact of the transfer of the OLS from PCO to PCH on the support for horizontal coordination of official languages at the federal level (see in particular Savoie 2008 for OCOL). However, the current evaluation can only note that there is no information about the effect this transfer may have had or on the effect that the subsequent merger of the OLS with OLSPB (to become the Official Languages Branch) may have had in terms of the effectiveness of coordination mechanisms pertaining to the Roadmap or to official language in general.

Second, it is difficult to determine the quality and frequency of the advice and opinions provided by the OLB because it is not systematically documented. However, the partners generally seem to be satisfied with the collaboration. The OLB follows up on the advice provided in memoranda to Cabinet but not on ad hoc advice, including telephone discussions with partners, which are not documented but which, according to interviewees, seem to occur very frequently, especially as reporting deadlines approach. Some partners noted that they have access to useful and timely information and advice, while others did not address this issue. Some provided supporting examples, such as analysis produced to help answer questions from their minister, questions asked in the House of Commons or by parliamentary committees, and in the preparation of data for reporting (in this regard, see also questions 4 and 5.2).

Regarding the level of satisfaction, there is no formal mechanism for feedback on coordination mechanisms or specifically in relation to support from the OLB. A survey questionnaire of partners was abandoned owing to a lack of response. According to the OLB, partners express their concerns when they have them. A few stakeholders noted that there is no measure of the effectiveness of horizontal coordination of the Roadmap, suggesting that they are not aware of the strategy for this purpose (see questions 4 and 5.2 below). Finally, some partners have indicated that PCH could better support the coordination of partner activities and ultimately of Roadmap initiatives by exhibiting stronger leadership.

and

Following a recommendation of the 2012 evaluation, the OLB (OLS at the time) committed to strengthening data collection from Roadmap partners to extract the most relevant information in order to facilitate annual monitoring of progress and outcomes (recall that the previous performance measurement strategy was too complex and included some 200 indicators), and to inform Canadians about tangible results, a key commitment of the Roadmap. At the time, the OLS encouraged partners to better focus their data collection to emphasize the delivery of more direct services to citizens. The OLS also committed to "ensure that the structure for the annual report on progress and results is aligned with the Roadmap architecture and that the parameters for data collection from partners are clear and meaningful". This guided the structure of the annual report to CADMOL on progress and results. However, the evaluation noted that the results are not presented by pillar (the three pillars representing the essence of the architecture of this Roadmap). One explanation is that the performance measurement strategy does not include indicators to measure common outcomes by pillar and the ultimate outcome. Also, the structure of the Annual Report on Official Languages for the general public was reworked and simplified in 2015. However, as is the case with the report to CADMOL, information on the results of the Roadmap are not presented according to its architecture.

According to the Performance Measurement, Evaluation and Risk Management Strategy of the Official Languages Coordination Program (2013) overall, Roadmap partners must submit relevant information on the implementation of their initiatives and on the outcomes achieved. This simplified strategy includes 10 indicators supporting the three strategic outcomes of the program (described in section 2.1.2). According to the documentation, besides the survey on partner satisfaction with the coordination function, which was eliminated due to a lack of response (see finding related to question 5.1 above), data on the nine indicators are systematically collected.

Furthermore, starting in 2013, there was a shift in the collection of data for Roadmap reporting. The current strategy is an aggregation of information by the OLB based on the reporting per initiative and per partner, to demonstrate advancement towards their overall outcomes (as opposed to reporting from each partner with respect to common outcomes). Performance measurement focuses instead on the well-defined actions of each initiative and their effects, and not on the outcomes and impact of the Roadmap (related to the findings in question 4). The evaluation was unable to demonstrate progress toward the expected outcomes by pillar and ultimate outcome, as the performance measurement strategy of the current Roadmap does not include indictors to measure this progress.

As summarized in Table 3 below, the OLB is responsible for receiving information on the performance of each of the initiatives, and coordinating Roadmap reporting as a whole through the Canadian Heritage RPP and DPR. The PCH commitment also includes an annual report on the results and progress achieved by the various Roadmap initiatives, to be submitted to CADMOL.

This evaluation considered the reports listed below for the first two or three years, in the case of annual reports, and they have contributed some answers to several of the evaluation questions.

Table 3: partner reporting with respect to the Roadmap Coordination Program

Reports	Prepared by	Prepared for	Frequency
Progress Report	All Roadmap partners	OLS	Annually
Report on Results	All Roadmap partners provide an individual report. Integrated report prepared by the OLS.	CADMOL	Annually
Final horizontal evaluation report	The partners provide the input. The OLS prepares the integrated report.	CADMOL and all government Canadians	Once, on March 31, 2018
Summary report of the results	Partners provide input. The OLS drafts the consolidated report.	CADMOL and all government players Canadians	Once, on March 31, 2018
Report to Parliament (Departmental Performance Report)	PCH	Parliament	Annually
Evaluation	The ESD at PCH	PCH Treasury Board	Every 5 years

Note: The OLS is now integrated with the OLB

Source: PCH. (2013). Performance Measurement, Evaluation and Risk Management Strategy of the Program for the Horizontal Coordination of Official Languages (2013).

The Roadmap Coordination Program supports partners in their reporting through tools such as HRMAF, which includes the risk management strategy for the Roadmap. Moreover, a simplified template was developed to collect information from each partner, combining all the annual reporting requirements. According to PCH and some of the partners, this has had a positive effect on the quality of the information provided and on timelines. Moreover, the interviews and short questionnaires distributed to partners highlighted some of the benefits to results associated with the coordinated collection of data as it exists today:

• The reporting requirements for horizontal coordination can lead to improved internal reporting at each department.
• They can also enhance knowledge of best practices and lead to improvements to the Roadmap or some of its initiatives.

Many partners believe that reporting is effective, not too expensive, and that the OLB supports them. According to some managers and partners, OLB analysts work with partners to ensure that reporting and good relationships are established, and in some cases the quality of information has improved, and partners have improved their ability to measure outcomes and/or the number of questions has decreased. This contributes to the information provided to the Minister of Canadian Heritage and other ministers of the partner institutions. Furthermore, this allows the OLB at PCH to identify potential risks and develop mitigation strategies and an appropriate response.

Yet others point out that there is always room for improvement. There are questions about the quality of data from some partners, but no details were provided in this regard. Managers and partners also pointed out that certain partners do not provide information on the efficiency and economy of their initiatives, and this information is in fact not present in all the templates completed by the partners on an annual basis, but the evaluations of individual initiatives available to date do include such information. Others say that the performance measurement strategy is not clear with respect to the Roadmap versus other official languages activities.

With respect to the level of satisfaction, some partners confirm that performance measurement works well for their purposes, i.e., "vertical" reporting on their initiatives. However, there does not seem to be any horizontal reporting or accountability for common outcomes of the Roadmap, and some partners believe that there is no feedback about the information provided for reporting in general, and that they are merely responding to a PCH need to compile information. There is not enough exchange of information on outcomes; it is primarily a one-way process. Furthermore, according to some federal representatives, the internal annual report to CADMOL is too general and does not provide useful details on the activities and outcomes of the various Roadmap partners.

The recent parliamentary report on a new action plan also raises concerns about the accountability and transparency of federal official language interventions, and a flaw in the accountability of Roadmap partners more specifically:

"In terms of accountability, it can be a challenge to identify official languages expenditures and their source. The difficulty lies in the fact that there is no central agency responsible for official languages, and each department is responsible for managing its own official languages programs and budgets." Furthermore, "information on programs and expenditures is not compiled or presented consistently. As a result, it is impossible to obtain a comprehensive view of federal activities to promote the official languages and develop OLMCs. Treasury Board (TB), which is responsible for parts IV, V, VI and VIII of the Official Languages Act, and PCH, which is responsible for Part VII, prepare annual reports on implementation of the Act. These reports are prepared using the reviews on official languages produced by federal institutions concerning their implementation of the Act. However, as of 2011, institutions submit their official languages review according to a three-year cycle. As a result, the annual reports prepared by PCH and TB do not give a comprehensive view of accomplishments throughout the federal public service." Finally, the parliamentary report points out that plans, spending and results for the 2008-2013 and 2013-2018 Roadmap can be tracked using the horizontal initiatives database on the TB website. Although this information is available, PCH concedes that there is room to "improve the way [it] is presented".^Footnote131  It is important to note that this is more of an inventory of activities and outputs than progress towards the expected outcomes of the Roadmap.

The stakeholders interviewed and surveyed are divided on this question. According to the managers of Roadmap coordination, the PCH Minister is well informed, whereas there is insufficient information exchange according to some partners, and what information is exchanged is not detailed enough to inform their managers or minister.

OLB managers indicate that the reporting template is completed three times a year; amalgamated information from partners is used for the DPR and RPP and to inform management and the PCH Minister. Furthermore, more detailed analyses are produced on request for the Minister, for example, for the Ministerial Conference on the Canadian Francophonie (MCCF) or on an ad hoc basis for partners. With respect to the level of satisfaction of policymakers with the availability, usefulness, quality and timeliness of the information available to them, the only documented indicator is the percentage of opinions and advice provided to senior management within the required time, but this is available only for the Official Languages Coordination Program as a whole. Therefore, it is unable to directly answer the question here. However, some examples were identified in the documentation of how stakeholders are kept informed of certain aspects, including the meeting agendas of the three governance committees and the minutes of public consultations. Managers also stressed the importance of analyses incorporated into the contents of the DPR, RPP and the Annual Report on Official Languages to Canadians, and sharing with partners via the internal annual report to CADMOL. It should be noted that despite the initiatives, some stakeholders, such as OLMC representatives and OCOL, have raised some concerns over the lack of sufficient communication to Canadians, particularly with regard to the horizontal governance of official languages (refer to the findings to question 1).

Moreover, as indicated in the findings to questions 4 and 5.2 above, according to some partners, there is an insufficient exchange of information on the outcomes, and the information collected in the accountability process is not detailed enough to inform their managers or ministers.

Managers and the documentation, however, point to other progress since 2013 in the coordination of efforts to produce and exchange information. While it ceased operations in 2012, the CCOLR mandate was revised in 2013; it was renamed to the Working Group on Official Languages Research (WGOLR), and meetings resumed in 2014. It is also expected that the Working Group will now report to CADMOL on its activities annually. The purpose of re-establishing the WGOLR is to share information related to research on official languages, to create partnerships, to inform decision-making and to support the renewal of the Roadmap and the importance of forums with external researchers.^Footnote132 The OLB also has a new institutional database on various programs affecting OLMCs and which is accessible to all partners. It was mentioned by the coordination managers, but there is no documentation as to its content.

Furthermore, outside the special edition of the Official Languages Best Practices Forum in 2014, PCH sent a special invitation to all the key contacts in the implementation of Part VII of the Act to an information session on the review of the horizontal governance of official languages in February 2015. The session informed official languages stakeholders of the objective of the review, the approach and next steps. As indicated, the review is not finished, but the information available at the time of the evaluation shows that the review of the horizontal governance of official languages offers some ideas about the horizontal coordination of official languages in the federal system (beyond coordination of the Roadmap), and these were included when relevant to the evaluation questions.

Besides the complexity and confusion outlined in questions 1, 2 and 3, according to the interviews, short partner questionnaires and the documentation, the governance structure for the horizontal coordination of the Roadmap is working reasonably well, and changes made to it since 2013 have been positive.

In particular, the committees' mandates were revised in 2013 and are now reviewed regularly; the frequency of meetings is appropriate, and additional meetings are held as needed. Moreover, participation is appropriate, and certainly higher than in the past (low turnout had been mentioned in the last evaluation). Moreover, the annual report to CADMOL indicated that in 2013-2014, "partners met regularly and discussed on official languages related issues";^Footnote133 and the Performance Measurement, Evaluation and Risk Management Strategy (updated in 2015-2016) notes that since April 2013, the percentage of representatives of federal institutional partners in attendance at meetings is relatively high "(on average): CADMOL 83%, Ex-CADMOL 83%, DG Forum 72%."^Footnote134 According to the documentation available at the time of the evaluation, between 2013 and 2016, there were two CADMOL meetings, nine Ex-CADMOL meetings and sixteen DG Forum meetings.

Coordination managers acknowledge that it remains a challenge to find ways to meaningfully assess the effectiveness of the governance structure, including the work of committees—beyond counting meetings and the turnout rate indicated above—in order to examine the quality of informed decision-making, effective communication and implementation of decisions. Some indicators emerged from the interviews and short questionnaires to partners. These confirm that partners can put questions to the committees directly, or they can first address the OLB for referral to the appropriate body. However, partners indicate that many of the meetings are devoted solely to the exchange of information, go well beyond the Roadmap, and that the information does not necessarily circulate from executives who sit on the committees to the director or analyst level, leading some to suggest adding a director-level committee as a solution. However, some managers and partners confirmed that working groups were established for specific issues, and that their findings were then shared with the governance committees. This process seems to work well.

However, the review of the horizontal governance of official languages identified challenges that, even though they go beyond the horizontal coordination of Roadmap initiatives, are still relevant insofar as they add to the confusion expressed in questions 1, 2 and 3. First, jurisdictions overlap and there is insufficient communication between the Network of Official Languages Champions and CADMOL. Furthermore, as indicated in section 2.1.1, the governance and decision-making structures that bring together stakeholders (ministers and deputy ministers) directly accountable for the implementation of the Act are responsible for adopting adequate measures to carry out their responsibilities.^Footnote135 However, there has been a break in the chain of accountability since the dissolution of the high-level committees (MRGOL and CDMOL); the most influential committees are CADMOL and the Champion Network, but they have no established decision-making responsibilities. Added to this is the fact that there is currently no minister officially responsible for official languages.

Two unexpected impacts were mentioned during the evaluation. First, as mentioned in connection with question 3, following the merger of the OLS with the OLB, the administrative structure supporting OLSPs ended up supporting the coordination of the Roadmap and then the horizontal coordination of official languages at the federal level. This was not expected in accordance with the HRMAF established in 2013.

Second, HRMAF does not clearly set out any expectations with respect to the coordination of public consultations of the Roadmap partner departments, whereas some partners make a joint effort and others do not. According to one partner, this multiplies public consultations and leads to burnout of the communities and the groups that represent them.

2.4.2 Core issue 5: demonstration of efficiency and economy

This section presents the key findings on demonstration of efficiency and economy.

Summary: the review of resources found that since the administrative structures of the OLS were merged with the former OLSPB (now the OLB), it is impossible to clearly identify the share of human and financial resources specifically dedicated to the horizontal coordination of the Roadmap, since the data now cover the resources of the Official Languages Coordination Program as a whole.

There are also no data concerning potential alternatives, potential improvements to resource use or duplication with other initiatives.

None of the available documents helps to address this issue and none of the stakeholders interviewed could comment on this.

Allocation of resources

Table 4 provides a comparison of the additional resources provided in 2013 and the reconciliation of current financial data from the OLB. This table shows the differences between the funds granted in 2013 for Roadmap coordination, the funds allocated to Roadmap coordination according to departmental performance reports and actual expenditures. This comparison is available for three fiscal years of the current Roadmap.

This examination of resources shows that the $10,450,000 over five years and the 13 full-time equivalents (FTEs) allocated to the OLS in 2013 are redistributed in the budget of the OLB's for the Official Languages Coordination Program. As indicated above (section 2.1), the coordinating role of the OLB now consists of three parts. According to OLB management, when the funds were approved by the Treasury Board, the third role—that of the broader interdepartmental coordination—was not included. Hence, the difference between the funds planned and allocated in 2013 and the expenditures presented in the DPR, which themselves reflect the three roles.

In 2013, the $10,450,000 allocated over five years included an additional $8,500,000 (new funds) plus $1,950,000 in ongoing funds. These additional funds were originally granted specifically for Roadmap coordination.

During the first three years of the five-year cycle, expenditures were around $2,090,000 million per year. However, while expenditures may vary, it bears repeating that these expenditures are now those of the Official Languages Coordination Program, with the share of financial and human resources specifically dedicated to the coordination of the Roadmap no longer being identifiable because it is combined with all the resources allocated to the OLB for the Official Languages Coordination Program. Human resources allocated to Roadmap coordination are actually included in the Policy and Research or Interdepartmental Relations and Accountability Directorate, which are part of the OLB and contribute to the Program for the Official Languages Coordination Program. However, the interviews show that at least four FTEs among the staff spend much of their time on Roadmap coordination, in addition to two managers who devote part of their time to it.

Table 4: reconciliation of financial data, program for the Official Languages Coordination Program (Vote 1)

Year	Planned per the DPR	Ongoing FundsFootnote4-1	New FundsFootnote4-1	TotalFootnote4-1	Difference between the DPR and planned spending	Actual spending
2013-2014	NA	$390,000	$1,700,000	$2,090,000	NA	$3,093,277
2014-2015	$3,661,185	$390,000	$1,700,000	$2,090,000	($1,571,185)	$2,900,504
2015-2016	$3,529,236	$390,000	$1,700,000	$2,090,000	($1,439,236)	$2,755,438
2016-2017	$3,731,189	$390,000	$1,700,000	$2,090,000	($1,641,189)	—
2017-2018	—	$390,000	$1,700,000	$2,090,000	—	—
Total	—	$1,950,000	$8,500,000	$10,450,000	—	$8,749,219

NA: not available.

Source: According to the financial information provided by PCH on December 5, 2016.

More cost-effective alternatives

In terms of possible improvements to resource use, the stakeholders interviewed indicated that:

• an increase in resources is required given the increased horizontal coordination and communication responsibilities;
• encouraging increased information sharing among partners will allow them to remain efficient, in particular by proactively identifying opportunities for collaboration between partners; and
• it is recommended to continue to combine reporting on the Roadmap and the Official Languages Program at the federal level to avoid duplication of effort for the departments that have responsibilities at both levels.

Finally, the governance review underway could provide alternatives for changes to the governance structure specifically.

This evaluation did not provide any potential answers to this question directly. Moreover, as noted in questions 4 and 5.2, some partners do not provide information on the efficiency and economy of their initiatives, which is an obstacle to addressing this issue.

While a question specifically on the performance measurement strategy was asked in the interviews and questionnaire to partners, namely whether it gathers relevant information on the outcomes of the Roadmap Coordination Program, most participants could not answer the question due to a lack of knowledge about the strategy. However, two partners did provide comments. One of them said that his agency has created a new type of report to be completed two or more years after the end of a funded project to measure its contribution to the longer-term deliverables of the Roadmap. Another said that the performance measurement strategy guided the intervention of his department; however, he did not make a direct link between the strategy and improvements to their programs.

Rather, the evaluation stresses that to the partners, Roadmap coordination seems to be more effective than before because of simplified performance measurement and because reporting is combined in one tool and is, therefore, more efficient for partners. In the interviews and questionnaires distributed to partners, some also suggested potential improvements that could have a positive impact on efficiency of coordination, such as:

• An additional governance committee for directors to overcome the problem of disclosure of management information at a more operational level in the various partner departments and agencies (see question 7 on this issue).
• Additional tools to support partners such as an official languages media watch by the OLB, an inventory of roles within the OLB, the use of online tools such as GCconnex, an intranet site, or a portal on coordination of the Roadmap and official languages;
• Dedicated resources for an official languages centre of expertise, or at least experts in various official languages topics.

In the minds of some partners and coordination managers, future improvements to governance could also lead to gains in efficiency. An opinion cannot be formed until the conclusion of the governance review.

Partners also noted that joint public consultations coordinated by multiple partners could lead to better coordination and cost savings for the participating institutions. As stated in the Roadmap evaluation, although joint consultations do exist, there are very few, and many departments and agencies instead hold consultations individually with OLMC representatives and other stakeholders.

2.5 Conclusions

Relevance

Section 42 of Part VII of the OLA states that the Minister of Canadian Heritage, in consultation with other Ministers of the Crown, shall encourage and promote a coordinated approach to the implementation of the commitments set out in section 41. Since its initiatives and outcomes are aligned with Part VII of the OLA, the Roadmap 2013-2018 is aligned with this legislative framework, as is the horizontal coordination function of the Roadmap.

The horizontal management of official languages existed at the federal level before the Roadmap, and it is still necessary for the coordinated pursuit of common official languages objectives. However, there is some confusion between the horizontality of the Roadmap, the broad leadership and coordination role conferred on PCH, and the "vertical" decision-making processes of each of the partner departments, which are not designed for horizontal collaboration. As noted in the last evaluation, there is room for improvement in the horizontal management of official languages at the federal level and in the horizontal coordination of the Roadmap to clarify the roles and responsibilities and better align governance, the chain of accountability and decision-making. The OLB is also conducting a review of the horizontal governance of official languages within the federal government, including horizontal coordination of the Roadmap.

Performance — effectiveness

The simplified collection of data on progress and outcomes coordinated among partners seems to be working. In 2013, there was a shift in how data is collected for Roadmap reporting. The current strategy involves aggregation of the information by the OLB based on the reporting per initiative, per partner. In general, partners believe that reporting is effective, not too expensive, and that the OLB supports them. It contributes the information provided to the PCH Minister and other ministers of the partner institutions. However, according to some partners, there is not enough information exchange on outcomes, and the level of detail in the annual report to CADMOL is insufficient.

With respect to information for the public, despite the initiatives undertaken, such as the redesign of the annual report to Canadians, stakeholders such as OLMC representatives and OCOL have nonetheless raised concerns over the lack of communication to Canadians, particularly with regard to the horizontal governance of official languages. The recent parliamentary report on the next plan, meanwhile, confirmed that in terms of the Roadmap, it is possible to track plans, expenditures and outcomes by consulting the database on horizontal initiatives found on the TB website. It is important to note that this is more of an inventory of activities and outputs than progress towards the expected outcomes of the Roadmap.

There is no evidence on the impact of the merger of the OLS with the OLSPB nor any data on the effectiveness of the coordination function. It is difficult to determine the quality and frequency of the advice and opinions provided by the OLB because it is not systematically documented. However, the partners generally seem satisfied. Some provided supporting examples, such as analyses produced to help answer questions from their minister, questions asked in the House of Commons or by parliamentary committees, and in the preparation of data for reporting.

Finally, changes made to the Roadmap governance structure since 2013 have been positive; committee mandates have been renewed, and the frequency of meetings and participation rate are appropriate. However, some partners suggest an additional committee at the Director level to improve information sharing. Moreover, as noted in connection with the relevance of the Official Languages Coordination Program, a preliminary discussion stemming from the official languages governance review conducted by PCH suggests that there is still a need to clarify departmental responsibilities and accountabilities in official languages. The most influential committees currently are CADMOL and the Champion Network, but they have no established decision-making responsibilities. Added to this is the fact that there is currently no minister officially responsible for official languages. While this goes beyond the horizontal coordination of Roadmap initiatives, it bears noting that this contributes to the confusion expressed.

Efficiency and economy

Since the merger of the OLS and the OLB, it is impossible to clearly identify the share of the human and financial resources dedicated specifically to the horizontal coordination of the Roadmap. The data include resources for both the coordination of the Roadmap and coordination of official languages. A value-for-money analysis is, therefore, impossible. There are also no data concerning potential alternatives, potential improvements to resource use or duplication with other initiatives.

2.6 Recommendations and management response

Recommendation 1 - Better align the governance of official languages

Acknowledging that this evaluation and the reflection stemming from PCH's review of the governance of official languages both suggest that there is still a need to clarify roles and responsibilities, as well as ministerial accountability regarding official languages:

Recommendation

It is recommended that the Assistant Deputy Minister, Citizenship, Heritage and Regions (in cooperation with the appropriate departments):

1. take the measures needed to better align actions, mechanisms and horizontal governance tools for official languages at the federal level, including those that are related to the horizontal coordination of the future federal official languages strategy;
2. update the 2003 Accountability and Coordination Framework for Official Languages; and
3. communicate to all stakeholders the measures put forth.

Those measures should help to better align the governance of official languages, particularly by improving the decision-making process and by clarifying ministerial roles and responsibilities.

Management response

Recommendation 1: accepted.

The OLB agrees that there is still some confusion about roles and responsibilities, as well as ministerial accountability regarding official languages. Measures are also needed to better align actions, mechanisms and horizontal governance tools as well as the accountability and coordination framework for official languages.

The Accountability and Coordination Framework of the Roadmap 2013-2018 was made public in fall 2016 to share information about the coordination mechanisms and the roles and responsibilities of various stakeholders.

The overall coordination of official languages will be reviewed within the context of the development of the next federal official languages action plan. The publication of the next plan will provide the opportunity to communicate clearly with all stakeholders in regard to the related accountability and coordination mechanisms.

Action plan

Action	Deliverables	Timetable	Program sponsor
1.1 Coordination measures taken within the framework of the next official languages plan and communication with all stakeholders	New federal official languages action plan	April 2018	Senior Director, Policy and Research

Date of full implementation

April 2018

Recommendation 2 - Improve reporting and communication

Simplified data collection regarding partners' progress and results appears to be working well. However, data to demonstrate efficiency and economics of the horizontal initiative are limited and, according to some partners, there is not enough exchange of information regarding results, at least not detailed information.

Recommendation

To contribute to improved reporting to PCH, to the sharing of information among partners, and ultimately to the demonstration of progress toward the broad results of the federal strategy, it is recommended that the Assistant Deputy Minister, Citizenship, Heritage and Regions:

1. establish measures and tools to improve the capacity of partners to report on their contribution to horizontal results and their efficiency.

Management response

Recommendation 2: Accepted.

We agree that the data and exchange of information on outcomes are sometimes limited, and the accountability and information sharing between partners need improvement.

The next federal official languages action plan will be developed in accordance with the new Policy on Results requirements and the OLB will continue to enhance and define measures and tools in order to report on their contribution to horizontal results and their efficiency.

Action plan

Action	Deliverables	Timetable	Program sponsor
1.1 Review the design of the new federal action plan for official languages	New federal official languages action plan	April 2018	Senior Director, Policy and Research

Date of full implementation

April 2018

Recommendation 3 - Improve reporting and communication

The evaluation notes that there is no data on the effectiveness of the coordination function. This aspect is not systematically documented. Moreover, it is impossible to clearly identify the portion of financial and human resources dedicated specifically to the horizontal coordination of the Roadmap compared with the Official Languages Coordination Program. An analysis of the optimization of resources is therefore impossible.

Recommendation

It is recommended that the Assistant Deputy Minister, Citizenship, Heritage and Regions:

1. implement a mechanism for monitoring the human and financial resources dedicated to horizontal coordination for the next federal official languages plan in order to link the expected results and be able to meet the requirements of the 2016 Policy on Results.

Management response

Recommendation 3: accepted.

The OLB recognizes that it is difficult to clearly identify the portion of financial and human resources dedicated specifically to the part of the Horizontal Coordination of the Roadmap compared with the Official Languages Coordination Program.

The OLB has responsibilities in regard to the development of policies and horizontal coordination that involve both the Department of Canadian Heritage under Part VII of the Act, and the coordination of the horizontal strategy for official languages. These are two components of the Official Languages Coordination Program.

Since the merger of the Official Languages Support Programs Branch and the Official Languages Secretariat (as part of the Deficit Reduction Action Plan - DRAP), the two coordination functions have been combined: the same teams, people, accountability mechanisms and budgets are used for the two components. This measure has resulted in improved accountability and economy, as well as significant gains in efficiency.

In the event that additional financial and human resources are provided for this purpose, the OLB will see to the implementation of an accountability mechanism that clearly separates the resources related to the horizontal coordination of the future federal action plan for official languages and those dedicated to the coordination of official languages. Without these resources, the OLB will continue with the current practice.

Action plan

Action	Deliverables	Timetable	Program sponsor
1.1 In the eventuality that additional financial and human resources are allocated for this purpose, the OLB would establish an accountability mechanism that clearly separates resources for horizontal coordination from the future federal action plan in Official languages and those devoted to the coordination of official languages. In the absence of such resources, OLB will continue the current practice.	N/A	N/A	N/A

Date of full implementation

N/A

Recommendation 4 - Improve reporting and communication

Stakeholders such as OLMC representatives and the Commissioner of Official Languages perceive a lack of sufficient communication aimed at Canadians:

Recommendation

It is recommended that the Assistant Deputy Minister, Citizenship, Heritage and Regions:

1. adopt other appropriate means of communicating more detailed information on expected results in order to address the government's priorities regarding results and transparency and the information needs of numerous parties interested in the results and impact of the future federal official languages strategy.

Management response

Recommendation 4: accepted.

The OLB notes a perceived lack of sufficient communication aimed at Canadians and sees that this needs to be addressed.

The OLB already communicates information about results and achievements of the Roadmap 2013-2018 in the Annual Report on Official Languages and in the Canadian Heritage Departmental Performance Report. These reports will be subject to continuous improvement in view of the next federal official languages plan and the new departmental results framework.

Action plan

Action	Deliverables	Timetable	Program sponsor
1.1 Improve the information in the online table of the new federal action plan for official languages	Online table of the Canadian Heritage Departmental Performance Report 2018‑2019	April 2019	Director, Interdepartmental Relations and Accountability
1.2 Improve accountability of the new federal action plan for official languages in the Canadian Heritage Official Languages Annual Report	Canadian Heritage Official Languages Annual Report 2018-2019	March 2020	Director, Interdepartmental Relations and Accountability

Date of full implementation

Spring 2020

2.7 Looking forward

Recognizing that the mandate letter for the Minister of Canadian Heritage includes the drafting of a new multi-year official languages plan, this section provides a brief summary of the main trends and other future prospects that the federal government will have to take into consideration in promoting linguistic duality and support for the vitality of OLMCs in the coming years.

Socio-political context

The policy and legislative framework in the provinces and territories continues to evolve

Federal involvement in official languages occurs in an increasingly dynamic environment that entails both advantages and challenges.

Since the first action plan was released in 2003, several federal departments and agencies have taken responsibility for increasingly specific niche areas relative to the development of official languages in Canada, particularly the areas of health, justice and immigration. This is in addition to the more traditional, but nonetheless fundamental, role that the Official Languages Support Programs play. These programs help strengthen the basic components for developing and enhancing the vitality of OLMCs, while also facilitating second-language learning, particularly through immersion programs.

Although official languages development activities have long been mainly a federal government responsibility, there is no doubt nowadays that the provincial and territorial governments play a significant role in that regard. Since the adoption of the Official Languages Act in New Brunswick in 1969 and the enactment of the French Language Services Act in Ontario in 1986, several provinces and territories have followed suit and adopted or strengthened their legislative framework or policies regarding the official language minority, as follows:

• In 2013, Alberta adopted regulations that specify the use of French in the courts;
• In 2013, New Brunswick amended its Official Languages Act, making several improvements;
• Also in 2013, Prince Edward Island enacted a new French Language Services Act, which requires government organizations in the province to provide services of equal quality in English and French;
• Since June 2016, services in French in Manitoba have been protected by the Francophone Community Enhancement and Support Act.

This environment also includes a variety of organizations that represent OLMCs and active and engaged sector organizations (arts and culture, health, justice, etc.). As the parliamentary committee pointed out: "To deal with the various levels of government, communities have established local, provincial and national organizations, and designated certain organizations to represent their interests on the issue of enhancing their vitality."^Footnote136 They underpin and defend a long tradition of community action in OLMCs. There is also a long tradition of pan-Canadian organizations contributing to the promotion of linguistic duality, such as Canadian Parents for French and French for the Future.

The specific situation in Quebec: distinction between language and community

On the issue of official languages development, Quebec represents a unique situation for both the provincial and federal governments. While acknowledging the importance of protecting and promoting the French language in Quebec, it seems particularly important to more actively support the development of the English-speaking communities in Quebec, particularly in the regions outside Montreal. In the evaluation of the entire Roadmap, it was found that some activities would have to be adapted to more effectively meet the specific needs of these communities. For example, the English-speaking communities in Quebec have not benefited from immigration initiatives as much as Francophone communities outside Quebec owing to the limitations imposed by the Canada-Quebec Accord. Moreover, the recent report from the parliamentary committee for the next federal multi-year plan on official languages recommends that PCH "work with the representatives of the English-speaking communities in Quebec to help federal institutions find innovative ways to foster the development of those communities, particularly in areas that require intergovernmental collaboration, while at the same time respecting Quebec's prerogatives."^Footnote137

The increase in multilingualism will continue

Over the past several years, demographic changes related to immigration point to an increase in multiculturalism and multilingualism in Canada, and, according to Statistics Canada's language projections, by 2036, the percentage of the population with English as a mother tongue and French as a mother tongue in Canada could decrease, while the portion of the population with a language other than French as a mother tongue could increase:

• English mother tongue: 59% in 2011, between 52% and 56% in 2036;^{Footnote o}
• French mother tongue: 21% in 2011, between 17% and 18% in 2036;
• Other mother tongue: 20% in 2011, between 26% and 31% in 2036.

These changes would affect populations defined by their first official language spoken (FOLS) differently:

• English FOLS: 75% in 2011, between 77% and 78% in 2036;
• French FOLS: 23% in 2011, between 20% and 21% in 2036.

In 2036, the French-English bilingual population could represent between 18.3% and 18.8% of Canada's population, up from 2011 (17.5%).^Footnote138

As the Office of the Commissioner of Official Languages pointed out: "Today, Canada has one of the highest per capita immigration rates in the world, and" … "Although this is a significant number, experts are relatively confident that Canada's multiculturalism policy will ensure continued political and economic integration of these newcomers. Maintaining the vitality of Canada's two official languages in the context of these changing demographics may prove rather more challenging, however."^Footnote139

Canadians' main perceptions

Official languages remain one of the foundations of the Canadian identity and are generally viewed favorably

According to the comparative analysis of trends in Canadian public opinion with regard to official languages conducted by PCH, Canadians' support for linguistic duality remains relatively high. In addition, this support has been growing over the past decade. Canadian's level of agreement with the fact that official languages make up a significant part of the Canadian identity was on average 6.7 among Anglophones and 8.0 among Francophones (on a scale of 0 to 10) in 2016, compared with 7.1 and 5.5 respectively in 2005.^Footnote140

The results of a survey done by the Office of the Commissioner of Official Languages (OCOL) in 2016 gave rise to a similar conclusion, indicating in particular that eight out of ten Canadians agree that "Having two official languages is one of the things that truly defines Canada". Ninety percent of respondents between ages 18 and 34 are in favor of bilingualism for all of Canada. For all age groups combined, the proportion was 73% in 2016. According to an Environics survey, it was 57% in 2003 and 63% in 2012, and, although the results are not directly comparable, the trend seems to be that the number is rising.^{Footnote141,Footnote142}

PCH's 2016 survey indicated that the Government of Canada is effective in protecting both of Canada's official languages (71% of respondents agreed) and that its policy on official languages strengthens national unity (67% agreed including 46% that strongly agreed).^Footnote143 The OCOL survey, which specifically referred to support for the Official Languages Act, revealed that 88% of Canadians surveyed support the Act's objectives. This support is higher (93%) among bilingual people (French-English).^Footnote144

The importance of access to services in the language of choice remains high

PCH's 2016 survey indicates that Canadians place a high value on access to services in their language, both government services and educational services, whether they lived in Quebec or elsewhere in Canada. Regarding access to education, Canadians overall thought it was slightly more important for members of minority-language communities to learn the minority official language (84%) than to learn the second official language (78%).^Footnote145 The OCOL survey confirmed that 96% agree that "federal services should be in English and French."^Footnote146

Learning the second official language is still important

PCH's 2016 survey also noted that both official languages still represent the second most important language to learn for a majority of Canadians. For unilingual Anglophones, French (52%) is the most important language to learn, followed by Spanish (21%) and Mandarin (6%). Among unilingual Francophones, English (83%) was the most important language to learn, then Spanish (12%).^Footnote147 The OCOL survey indicated that 65% of respondents did not agree that "It is more useful for children to learn to speak other languages, such as Mandarin or Arabic, than it is for them to learn to speak both official languages." This survey also confirms that there is significant support for teaching in the second official language. Two thirds of respondents (66%) believe that the provincial governments should make more spaces available in immersion programs.^Footnote148

Emerging needs and main concerns of OLMCs

There is considerable convergence in terms of the main concerns and priority areas of intervention identified in various studies and consultations carried out recently (2012 consultations for the next Roadmap, CADMOL consultations in 2014 and 2015, consultations with the PCH Minister, hearings before the parliamentary committee in 2016 for the next federal multi-year plan on official languages, reports from the OCOL on immigration in 2014 and early childhood in 2016, as well as this evaluation). As part of the evaluation, the partners and parties involved confirm that the Roadmap continues to meet well-established needs in education, health and justice. However, it was pointed out in all of the studies and consultations that there is a perception of under-funding of OLMC organizations and institutions and organizations that support duality. This is attributed to a freeze in federal contributions in the past eight to ten years. This adds to the challenge of finding a new generation to take over within these organizations and institutions and ensuring their survival.

In addition, according to recent studies and consultations, there is a call for greater support to meet the many emerging needs and growing concerns of OLMCs:

• In the area of early childhood and youth, to continue efforts to support a continuum of education in the minority official language from early childhood to university and move toward better integration of young people in the labour market to curb the exodus from minority-language communities;
• To help improve and maintain infrastructure in OLMCs, in particular to continue to provide education of equal quality in the minority language;
• To help address the high demand for second official language learning, particularly French immersion programs in some provinces and territories where the demand greatly exceeds the supply;
• To better meet the specific needs of seniors (healthcare, social services, adapted cultural activities, etc.) and the more evident consequences of population aging in OLMCs (increased need for a new generation to take over in OLMC organizations and institutions, decreasing proportion of labour market participants, etc.);
• To continue efforts to integrate and retain newcomers and strengthen intergovernmental collaboration with regard to immigration;
• To better determine the development needs of the workforce and identify promising economic development sectors;
• To help create physical and digital "French spaces" outside of Quebec; and
• To counteract the vulnerability of official language minority community media and facilitate their transition to using digital technology.

It was also highlighted in the evaluation of the Roadmap that there is a desire to see greater support for research in order to more effectively identify needs and steer government activities in the coming years, as well as to better determine the impact of federal activities, particularly in areas where they complement provincial or territorial activities.

Official languages governance

The desire for horizontality

As early as 2008, as part of his study on the horizontal management of official languages in the federal government, Donald Savoie recalled that "it would be difficult to imagine still more mechanisms or to point to any policy field within the federal government that has as many instruments to promote a coordinated or a horizontal perspective as does official languages policy. That said, all of these instruments and legal requirements have also, over the years, raised expectations in minority-language communities that the federal government will be more effective in promoting a horizontal perspective for official languages." Moreover, he concluded that "horizontality requires above all a firm and ongoing political commitment, targeted action, and a clear objective in order to be successful."^Footnote149 Although his thoughts concerned the governance associated with the first 2003–2008 action plan, the formula was kept and this strong and consistent government-wide commitment that Savoie was talking about then appeared in two other Government of Canada five-year horizontal official languages plans.

The evolution of official languages governance mechanisms

As the evaluations of the Roadmap and the Coordination Program point out, in recent years, several decisions had an effect on the federal horizontal governance of official languages. The various measures, tools and mechanisms resulting from this development have shaped the current state of official languages governance and horizontal coordination. Let us remember the evolution of departmental responsibility for official languages, which had an impact on the administrative structure for the horizontal coordination of official languages. This departmental responsibility for official languages was clearly assigned as early as 2007, but has not officially existed since 2015, even though it has remained a de facto responsibility of the Minister of Canadian Heritage. The structure has also changed, since the OLS, which was created within the PCO in 2003, was transferred to PCH in 2006, and then integrated with the OLSPB in 2010 to establish the current OLB. Let us also remember that, during this time, official languages governance mechanisms evolved as well. The first mechanisms were the Committee of Deputy Ministers of Official Languages (CDMOL) in 1999, followed by the Reference Group of Ministers on Official Languages (RGMOL) in 2001, which were set up to coordinate horizontal official languages issues, and, starting in 2003, to coordinate the first Action Plan. The CADMOL was established following the dissolution of the CDMOL in 2006. Since that time, it has expanded its membership to include other federal institutions, and not just the partners in the Action Plan and, subsequently, in the Roadmap. The CADMOL mandate has also been expanded to deal with official languages issues at the federal level that go beyond the strict framework of the Roadmap.^Footnote150 However, the Official Languages Accountability Coordination Framework, established in 2003 to specify the responsibilities of key departments (TBS, PCH and Justice Canada) has not been updated, despite the developments and changes that have occurred since its inception.

To ensure sound governance of official languages at the federal level, lessons learned from these developments must be applied, more specifically regarding departmental responsibilities and the governance and administrative structure. In addition, the roles of federal departments and agencies must be specified and clearly communicated in the future.

The "by and for" the communities

Following the 2016 hearings, the parliamentary committee concluded that "In order to support OLMCs, it is important to respect the existing community structure, especially the key role of organizations that represent OLMCs."^Footnote151 In their submission to the parliamentary committee, Cardinal and Léger (2016) pointed out that "official language governance cannot be limited to a greater proximity relationship between the government and the communities [translation]" and that, in particular, the government must respect and value the expertise of OLMC organizations.^Footnote152 Moreover, according to the committee's report, community representatives are asking that the next federal multi-year plan on official languages "re‑establish OLMCs as the focus of the government's official languages activities" and that program development and delivery be guided by the concept of "by and for" the communities.^Footnote153 The FCFA and the QCGN have insisted for several years on this concept that they deem essential to the enhanced vitality of OLMCs.

Conclusion

As stated by Cardinal and Léger, "English and French in the country are a central part Canada's high degree of diversity [translation]."^Footnote154 In the coming years, it will be even more important that the government continue to encourage and enhance provincial and territorial government activities, to strengthen the capacities of OLMC organizations, institutions and community networks, and to support the promotion of linguistic duality. Lastly, it is up to the federal government to provide governance for official languages that includes an overall vision and establishes our status as a bilingual country.

Annex A: evaluation matrices

Matrix of Roadmap 2013-2018 evaluation questions

Table 1.1: Relevance of the Roadmap

Evaluation questions	Indicators	Sources/methods
To what extent does the Roadmap meet the needs and aspirations of Canadians?	Extent to which the Roadmap continues to address a demonstrable need and takes into account the needs of Canadians and the priorities of OLMCs Changes observed since the Roadmap for Linguistic Duality 2008-2013, which may have an impact on official languages needs Public opinion on official languages (OL) Perceptions of key stakeholders Expert opinion	Review of administrative documents: Treasury Board submissions, business plans, Reports on Plans and Priorities (RPPs), Departmental Performance Reports (DPRs), Speeches from the Throne, budget announcement, Roadmap partner evaluation products, community strategic plans and their reviews, and other documents. Review of the literature and census data Analysis of the results of existing surveys Canadian public opinion surveys OLMC case studies Interviews with: PCH senior management responsible for the Roadmap Federal Roadmap partners Roadmap stakeholders. Expert panels

Table 1.2: Alignment with government priorities

Evaluation questions	Indicators	Sources/methods
To what extent is the Roadmap aligned with the priorities of Canadian Heritage, departmental Roadmap partners and the federal government overall?	Linkages between Roadmap objectives and federal government priorities Linkages between Roadmap objectives and the strategic objectives of Canadian Heritage and partners Perceptions of key stakeholders	Review of administrative documents: Treasury Board submissions, RPPs, DPRs, Speeches from the Throne, budget announcement, Roadmap partner evaluation products, community strategic plans and their reviews, and other documents. Interviews with: PCH senior management responsible for the Roadmap Federal Roadmap partners Roadmap stakeholders

Table 1.3: Alignment with federal roles and responsibilities

Evaluation questions	Indicators	Sources/methods
Is the Roadmap consistent with the federal government's role and responsibilities?	Link between the Roadmap and the government's official languages role and responsibilities Perceptions of key stakeholders Expert opinion	Review of administrative documents: Treasury Board submissions, RPPs, DPRs, Speeches from the Throne, OL Act and Regulations, budget announcement, Roadmap partner evaluation products, community strategic plans and their reviews, and other documents Review of the literature and census data Interviews with: PCH senior management responsible for the Roadmap Federal Roadmap partners Roadmap stakeholders Expert panels

Table 1.4: Design and implementation of the Roadmap

Evaluation questions	Indicators	Sources/methods
Is the specific contribution of the Roadmap within all Government of Canada activities in official languages clearly defined and understood?	Clarity of the Roadmap mandate Perceptions of key stakeholders Expert opinion	Review of administrative documents: Treasury Board submissions, RPPs, DPRs, Speeches from the Throne, OL Act and Regulations, budget announcement, Roadmap partner evaluation products, community strategic plans and their reviews, and other documents Interviews with: PCH senior management responsible for the Roadmap Federal Roadmap partners Roadmap stakeholders Expert panels
Is the logic behind the design of the three-pillar Roadmap and of its contribution to the overall outcomes properly understood and considered to be adequate?	Consistency between initiatives and overall outcomes Complementarity between initiatives Perceptions of key stakeholders Expert opinion	Review of administrative documents: Treasury Board submissions, RPPs, DPRs, Speeches from the Throne, OL Act and Regulations, budget announcement, Roadmap partner evaluation products, community strategic plans and their reviews, and other documents Interviews with: PCH senior management responsible for the Roadmap Federal Roadmap partners Roadmap stakeholders Expert panels
What is the added value of grouping federal partner initiatives under the Roadmap?	Nature of the collaboration between the Roadmap initiatives Perceptions of key stakeholders Expert opinion	Review of administrative documents: Treasury Board submissions, RPPs, DPRs, Speeches from the Throne, OL Act and Regulations, budget announcement, Roadmap partner evaluation products, community strategic plans and their reviews, and other documents Interviews with: PCH senior management responsible for the Roadmap Federal Roadmap partners Roadmap stakeholders Expert panels
Are there any programs/initiatives that are not part of the Roadmap but should be?	Other relevant official languages programs/initiatives with respect to expressed needs Consideration of the complementarity between support to OLMC vitality and linguistic duality Perceptions of key stakeholders Expert opinion	Review of administrative documents: Treasury Board submissions, business plans, Reports on Plans and Priorities (RPPs), Departmental Performance Reports (DPRs), Speeches from the Throne, budget announcement, Roadmap partner evaluation products, community strategic plans and their reviews, and other documents Review of the literature and census data Analysis of existing survey results Interviews with: PCH senior management responsible for the Roadmap Federal Roadmap partners Roadmap stakeholders OLMC case studies Expert panels

Table 1.5: Achievement of expected outcomes

Evaluation questions	Indicators	Sources/methods
To what extent has the Roadmap contributed to the achievement of its outcome in terms of education, i.e., that "Canadians benefit from education and training opportunities in their first official language and for learning the other official language of the country, and from access to technological tools, taking advantage of the many economic, cultural and national identity advantages resulting from these"?	As regards education and training services (vocational, literacy, postsecondary) in the minority language: Evidence of access to services and the degree of proximity to these services Assessment of the quality of services Increase in the school population Academic performance compared to education in the minority language As concerns second-language education and training services: Evidence of access to services and the degree of proximity to these services Assessment of the quality of services Increase in the school population Evidence of acquired second-language skills Evidence that Canadians have access to and take advantage of language technologies Evidence of the economic benefits created by the language technology industry Public opinion regarding OL Perceptions of key stakeholders Expert opinion	Review of administrative documents: business plans, RPPs, DPRs, Roadmap partner evaluation products, OLB annual reports and files, community strategic plans and their reviews, and other documents Quantitative data from the OLB Review of the literature and census data Analysis of existing survey results Interviews with: PCH senior management responsible for the Roadmap Federal Roadmap partners Roadmap stakeholders Canadian public opinion surveys OLMC case studies Expert panels
How has the Roadmap contributed to the achievement of its outcome in terms of immigration, i.e., that "newcomers' language skills in either official language are strengthened and enable them to further contribute to Canada's economic, social and cultural development"?	Evidence that newcomers' language skills in either official language have been reinforced Evidence of newcomers' contribution to the economic, social and cultural development of Canada Evidence of the contribution of the immigration pillar to the demographic vitality of Francophone OLMCs Perceptions of key stakeholders Public opinion regarding OL Expert opinion	Review of administrative documents: business plans, RPP, DPR, evaluation products of CIC partners, OLB annual reports and files, community strategic plans and their reviews, and other documents Quantitative data from the CIC Review of the literature and census data Analysis of existing survey results Interviews with: PCH senior management responsible for the Roadmap Federal Roadmap partners Roadmap stakeholders Canadian public opinion surveys OLMC case studies Expert panels
How has the Roadmap contributed to the achievement of its outcome in terms of communities, i.e., "the vitality of both official languages and the communities that embody them is increased, enabling them to contribute fully to Canadian society, and to Canada's history, national identity, development and prosperity"?	Composite indicators of OLMCs: Socioeconomic Demographic Demo-linguistic Institutional completeness Financial independence Proximity of services in their language Perceptions of key stakeholders Public opinion regarding OL Expert opinion	Review of administrative documents: business plans, RPPs, DPRs, Roadmap partner evaluation products, OLB annual reports and files, community strategic plans and their reviews, and other documents Quantitative data from the OLB Review of the literature and census data Analysis of existing survey results Interviews with: PCH senior management responsible for the Roadmap Federal Roadmap partners Roadmap stakeholders Canadian public opinion surveys OLMC case studies Expert panels
To what extent, thanks to the Roadmap, can "Canadians live and thrive in both official languages and recognize the importance of French and English for Canada's national identity, development and prosperity"?	Degree of advancement of demo-linguistic indicators in Canada Public opinion regarding OL Perceptions of key stakeholders Expert opinion	Review of administrative documents: business plans, RPPs, DPRs, Roadmap partner evaluation products, OLB annual reports and files, community strategic plans and their reviews, and other documents Quantitative data from the OLB Review of the literature and census data Analysis of existing survey results OLMC case studies Public opinion surveys Interviews with: PCH senior management responsible for the Roadmap Federal Roadmap partners Roadmap stakeholders Expert panels
Did Roadmap measures cause any unexpected positive or negative impacts?	Impact of unexpected effects (positive and negative) Perceptions of key stakeholders Expert opinion	Review of administrative documents: Roadmap partner evaluation products, OLB annual reports and files, community strategic plans and their reviews, and other documents Interviews with: PCH senior management responsible for the Roadmap Federal Roadmap partners Roadmap stakeholders OLMC case studies Expert panels

Table 1.6: Demonstration of efficiency and economy

Evaluation questions	Indicators	Sources/methods
Do the initiatives under the Roadmap complement or overlap with existing programs?	Perception of the complementarity or overlapping with other initiatives	Interviews with PCH senior management responsible for the Roadmap
Are the most efficient actions being taken to achieve the expected outcomes?	Evidence of measures put in place to more effectively coordinate activities and manage Roadmap funds more economically Evidence that investments have had a leveraging effect Perceptions of key stakeholders	Review of administrative documents: business plans, RPPs, DPRs, Roadmap partner evaluation products, OLB annual reports and files, community strategic plans and their reviews, and other documents Interviews with: PCH senior management responsible for the Roadmap Federal Roadmap partners Roadmap stakeholders
Are there other ways of achieving the same results more effectively?	Other innovative models or approaches used by horizontal initiatives Comparisons with horizontal initiatives in Canada or elsewhere and acknowledged as successes Perceptions of key stakeholders Expert opinion	Review of administrative documents: Roadmap partner evaluation products, OLB files, and other documents Review of the literature on horizontal initiatives Interviews with: PCH senior management responsible for the Roadmap Federal Roadmap partners Roadmap stakeholders Expert panels OLMC case studies

Matrix of evaluation questions, Official Languages Coordination Program - part of the Horizontal Coordination of the Roadmap for Canada's Official Languages 2013-2018

Table 2.1: Continued need for the program

Evaluation questions and sub-questions	Indicators	Data collection methods	Information sources
To what extent is there still a need for the Coordination Program managed by the OLB? To what extent does the Coordination Program managed by the OLB meet the needs of the Roadmap partners, thereby ensuring that the Roadmap initiatives meet the needs of Canadians?	Changes observed since the last evaluation that might have influenced horizontal coordination needs Impact on official language accountability in the federal system Perception of Roadmap partners of the needs met by the coordination functions	Review of administrative documents	Treasury Board submissions, business plans, Speeches from the Throne, budget announcements Administrative records Policy statements and government responses Press releases RPPs, DPRs Other documents
		Review of the literature and analysis of survey data	Publications and articles
		Interviews with key stakeholders	Key stakeholders at PCH Roadmap partners Members of departmental committees Official languages champions and coordinators

Table 2.2: Alignment with government priorities

Evaluation questions and sub-questions	Indicators	Data collection methods	Information sources
To what extent does the Coordination Program still align with the priorities and policy directions of PCH and government as a whole?	Degree of alignment between the objectives of the Coordination Program and federal government priorities Degree of alignment between the objectives of the Coordination Program and PCH priorities Degree of alignment between the objectives of the Coordination Program and the objectives of the Roadmap	Review of administrative documents	Documents related to the coordination program, whole-of-government priorities and PCH Administrative records RPPs, DPRs Other documents
		Interviews with key stakeholders	Key stakeholders at PCH Roadmap partners Members of departmental committees Official languages champions and coordinators

Table 2.3: Alignment with federal roles and responsibilities

Evaluation questions and sub-questions	Indicators	Data collection methods	Information sources
Is horizontal coordination compatible with the role and responsibilities of the government, and more specifically PCH?	Degree of alignment between the services offered by the Coordination Program and the federal government's official language role and responsibilities Level of key stakeholder satisfaction with the services offered by the Coordination Program regarding the federal government's exercise of its role and responsibilities	Review of administrative documents	Documents related to coordination program, whole-of-government priorities and PCH Administrative records RPPs, DPRs Other documents
		Review of the literature and analysis of survey data	Publications and articles
		Interviews with key stakeholders	Key stakeholders at PCH Roadmap partners Members of departmental committees Official languages champions and coordinators

Table 2.4: Achievement of expected outcomes

Evaluation questions and sub-questions	Indicators	Data collection methods	Information sources
To what extent does the Coordination Program support the collection of sufficient, appropriate and quality information on the outcomes achieved by Roadmap partners, including efficiency and economy outcomes?	Frequency and quality of performance reports, departmental reports and reports on results provided by partners, including efficiency and economy results Level of partner satisfaction with the development, planning, implementation and data collection for performance measurement and integrated reporting Perceptions on the frequency, quality, accuracy and timeliness of the information collected to support reporting Implementation of the recommendations of the 2012 evaluation Effectiveness of the current information management system on OL performance	Review of administrative documents	RPPs, DPRs 2012 evaluation and documents on the implementation of recommendations Administrative records Reports on results of the Roadmap partners Other documents
		Interviews with key stakeholders	Key stakeholders at PCH Roadmap partners
To what extent does the Coordination Program support partner initiatives? To what extent does the Coordination Program support partners to improve the reporting process?	Degree of effectiveness of coordination mechanisms Frequency of requests for information from partners Type, frequency and level of collaboration between the OLB and the Roadmap partners Level of partner satisfaction with the usefulness of the guidance and tools provided by the OLB and the quality of the support provided Specifically, partners' satisfaction with the accountability mechanisms Implementation of the recommendations of the 2012 evaluation, e.g., improvements in accountability since the last evaluation	Review of administrative documents	RPPs, DPRs Records on applications received and processed 2012 evaluation and documents on the implementation of recommendations Roadmap partner evaluation products Other documents
		Interviews with key stakeholders	Key stakeholders at PCH Roadmap partners
To what extent does the Coordination Program contribute to ensure that the Minister of Canadian Heritage and stakeholders are adequately informed about the official languages file?	Level of satisfaction of decision makers with the availability, usefulness, quality and timeliness of information available to them Quality of strategic advice and guidance in official languages provided to the Minister of Canadian Heritage and Official Languages and stakeholders Evidence that probative data and rigorous analyses were used to support decision making Level of satisfaction of the Minister and stakeholders with the support from the OLB regarding policy statements Effectiveness of the internal and external communication strategy	Review of administrative documents	Treasury Board submissions, memoranda to Cabinet and other similar documents Departmental correspondence and/or memos RPPs, DPRs Policy statements and government responses Other documents
		Interviews with key stakeholders	Key stakeholders at PCH Roadmap partners Members of interdepartmental committees Official languages champions and coordinators
To what extent is the governance structure effective and does it contribute to better coordination of Roadmap initiatives?	Degree of effectiveness of governance mechanisms Frequency and percentage of requests for information from federal institutions Level of satisfaction of federal institutional representatives with the quality of the support provided (Treasury Board Submissions, reports on results of partners, etc.) Number and nature of the changes made by federal institutions to the mechanisms of implementation of their initiative(s) following the support received Type, frequency and level of collaboration between Roadmap partners Frequency of committee meetings Quality of the operational directives and management tools made available to Roadmap partners	Review of administrative documents	Operational directives and management tools supporting the Roadmap Minutes of committee meetings Administrative records RPPs, DPRs Other documents
		Interviews with key stakeholders	Key stakeholders at PCH Roadmap partners Members of interdepartmental committees Official languages champions and coordinators
What were the unexpected negative or positive impacts resulting from the implementation of the Horizontal Coordination Framework for the Roadmap 2013-2018?	Evidence of unexpected negative or positive impacts	Review of administrative documents Interviews with key stakeholders	Administrative documents and records Key stakeholders at PCH Roadmap partners Members of interdepartmental committees Official languages champions and coordinators

Table 2.5: Efficiency and economy

Evaluation questions and sub-questions	Indicators	Data collection methods	Information sources
To what extent are the available resources wisely used to produce the expected outcomes of the Coordination Program?	Existence of effective measures to achieve the expected outcomes (operating and resource use processes) Perceptions of key stakeholders with regard to the use of available resources	Review of administrative documents	Reports on results of the Roadmap partners Administrative records Program budgets and other financial documents, and documents on staff RPPs, DPRs Other documents
		Interviews with key stakeholders	Key stakeholders at PCH Roadmap partners
To what extent would it be possible to use the financial resources allocated to the Coordination Program differently to achieve the same outcomes? Are there more cost-effective solutions to ensure a horizontal coordination?	Adequate level and nature of the financial resources Possible alternatives according to Roadmap partners Existence of other mechanisms within or outside government that could to be used to obtain similar results Situations of duplication or overlapping with other programs or initiatives Possible improvements resulting from a change to the Roadmap governance structure	Review of administrative documents	Reports on results of the Roadmap partners Administrative records Program budgets and other financial documents, and documents on staff Other documents
		Review of the literature and analysis of survey data	Publications and articles
		Interviews with key stakeholders	Key stakeholders at PCH Roadmap partners Members of departmental committees
To what extent are the Roadmap partners able to demonstrate improvements in the efficiency and economy of their initiatives related to horizontal coordination efforts?	Evidence of measures put in place to more effectively coordinate the activities and manage Roadmap funds more economically (same indicator as in the Roadmap evaluation) Improvements resulting from a change to the performance measurement of the Roadmap	Review of administrative documents	Reports on results of the Roadmap partners DPRs Other documents
		Interviews with key stakeholders	Key stakeholders at PCH Roadmap partners

Annex B: Roadmap 2013-2018 resources

Table 1: PCH financial commitments for the Program for the Horizontal Coordination of Official Languages^{Footnote p} (Vote 1)

Year	Ongoing funds	New funds	Total
2013-14	$390,000	$1,700,000	$2,090,000
2014-15	$390,000	$1,700,000	$2,090,000
2015-16	$390,000	$1,700,000	$2,090,000
2016-17	$390,000	$1,700,000	$2,090,000
2017-18	$390,000	$1,700,000	$2,090,000
Total	$1,950,000	$8,500,000	$10,450,000

Source: According to financial information provided by PCH on December 5, 2016.

Table 2: Financial commitments of the Roadmap 2013-2018 (5 year period)

Table 2.1: Canadian Heritage

Organization/Initiative	Ongoing funds	Renewed/new funds	Total
Support for minority-language education	$265,020,000	$0	$265,020,000
Support for second-language learning	$175,020,000	$0	$175,020,000
Summer language bursaries	$36,600,000	$0	$36,600,000
Official language monitors	$18,600,000	$0	$18,600,000
Exchanges Canada	$11,250,000	$0	$11,250,000
Intergovernmental cooperation	$22,260,000	$0	$22,260,000
Support to OLMCs	$22,260,000	$0	$22,260,000
Community Cultural Action Fund	$0	$10,000,000	$10,000,000
National Translation Program for Book Publishing	$0	$4,000,000	$4,000,000
Music showcases for artists from OLMCs	$0	$5,750,000	$5,750,000
Subtotal	$551,010,000	$19,750,000	$570,760,000

Table 2.2: Canada Arts Council

Organization/Initiative	Ongoing funds	Renewed/new funds	Total
Market Access Strategy for OLMC Artists	$0	$2,750,000	$2,750,000
Subtotal	$0	$2,750,000	$2,750,000

Table 2.3: Justice Canada

Organization/Initiative	Ongoing funds	Renewed/new funds	Total
Contraventions Act Fund	$49,600,000	$0	$49,600,000
Training, networks and access to justice services	$21,200,000	$19,000,000	$40,200,000
Subtotal	$70,800,000	$19,000,000	$89,800,000

Table 2.4: Health Canada

Organization/Initiative	Ongoing funds	Renewed/new funds	Total
Training, networks and access to health services	$115,000,000	$59,300,000	$174,300,000
Subtotal	$115,000,000	$59,300,000	$174,300,000

Table 2.5: Employment and Social Development Canada

Organization/Initiative	Ongoing funds	Renewed/new funds	Total
Social Partnership Initiative in OLMCs	$4,000,000	$0	$4,000,000
OLMC Literacy and Essential Skills Initiative	$7,500,000	$0	$7,500,000
Enabling Fund for OLMCs	$1,730,000	$67,270,000	$69,000,000
Subtotal	$13,230,000	$67,270,000	$80,500,000

Table 2.6: Immigration, Refugees and Citizenship Canada

Organization/Initiative	Ongoing funds	Renewed/new funds	Total
Immigration to OLMCs	$6,850,000	$22,550,000	$29,400,000
Language training for economic immigrants	$120,000,000	$0	$120,000,000
Subtotal	$126,850,000	$22,550,000	$149,400,000

Table 2.7: Public Service and Procurement Canada

Organization/Initiative	Ongoing funds	Renewed/new funds	Total
Language Portal of Canada	$0	$16,000,000	$16,000,000
Subtotal	$0	$16,000,000	$16,000,000

Table 2.8: Innovation, Science and Economic Development Canada

Organization/Initiative	Ongoing funds	Renewed/new funds	Total
Industry Canada (coordination)	$0	$16,000,000	$16,000,000
FedNor — Economic Development Initiative	$0	$4,450,000	$4,450,000
Subtotal	$0	$6,050,000	$6,050,000

Table 2.9: Atlantic Canada Opportunities Canada

Organization/Initiative	Ongoing funds	Renewed/new funds	Total
Economic Development Initiative	$0	$6,200,000	$6,200,000
Subtotal	$0	$6,200,000	$6,200,000

Table 2.10: Canadian Northern Economic Development Agency

Organization/Initiative	Ongoing funds	Renewed/new funds	Total
Economic Development Initiative	$0	$400,000	$400,000
Subtotal	$0	$400,000	$400,000

Table 2.11: Federal Economic Development Agency of Southern Ontario

Organization/Initiative	Ongoing funds	Renewed/new funds	Total
Economic Development Initiative	$0	$4,450,000	$4,450,000
Subtotal	$0	$4,450,000	$4,450,000

Table 2.12: Canada Economic Development for Quebec Regions

Organization/Initiative	Ongoing funds	Renewed/new funds	Total
Economic Development Initiative	$0	$10, 200,000	$10, 200,000
Subtotal	$0	$10, 200,000	$10, 200,000

Table 2.13: Western Economic Diversification Canada

Organization/Initiative	Ongoing funds	Renewed/new funds	Total
Economic Development Initiative	$0	$3,200,000	$3,200,000
Subtotal	$0	$3,200,000	$3,200,000

Table 2.14: National Research Council of Canada

Organization/Initiative	Ongoing funds	Renewed/new funds	Total
Strengthening the language industry and technologies	$10,000,000	$0	$10,000,000
Subtotal	$10,000,000	$0	$10,000,000

Table 2.15: Orginasation financial commitments totals

Organization/Initiative	Ongoing funds	Renewed/new funds	Total
Canadian Heritage	$551,010,000	$19,750,000	$570,760,000
Canada Arts Council	$0	$2,750,000	$2,750,000
Justice Canada	$70,800,000	$19,000,000	$89,800,000
Health Canada	$115,000,000	$59,300,000	$174,300,000
Employment and Social Development Canada	$13,230,000	$67,270,000	$80,500,000
Immigration, Refugees and Citizenship Canada	$126,850,000	$22,550,000	$149,400,000
Public Services and Procurement Canada	$0	$16,000,000	$16,000,000
Innovation, Science and Economic Development Canada	$0	$6,050,000	$6,050,000
Atlantic Canada Opportunities Agency	$0	$6,200,000	$6,200,000
Canadian Northern Economic Development Agency	$0	$400,000	$400,000
Federal Economic Development Agency of Southern Ontario	$0	$4,450,000	$4,450,000
Canada Economic Development for Quebec Regions	$0	$10,200,000	$10,200,000
Western Economic Diversification Canada	$0	$3,200,000	$3,200,000
National Research Council of Canada	$10,000,000	$0	$10,000,000
TOTAL	$886,890,000	$237,120,000	$1,124,010,000

Source: Canadian Heritage. (2015). Horizontal Evaluation of the Roadmap for Canada's Official Languages 2013-2018: Education, Immigration, Communities. Framework. October 27, 2015.

Table 3: Breakdown of planned and actual spending of Roadmap 2013-2018 initiatives

Table 3.1: Canadian Heritage

Organization/Initiative	Planned spending 2013-2014	Actual spending 2013-2014*	Planned spending 2014-2015	Actual spending 2014-2015*	Planned spending 2015-2016	Actual spending 2015-2016	Planned spending 2016-2017	Planned spending 2017-2018
Support for minority-language education	$53,004,809	$53,004,809	$53,004,808	$53,004,808	$53,004,809	$53,004,808	$53,004,808	$53,004,808
Support for second-language learning	$35,004,809	$35,004,809	$35,004,808	$35,004,808	$35,004,809	$35,004,808	$35,004,809	$35,004,808
Summer language bursaries	$7,320,966	$7,320,966	$7,320,966	$7,320,966	$7,320,966	$7,320,966	$7,320,966	$7,320,966
Official language monitors	$3,720,966	$3,720,966	$3,720,965	$3,720,965	$3,720,965	$3,720,965	$3,720,965	$3,720,965
Exchanges Canada	$2,250,000	$2,250,000	$2,250,000	$2,250,000	$2,250,000	$2,250,000	$2,250,000	$2,250,000
Intergovernmental cooperation	$4,452,455	$4,452,455	$4,452,455	$4,452,455	$4,452,455	$4,452,455	$4,452,455	$4,452,455
Support to OLMCs	$4,452,455	$4,452,455	$4,452,455	$4,452,455	$4,452,455	$4,452,455	$4,452,455	$4,452,455
Community Cultural Action Fund	$0	$0	$2,750,000	$1,923,726	$2,750,000	$2,750,000	$2,500,000	$2,000,000
National Translation Program for Book Publishing	$800,000	$800,000	$800,000	$800,000	$800,000	$800,000	$800,000	$800,000
Music showcases for artists from OLMCs	$1,150,000	$1,150,000	$1,150,000	$1,150,000	$1,150,000	$1,150,000	$1,150,000	$1,150,000
Subtotal	$112,156,460	$112,156,460	$114,906,457	$114,080,183	$114,906,459	$114,906,459	$114,656,458	$114,156,457

Table 3.2: Canada Arts Council

Organization/Initiative	Planned spending 2013-2014	Actual spending 2013-2014*	Planned spending 2014-2015	Actual spending 2014-2015*	Planned spending 2015-2016	Actual spending 2015-2016	Planned spending 2016-2017	Planned spending 2017-2018
Market Access Strategy for OLMC Artists	$250,000	$248,820	$500,000	$501,060	$500,000	$498,745	$750,000	$750,000
Subtotal	$250,000	$248,820	$500,000	$501,060	$500,000	$498,745	$750,000	$750,000

Table 3.3: Justice Canada

Organization/Initiative	Planned spending 2013-2014	Actual spending 2013-2014*	Planned spending 2014-2015	Actual spending 2014-2015*	Planned spending 2015-2016	Actual spending 2015-2016	Planned spending 2016-2017	Planned spending 2017-2018
Contraventions Act FundFootnote q	$9,518,371	$5,969,676	$9,518,371	$4,341,386	$9,518,371	$3,888,754	$9,518,371	$9,518,371
Training, networks and access to justice services (Education component)Footnote r	$3,787,240	$2,883,092	$3,787,240	$3,674,789	$3,787,240	$3,765,313	$3,787,240	$3,787,240
Training, networks and access to justice services (Communities pillar)	$4,086,600	$2,589,362	$4,086,600	$3,924,728	$4,086,600	$3,641,984	$4,086,600	$4,086,600
Internal services	$569,115	(included in initiative spending)	$569,115	(included in initiative spending)	$569,115	$268,006	$569,115	$569,115
Subtotal	$17,961,326	$11,442,130	$17,961,326	$11,940,903	$17,961,326	$11,564,057	$17,961,326	$17,961,326

Table 3.4: Health Canada

Organization/Initiative	Planned spending 2013-2014	Actual spending 2013-2014*	Planned spending 2014-2015	Actual spending 2014-2015*	Planned spending 2015-2016	Actual spending 2015-2016	Planned spending 2016-2017	Planned spending 2017-2018
Training, networks and access to health services (Education component)Footnote s	$18,000,000	$18,929,302	$22,400,000	$21,256,122	$22,400,000	$21,733,333	$22,400,000	$21,300,000
Training, networks and access to health services (Communities component)Footnote t	$9,650,000	$6,582,540	$14,670,000	$15,248,952	$14,670,000	$15,336,667	$15,250,000	$13,560,000
Subtotal	$27,650,000	$25,511,842	$37,070,000	$36,505,074	$37,070,000	$37,070,000	$37,650,000	$34,860,000

Table 3.5: Employment and Social Development Canada

Organization/Initiative	Planned spending 2013-2014	Actual spending 2013-2014*	Planned spending 2014-2015	Actual spending 2014-2015*	Planned spending 2015-2016	Actual spending 2015-2016	Planned spending 2016-2017	Planned spending 2017-2018
Social Partnership Initiative in OLMCsFootnote u	$800,000	$0	$800,000	$0	$800,000	$0	$800,000	$800,000
OLMC Literacy and Essential Skills Initiative	$1,500,000	$1,500,000	$1,500,000	$1,277,166	$1,500,000	$1,052,434	$1,500,000	$1,500,000
Enabling Fund for OLMCs	$13,730,000	$13,730,000	$13,670,000	$13,670,000	$13,800,000	$13,203,946	$14,060,000	$13,740,000
Subtotal	$16,030,000	$15,230,000	$15,970,000	$14,947,166	$16,100,000	$14,256,380	$16,360,000	$16,040,000

Table 3.6: Immigration, Refugees and Citizenship Canada

Organization/Initiative	Planned spending 2013-2014	Actual spending 2013-2014*	Planned spending 2014-2015	Actual spending 2014-2015*	Planned spending 2015-2016	Actual spending 2015-2016	Planned spending 2016-2017	Planned spending 2017-2018
Immigration towards OLMCs	$5,668,098	$5,670,000	$5,719,370	$5,720,000	$6,037,726	$6,037,726	$6,061,638	$5,911,638
Language training for economic immigrants	$24,000,000	$24,000,000	$24,000,000	$24,000,000	$24,000,000	$24,000,000	$24,000,000	$24,000,000
Subtotal	$29,668,098	$29,670,000	$29,719,370	$29,720,000	$30,037,726	$30,037,726	$30,061,638	$29,911,638

Table 3.7: Public Services and Procurement Canada

Organization/Initiative	Planned spending 2013-2014	Actual spending 2013-2014*	Planned spending 2014-2015	Actual spending 2014-2015*	Planned spending 2015-2016	Actual spending 2015-2016	Planned spending 2016-2017	Planned spending 2017-2018
Language Portal of Canada	$3,100,000	$3,062,065	$3,200,000	$3,094,371	$3,200,000	$2,915,673	$3,250,000	$3,250,000
Subtotal	$3,100,000	$3,062,065	$3,200,000	$3,094,371	$3,200,000	$2,915,673	$3,250,000	$3,250,000

Table 3.8: Innovation, Science and Economic Development Canada^{Footnote v}

Organization/Initiative	Planned spending 2013-2014	Actual spending 2013-2014*	Planned spending 2014-2015	Actual spending 2014-2015*	Planned spending 2015-2016	Actual spending 2015-2016	Planned spending 2016-2017	Planned spending 2017-2018
Industry Canada (coordination)	$70,000	$62,534	$390,000	$364,062	$380,000	$380,000	$380,000	$380,000
FedNor — Economic Development Initiative	$165,000	$90,000	$1,115,000	$432,269	$1,065,000	$1,335,400	$1,015,000	$1,090,000
Subtotal	$235,000	$152,534	$1,505,000	$796,331	$1,445,000	$1,715,400	$1,395,000	1,470,000

Table 3.9: Atlantic Canada Opportunities Agency

Organization/Initiative	Planned spending 2013-2014	Actual spending 2013-2014*	Planned spending 2014-2015	Actual spending 2014-2015*	Planned spending 2015-2016	Actual spending 2015-2016	Planned spending 2016-2017	Planned spending 2017-2018
Economic Development Initiative	$330,000	$37,903	$1,470,000	$853,337	$1,470,000	$1,227,733	$1,470,000	$1,460,000
Subtotal	$330,000	$37,903	$1,470,000	$853,337	$1,470,000	$1,227,733	$1,470,000	$1,460,000

Table 3.10: Canadian Northern Economic Development Agency

Organization/Initiative	Planned spending 2013-2014	Actual spending 2013-2014*	Planned spending 2014-2015	Actual spending 2014-2015*	Planned spending 2015-2016	Actual spending 2015-2016	Planned spending 2016-2017	Planned spending 2017-2018
Economic Development Initiative	$80,000	$0	$80,000	$74,135	$80,000	$80,000	$80,000	$80,000
Subtotal	$80,000	$0	$80,000	$74,135	$80,000	$80,000	$80,000	$80,000

Table 3.11: Federal Economic Development Agency of Southern Ontario

Organization/Initiative	Planned spending 2013-2014	Actual spending 2013-2014*	Planned spending 2014-2015	Actual spending 2014-2015*	Planned spending 2015-2016	Actual spending 2015-2016	Planned spending 2016-2017	Planned spending 2017-2018
Economic Development Initiative	$120,000	$45,060	$1,082,500	$1,114,865	$1,082,500	$1,090,818	$1,082,500	$1,082,500
Subtotal	$120,000	$45,060	$1,082,500	$1,114,865	$1,082,500	$1,090,818	$1,082,500	$1,082,500

Table 3.12: Canada Economic Development for Quebec Regions

Organization/Initiative	Planned spending 2013-2014	Actual spending 2013-2014*	Planned spending 2014-2015	Actual spending 2014-2015*	Planned spending 2015-2016	Actual spending 2015-2016	Planned spending 2016-2017	Planned spending 2017-2018
Economic Development Initiative	$440,000	$1,409,282	$2,660,000	$1,720,489	$2,360,000	$1,942,788	$2,370,000	$2,370,000
Subtotal	$440,000	$1,409,282	$2,660,000	$1,720,489	$2,360,000	$1,942,788	$2,370,000	$2,370,000

Table 3.13: Western Economic Diversification Canada

Organization/Initiative	Planned spending 2013-2014	Actual spending 2013-2014*	Planned spending 2014-2015	Actual spending 2014-2015*	Planned spending 2015-2016	Actual spending 2015-2016	Planned spending 2016-2017	Planned spending 2017-2018
Economic Development Initiative	$240,000	$110,000	$740,000	$523,590	$740,000	$710,465	$740,000	$740,000
Subtotal	$240,000	$110,000	$740,000	$523,590	$740,000	$710,465	$740,000	$740,000

Table 3.14: National Research Council of Canada

Organization/Initiative	Planned spending 2013-2014	Actual spending 2013-2014*	Planned spending 2014-2015	Actual spending 2014-2015*	Planned spending 2015-2016	Actual spending 2015-2016	Planned spending 2016-2017	Planned spending 2017-2018
Strengthening the language industry and technologies	$2,000,000	$2,000,000	$2,000,000	$2,000,000	$2,000,000	$2,000,000	$2,000,000	$2,000,000
Subtotal	$2,000,000	$2,000,000	$2,000,000	$2,000,000	$2,000,000	$2,000,000	$2,000,000	$2,000,000

Table 3.15: National Research Council of Canada

Organization/Initiative	Planned spending 2013-2014	Actual spending 2013-2014*	Planned spending 2014-2015	Actual spending 2014-2015*	Planned spending 2015-2016	Actual spending 2015-2016	Planned spending 2016-2017	Planned spending 2017-2018
Canadian Heritage	$112,156,460	$112,156,460	$114,906,457	$114,080,183	$114,906,459	$114,906,459	$114,656,458	$114,156,457
Canada Arts Council	$250,000	$248,820	$500,000	$501,060	$500,000	$498,745	$750,000	$750,000
Justice Canada	$17,961,326	$11,442,130	$17,961,326	$11,940,903	$17,961,326	$11,564,057	$17,961,326	$17,961,326
Health Canada	$27,650,000	$25,511,842	$37,070,000	$36,505,074	$37,070,000	$37,070,000	$37,650,000	$34,860,000
Employment and Social Development Canada	$16,030,000	$15,230,000	$15,970,000	$14,947,166	$16,100,000	$14,256,380	$16,360,000	$16,040,000
Immigration, Refugees and Citizenship Canada	$29,668,098	$29,670,000	$29,719,370	$29,720,000	$30,037,726	$30,037,726	$30,061,638	$29,911,638
Public Services and Procurement Canada	$3,100,000	$3,062,065	$3,200,000	$3,094,371	$3,200,000	$2,915,673	$3,250,000	$3,250,000
Innovation, Science and Economic Development Canada	$235,000	$152,534	$1,505,000	$796,331	$1,445,000	$1,715,400	$1,395,000	1,470,000
Atlantic Canada Opportunities Agency	$330,000	$37,903	$1,470,000	$853,337	$1,470,000	$1,227,733	$1,470,000	$1,460,000
Canadian Northern Economic Development Agency	$80,000	$0	$80,000	$74,135	$80,000	$80,000	$80,000	$80,000
Federal Economic Development Agency of Southern Ontario	$120,000	$45,060	$1,082,500	$1,114,865	$1,082,500	$1,090,818	$1,082,500	$1,082,500
Canada Economic Development for Quebec Regions	$440,000	$1,409,282	$2,660,000	$1,720,489	$2,360,000	$1,942,788	$2,370,000	$2,370,000
Western Economic Diversification Canada	$240,000	$110,000	$740,000	$523,590	$740,000	$710,465	$740,000	$740,000
National Research Council of Canada	$2,000,000	$2,000,000	$2,000,000	$2,000,000	$2,000,000	$2,000,000	$2,000,000	$2,000,000
TOTAL	$210,260,884	$201,076,096	$228,864,653	$217,871,504	$228,953,011	$220,016,242	$229,826,922	$226,131,921

* Spending was taken from the 2013-2014 and 2014-2015 Annual Official Languages Reports and from 2015-2016 Departmental Results Reports, while planned spending was taken from the summary report on the results achieved and progress made by the Roadmap for Canada's Official Languages.

Source: Canadian Heritage. (2014 & 2015). Summary Report on Results and Progress of the Roadmap for Canada's Official Languages 2013-2018; Canadian Heritage. (2015 & 2016). 2013-2014; 2014-2015 Annual Reports on Official Languages and Canadian Heritage. (2016) Departmental Results Report 2015-2016.

Table 4: breakdown of actual spending by Roadmap 2013-2018 pillar (2013-2014 and 2014-2015 fiscal years)

Pillar	Funds allocated over five years (2013-2018)	Planned spending 2013-2014	Actual spending 2013-2014	Planned spending 2014-2015	Actual spending 2014-2015	Planned spending 2015-2016	Actual spending 2015-2016	Total planned spending 2013-2016	Total actual spending 2013-2016
Education	$658,007,735	$128,188,790	$128,176,009	$132,688,787	$131,326,829	$132,688,789	$131,715,866	$393,566,366	$391,218,704
Immigration	$149,398,470	$29,668,098	$29,670,000	$29,719,370	$29,720,000	$30,037,726	$30,037,726	$89,425,194	$89,427,726
Community	$316,631,180	$51,834,881	$43,230,087	$65,887,381	$56,824,675	$65,657,381	$58,246,713	$183,379,643	$158,049,406
TOTAL	$1,124,037,385	$209,691,769	$201,076,096	$228,295,538	$217871504	$228,383,896	$219,748,236	$666,371,203	$638,695,836

Source: Canadian Heritage. (2015 & 2016). Annex 1 – Annual Reports on Official Languages and Canadian Heritage. (2016). Departmental Results Report 2015-2016.

Annex C: extract of the public opinion surveys

Figure 2: survey on the perception of both official languages (2016)

Questions	Strongly agree (8-10)	Somewhat agree (6-7)	Neutral (5)	Somewhat disagree (3-4)	Strongly disagree (0-2)	Don't know
Anglophones/Francophones from other countries are welcome in your language community	78%	11%	6%	2%	>2%	>2%
Learning both official languages contributes to better understanding among Canadians	67%	15%	7%	4%	7%	>2%
In Canada, knowing English and French improves the chances of finding a job	66%	14%	9%	5%	6.1%	>2%
The Government of Canada should continue to invest in exchange programs as a way to encourage understanding between the country's Francophones and Anglophones	63%	15%	10%	4%	8%	>2%
You would be (or would have been) interested in participating in school-based language exchanges to interact with young people from the other official language communities	61%	15%	9%	5%	9%	>2%
The fact that there are two official languages in Canada is, for you, an important part of what it means to be a Canadian.	54%	16%	13%	6%	12%	>2%
All high school graduates should have a working knowledge of English and French	52%	17%	14%	8%	10%	>2%
The Government of Canada is effective in protecting both of Canada's official languages	51%	20%	14%	7%	7%	>2%
Linguistic duality in Canada is, for you, a source of cultural enrichment	48%	17%	12%	8%	13%	>2%
The Government of Canada's Policy on Official Languages supports national unity.	46%	21%	16%	7%	7%	3%
In Canada, relationships between Francophones and Anglophones are more positive today than they were 10 years ago	39%	23%	20%	7%	6%	4%
In your region, relationships between Francophones and Anglophones are more positive than they were 10 years ago.	37%	18%	25%	6%	8%	7%
The future of French in Canada is threatened	27%	16%	17%	14%	24%	>2%

Baseline: All respondents (n=1501)

A score of 10 means “Strongly agree”; aggregate scores from 8 to 10 are identified as “Strongly agree”.

Categories of responses not showing percentages represent 2% or less of the respondents.

Source: Canadian Heritage. (2016). Study on the Appreciation and Perception of Canada's Two Official Languages.

The table below shows the level of agreement on a scale of 0 to 10, divided into 13 categories of respondents and covers three years, including 2005, 2012 and 2016.

Here is the proportion in accordance with the following statement. "The fact that he has two official languages is an important part of what it means to be Canadian"

Figure 3: comparative analysis of the surveys on the perception of Canada's official languages

Repondants	2005 environics	2012 TNS	2016 Ad Hoc
Francophones	7.1	7.8	8
Anglophones	5.5	6.1	6.7
Women	6.2	6.7	7
Men	5.6	6.4	7
Atlantic	6.1	7.1	6.5
Quebec	7.2	7.6	7.9
Ontario	5.9	6.6	7
MB / SK	4.6	5.2	5.7
Alberta	4.6	5.7	6.2
British-Columbia	5.2	5.6	6.7
18-24 years	6.6	8.2	7.2
25-30 years	6.5	6.7	7
31 years +	5.8	6.5	7

Source: Canadian Heritage. (2016). Analyse longitudinale des tendances de l'opinion publique des langues officielles (French only)

Annex D: component of the Official Languages Act

Components of the Official Languages Act (the Act)

Parts	Domain	Who is responsible for implementation?
I	Proceedings of parliamentary	House of Commons, Senate and Library of Parliament
II	Legislative and other instruments	House of Commons, Senate, Library of Parliament and Justice Canada
III	Administration of justice	Justice Canada and each of the federal courts
IV	Communications with and services to the public	All federal institutions and Treasury Board charged with the overall development and coordination of the application principles and programs for all three parts
V	Language of work	All federal institutions and Treasury Board charged with the overall development and coordination of the application principles and programs for all three parts
VI	Participation of English-speaking and French-speaking Canadians	All federal institutions and Treasury Board charged with the overall development and coordination of the application principles and programs for all three parts
VII	Advancement of English and French Development of official-language minority communities	All federal institutions and Canadian Heritage responsible, in consultation with the other federal ministers, for fostering and encouraging a coordinated approach to the implementation by federal institutions of the government commitments under Part VII
VIII	Responsibilities and duties of Treasury Board (Parts IV, V and VI)	Treasury Board
IX	Commissioner of Official Languages	Parliament and the Commissioner of Official Languages
X	Court remedy	Federal Court of Canada and Justice Canada
XI	General provisions of the Act, including creation of the standing committee(s) on official languages (section 88)	Parliament
XII	Related amendments	Parliament
XIII	Consequential amendments	Parliament
XIV	Transitional provisions, repeal and coming into force	Parliament
**	Justice also advises the federal institutions and represents the government in any litigation.	Justice Canada

Source: Canadian Heritage. (2015). Review of Horizontal Governance of Official Languages within the federal public service – Discussion paper. p. 4-5.