Evaluation of the Labour Funding Program

Preliminary findings, International Trade and Labour stream

Seventeen projects have been funded between April 2012 and March 2016. In terms of relevance, the grants are generally well aligned with the priorities and expected outcomes of the program and the overall department, as well as international labour standards, rights and principles. The grants also continue to be relevant to the needs of partner countries. Sixteen of the seventeen grants funded capacity-building activities to help partner countries with specific labour-related issues, and all were aligned with at least one international labour right or principle. One project funded research on international labour agreements, which is relevant to the stream’s outcomes.

Regarding performance, all eleven projects completed by December 2016 achieved most or all expected outputs, and outcomes. In the medium- to long-run, all completed projects are reported to have established conditions that could lead to improvements in partner countries’ labour situations, but as is common with development projects, it is difficult to attribute broad impacts directly back to such small grants. The scope and expected outcomes of the projects sometimes evolve due to changing context or difficulties engaging stakeholders in partner countries. Tracking the evolution of expected and achieved outcomes by projects is something of a challenge because the program lacks a database to systematically gather project updates and results in one place. Outcome sustainability is a challenge for technical assistance projects in general; however, six of eleven completed projects have reported or demonstrated a strong likelihood for sustainable results, and the other five reported producing at least one project component that is likely to be sustainable.

Findings, Occupational Health and Safety stream

Two projects were provided with funding through this stream between April 2012 and March 2016. Evidence gathered from program documents and key informant interviews ‎demonstrates the grants have delivered or are delivering on expected results. As of the 2014 to 2015 fiscal year, the first project had contributed to building and sharing knowledge and harmonizing occupational health and safety standards by producing twelve standards through a development process involving multiple stakeholders, including the provinces and territories. Six of these labour standards were also published on a website administered by the recipient. The remaining standards are expected to be published by 2019. The other grant provided funds for a research project.The project built knowledge on the different factors that should be considered to increase health and safety for all new workers and young workers in particular. This information was shared with representatives from federally-regulated employee and employer groups in high risk industries (for example transport, communications, and shipping).

1.1 Labour Funding Program background

The Labour Funding Program (LFP) is a grants and contributions program created to centralize the management of all the Labour Program’s grants and contributions under a single set of terms and conditions. Originally, the program had three funding streams: the Labour-Management Partnerships, International Trade and Labour (ITL), and Occupational Health and Safety and Fire Prevention^{Footnote 4}. However, the Labour-Management Partnerships stream and the Fire Prevention Grants were defunded soon after the creation of the program. In 2014 to 2015, the Employment Equity Program introduced the Workplace Opportunities: Removing Barriers to Equity stream^{Footnote 5}. The new stream will be evaluated with its parent program, leaving two streams to be covered in this evaluation. Expected outcomes differ by stream, but both are intended to contribute to safe, fair, and productive workplaces and cooperative workplace relations in Canada and internationally. Annex 1 presents the program’s logic model.

2.1 International Trade and Labour stream

Are International Trade and Labour projects aligned with government priorities related to the maintenance of international labour rights and principles?

Individual project approvals and planning documents for all projects demonstrate alignment between the projects’ expected results, expected outcomes, and ESDC’s mandate. The International Trade and Labour stream supports the Labour Program’s priority to provide technical assistance to countries with which Canada has negotiated or is negotiating a labour agreement, as well as broader government priorities for international relations and trade, including the progressive trade agenda.

In order of priority, projects are funded, first, in partner countries; second, in countries with active or concluded negotiations; and third, in countries of national interest or importance. Sixteen projects have contributed to Canada meeting its commitments under its agreements with partner countries, as demonstrated in Annex 2. Eight projects supported technical assistance projects in countries with existing agreements; four projects were undertaken in countries with which negotiations were active or had concluded^{Footnote 7}; and one project had activities in two countries, one of which already had an agreement in place with Canada (the other had recently concluded negotiations). The other three projects supported cooperative activities with China. The only project that did not fulfill a specific commitment to a partner country was a research project on the effectiveness of labour agreements conducted by the International Labour Organization.

Each of the sixteen country-specific projects pursued outcomes that aligned with the International Labour Organization's international labour standards and with the Decent Work agenda. Of those, fourteen projects included outcomes that aligned with the organization’s 1998 Declaration on Fundamental Rights and Principles at Work and three were also part of the organization’s Better Work program. This evidence indicates that projects are generally well aligned with government priorities related to the maintenance of internationally-recognized labour rights and principles.

Did the projects funded through the International Trade and Labour stream deliver the expected project results?

Out of seventeen projects, six have made strong progress toward achieving not just some, but all outcomes. ESDC’s 2016 to 2017 Report on Plans and Priorities defines International Trade and Labour project success as 85% of project objectives met in full or in part. All eleven projects completed as of December 2016 delivered partial or full results and, overall, meet this definition of success. Similarly, four of six ongoing projects are reported to have already achieved or are on track to implement activities, outputs and outcomes. Reports for the other two projects note that the projects have been facing delays. Based on reports by implementation agencies, major challenges to project implementation such as incomplete activities, outputs, outcomes or objectives often result from a lack of engagement with key stakeholders and/or a changing political context.

To what extent has the International Trade and Labour stream supported Canada's partners in addressing labour-related issues?

Projects usually support capacity building activities to address labour issues and/or improvements in labour legislation and policies, so that partners can fulfill their commitments under their agreements with Canada^{Footnote 8}. In general, these projects respond to partner countries’ labour-related needs. The labour conditions and technical assistance needs of partner countries are always articulated in project approval documents and funding arrangements based on research, discussions, and, when possible, meetings with partner countries. However, the process to assess partners’ needs is not always clearly articulated in the project files. In 2015, Bilateral and Regional Labour Affairs undertook a pilot project to produce compliance reports for five partner countries. These compliance reports highlight labour-related areas of need for technical assistance in partner countries, which can be used as formal needs assessments for technical assistance through the Labour Funding Program.

Direct consultation exchanges were documented between Canada and partner countries for ten of seventeen projects. Some program and project files also reference more informal exchanges about technical assistance needs during bilateral meetings, missions to partner countries, and meetings of multilateral labour forums such as the International Labour Organization^{Footnote 9}.

Approval documents also sometimes note linkages to previous projects, describing how the new project aligns with and/or leverages completed and ongoing labour technical assistance. The extent of coordination between donors is sometimes articulated in project files or implicit in the project design (for example, projects funded through large technical assistance frameworks such as the Better Work Vietnam program), but this type of coordination is not clearly documented for every project. Coordination with other donors has increased in recent years through such means as visits to fellow donor countries for the purpose of coordination and meetings with representatives from other donor countries during missions to beneficiary countries. Project implementation agencies also undertake some of the coordination work between beneficiary countries and donors and between various donors.

Except for one research project, the projects were designed to support some form of capacity building in partner countries. This encompassed activities such as training of inspectors, knowledge sharing, and development of inspections tools. The activities vary according to the stakeholders groups involved (in other words employers, employees, and government officials) and the specific labour standards to be addressed (occupational health and safety, child labour, protection of migrant workers, etc.). Progress reports and final reports for the eleven projects completed by December 2016 indicate that ten of the projects had made significant progress toward completing capacity-building activities and outputs. The remaining project, which covered two countries, completed capacity-building activities in one beneficiary country, but had less success in the other. Most activities were undertaken in the second country, but changing political priorities in the partner country posed a challenge to full implementation of all activities.

In addition to capacity building, four of the eleven completed projects also incorporated activities to help partner countries maintain or improve their labour legislation and policies. Progress was made on legislative and/or policy changes in three of those projects. The fourth responded to a specific request by the partner country, but received insufficient political and bureaucratic support from the partner during implementation to achieve any legislative or policy changes.

Time-only extensions had been granted to eight of seventeen projects as of January 2017. These extensions have helped projects complete their intended activities (or add activities) when there were delays in obtaining approvals in partner countries, key domestic stakeholders were not engaged (for instance, due to changing political conditions), or activities proved more difficult to complete than expected. Although documents and files suggest that completed projects have had a positive effect on specific labour issues in partner countries, the sustainability of project results is not always clear. Measurability of long-term impacts and sustainability of results are common issues with development funding and technical assistance projects for a number of reasons, including changing political situations and ministerial priorities in partner countries. Approval documents for ten projects explicitly discussed potential sustainability of project activities and outcomes. There is no evidence of data collection to measure medium- to long-run project effects past the final progress report provided by the implementation agency, which is usually provided six months post-project.

For evaluation purposes, project sustainability was considered based on project progress and final reports. Of eleven projects completed by December 2016, six had reported strong likelihood of sustainability in most project components. The other five were reported to have set conditions for sustainability in at least component, but noted potential areas of concern regarding the sustainability of other aspects of the project.

To what extent has the International Trade and Labour stream allowed Canada to fulfill its obligations under international labour agreements?

The stream has allowed Canada to fulfill its commitments to partner countries by funding technical assistance projects in eight partner countries since April 2012. The projects are an avenue for Canada to support the mission and goals of international organizations and some grants have funded specific projects implemented by the International Labour Organization. Still, none of the projects directly support the general operations of international labour-related forums or organizations.

Are there more efficient ways of achieving the expected outcomes?

This particular stream is relatively small compared with other labour-related technical assistance programs (for example, the United Sates Department of Labour’s technical assistance fund; the International Program for Professional Labour Administration, which was a temporary initiative funded jointly by Global Affairs Canada and the Labour Program). The equivalent of less than one Labour Program FTE is used per year on the projects. Direct program spending is low compared to the grant amounts disbursed (please see Annex 3 for financial information). The stream has used its maximum grant amount every year since it began. To help ensure efficiency and responsible use of funds, implementation agencies selected to deliver the projects have to meet certain criteria for labour expertise, financial administration, reporting on results, and field presence. Any funds unused at the end of a project implemented by the International Labour Organization in particular can be held by the implementation agency for use in future projects funded by Canada, with Canada’s agreement. Of twelve projects implemented by this organization, two were partially financed by unused funds from previous grants.

Lessons learned from completed projects are identified in briefing notes on an ad hoc basis and in project close out reports. Most progress and final reports include lessons learned for the same or future projects. At the stream level, program officials have adopted a best practice to solicit projects rather than issuing regular calls for proposals, in an effort to ensure projects are aligned with the stream’s objectives. One grant was provided to the International Labour Organization’s research division for a project that investigated the design, implementation, and impacts of international labour agreements in general. This project contributed to improving international understanding of labour agreements and technical assistance, yielding multiple papers and conferences, as well as a final research report.

In regards to program monitoring and performance measurement to improve efficiency, useful data is available in the project reports, site visit and mission reports, and informal status updates regarding project alignment with expected International Trade and Labour stream outcomes and individual project achievements. Sometimes, the outputs and outcomes of the projects do not always match across different reporting documents as projects evolve over time. The lack of a formal performance information database limits the ability to extract, organize, and measure performance data from project reports against the International Trade and Labour performance measurement framework and to report on program-level results. Program financial data indicates a low level of resources (less than one FTE) to administer the program.

Officials do, however, monitor project progress on an ongoing basis, either through formal or informal mechanisms. Evidence of formal and/or informal monitoring by program officials was available for all projects. These monitoring activities have allowed the program to provide guidance and, if necessary, correct course throughout the life of the projects. For some projects, this allowed the implementing agency to focus on delivering some key intended results, or to shift to delivering alternative results that still fit within the project scope and objectives.

Regarding formal monitoring, each funding arrangement requires the project implementation agency to submit project performance measurement and implementation plans shortly after the project begins, followed by periodic progress reports on set dates or intervals and final reports within six months of the project’s completion. For sixteen of the projects, all expected progress reports have been provided. The remaining project has not yet provided its expected formal progress reports; however, briefing notes have been provided with short updates about delays. Final reports expected to have been produced by the implementation agencies upon project completion are available for all projects completed by March 2016. Regarding more informal monitoring, the program collects information through email and telephone updates, as well as through occasional site visits when program officials participate in missions to partner countries.

In fall 2016, program officials began preparing analyses of project progress reports sent by implementation agencies. These analyses align better with the project reports than the standard grants and contributions monitoring templates. They summarize the project reports from a critical perspective against the project’s own performance indicators and expected outcomes, and provide space for the analyst to reflect on follow up questions and issues for further monitoring.

Conclusion

This report presents initial findings on the relevance and performance of the International Trade and Labour stream, based on a review of program documents and project files. Additional findings from key informant interviews will be incorporated in a final report, to be presented with the findings of the Bilateral and Regional Labour Affairs evaluation. The stream continues to be relevant to government and departmental priorities and contributes to the implementation of Canada’s international labour agreements. The projects respond to demonstrated needs of partner countries and are generally having a positive effect on the trade-related labour issues they are intended to address. All projects contribute to the stream’s expected outcomes, and all completed projects have achieved partial-to-full results on project-specific activities, outputs and outcomes. The stream does, however, lack a database of project performance information. Measuring long-term impacts and the sustainability of project results are challenges faced by all development and technical assistance programs, including this one. In the absence of long-term performance data from the projects, a database that tracks the expected and completed results of each project could help paint the picture of the stream’s contributions over time and the progress made across projects within each country.

2.2 Occupational Health and Safety Stream (OHS-G)

Did the projects funded through the Occupational Health and Safety-Grants deliver the expected projects results?

Although the funding agreement with the Canadian Standards Association continues until March 2018, the interview and document review suggests that the projects are on track to achieve the expected activities and short-term outcomes. As per their funding agreement, the Canadian Standards Association developed and maintained Occupational Health and Safety standards covering such topics as ergonomics, management systems, elevating work platforms, and protective headwear/footwear. In addition, the Canadian Standards Association supported the accessibility of standards through the View Access website which allows users to view Canadian Standards Association Occupational Health and Safety standards online at no cost.

As of February 20, 2015, ten of the twelve labour standards projects identified in the 2012 to 2014 Work Plan had been completed and made available for public review. Five of the completed projects had been published on the View Access website and it was expected that the remaining labour standards would be published by summer 2016. Moreover, of the sixteen labour standards projects identified in the 2014 to 2016 Work Plan, two had been completed and one had been published. The remaining projects are expected to be published by 2019.

The goal of the “Young Workers’ Injury Prevention Interventions in Canada” project was to develop appropriate metrics for evaluating youth injury prevention initiatives, and to present an evidence-based strategy to leverage these metrics for optimal use and effectiveness. The project produced an analysis of the factors that may impact youth injuries in workplaces (in other words environment, training and experience), an environmental scan of youth injury prevention strategies, and a literature review of metrics that could be used to measure youth injury prevention strategies.

Has the Occupation Health and Safety-Grants stream contributed to the building and sharing of knowledge on the provisions of employees' accidents and illnesses?

The program has contributed to the building and sharing of knowledge on the provisions of employee’s accidents and illness through the Canadian Standards Association standards development process. According to key informants, the process allows professional workers from various backgrounds to be part of the advisory committee and to participate in the publishing standards process. There is a public review period before it is published that allows stakeholders to provide input as the standards are developed. The funding was also used to support the View Access Project, which allows users such as workers and employers to view occupational health and safety standards online at no cost. There is also a public forum called Community, which is a platform for standard-based discussion that includes a specific section on Occupational Health and Safety.

The young workers project provided research on what factors could lead to accident and illness, and it found that those same factors can contribute to safety, security, and wellness in the work place. This research was also found to apply to new immigrants in Canada and all new workers. It was mentioned in the key informant interviews that the young workers’ prevention project was shared with members of an Occupational Health and Safety advisory committee with representatives from federally-regulated employee and employer groups in high risk industries (for example transport, communications, and shipping). The committee was asked to share this information widely with their groups so they could reach as many employees and employers as possible.

To what extent has the Occupational Health and Safety-Grants stream contributed to the harmonization of occupational health and safety standards across jurisdictions within Canada?

The Canadian Standards Association project has contributed to the harmonization of occupational health and safety standards across jurisdictions within Canada through the use of a committee with members from the national, provincial, and territorial governments and regulators, as well as groups from various sectors that creates and reviews the standards. The committee is structured and governed in order to harmonize and ensure that the standards they develop are relevant and applicable to the various occupational health and safety jurisdictions across the country.

Conclusion

Although the evaluation could not assess the medium and long term impacts of the projects, evidence gathered from program documents as well as key informant interviews shows that the grants have delivered, or are in the process of delivering, their expected results.

By the end of fiscal year 2014 to 2015, the Canadian Standards Association project had contributed to the building and sharing of knowledge and the harmonization of occupational health and safety standards by completing twelve standards through a development process involving multiple stakeholders, including the provinces and territories. Six of these labour standards were also published on the association’s View Access website. It is expected that the remaining labour standards will be published by 2019.

The grant for young workers has built knowledge on the different factors that should be considered to increase health and safety not only for young workers but for all new workers. This information was shared with representatives from federally-regulated employee and employer groups in high risk industries (for example transport, communications, and shipping).