Evaluation of the International Health Grants Program (2008–09 to 2012–13)
Prepared by
Evaluation Directorate
Health Canada and the Public Health Agency of Canada
July 2013
Table of contents
- Executive summary
- Management Response and Action Plan
- 1. Introduction
- 2. Evaluation methodology
- 3. Findings — relevance
- 4. Findings — performance
- 5. Evaluation conclusions and recommendations
- Appendix 1: Project component logic model
- Appendix 2: Framework for Cooperation: CANADA — Pan-American Health Organization
- Appendix 3: Canada — Pan American Health Organization Biennial Workplan Budget Funds
(2012-2013) - Appendix 4: Projects Funded through the International Health Grants Program
project component by Recipient and Year) 2008/09 – 2012/13 - Bibliography
- Endnotes
Executive summary
This report presents the results of an evaluation of the International Health Grants Program covering the period from 2008-2009 to 2012-2013. The evaluation was undertaken by the Public Health Agency of Canada and Health Canada’s Evaluation Directorate.
Evaluation purpose
The purpose of the evaluation is to provide senior management with evaluative information on the International Health Grants Program to support program planning and decision making and to inform the renewal of the Program, which is currently being finalized. The evaluation also meets the requirements of the following:
- Financial Administration Act and Treasury Board of Canada Secretariat’s Policy on Transfer Payments which require departments to conduct a review of their grant and contribution programs every five years
- 2009 Policy on Evaluation which requires comprehensive evaluation coverage of all direct program spending on a five-year cycle.
Background and context
During much of the period covered by the evaluation, the Program was managed by several different offices within Health Canada. In July 2012, the Program was moved to the Office of International Affairs for the Health PortfolioFootnote 1 in the Public Health Agency of Canada.
The Program is the Health Portfolio’s main mechanism for supporting international organizations and/or facilitating participation in international health initiatives whose mandates are consistent with the Health Portfolio’s policy objectives and priorities, or that complement Canada’s health research agenda.
The Program has two components:
- Pan American Health Organization ( PAHO) component
Health Canada pays the membership, on behalf of Canada, to PAHO through an assessed contributionFootnote 2. For 2012-2013, spending on the PAHO component was $11,869,159.Footnote 3The assessed contribution is based upon a formula established by the Member Governments of the Organization of American States, of which Canada is a member. The amount of the contribution is non-discretionary. The assessed contribution is used as core funding for the operation and core programs of the Organization. Voluntary funding is also provided by other Canadian federal departments and agencies, mainly by the Canadian International Development Agency. In 2011-2012, the assessed contribution to PAHO accounted for 83% of the Program budget. - Project component
In 2012-13, spending on the project component was $1,209,541. The project component includes the following:- commitments and obligations to other international organizations, including payment of the membership in the International Agency for Research on Cancer (IARC)Footnote 4, and a grant to the World Health Organization — Framework Convention on Tobacco Control
- grants to support Canada’s international commitments to the Organization for Economic Co-operation and Development Health Committee and the Global Health Research Initiative of the International Development Research Centre
- individual grants for national and international projects and initiatives that support current federal priorities and emerging global health issues
Total expenditure for the International Health Grants Program for the five year period (2008-2009 to 2012-2013) was $71,924,577. The assessed contribution to PAHO accounted for 86% of expenditures. The project component accounted for the remaining 14%, of which international commitments and obligations accounted for 10% of the total Program expenditures. Once these commitments and obligations were met, the remaining 4% was used to support individual projects and initiatives. Expenditures for individual projects varied from year to year and ranged anywhere from 0% to 7% of the total Program budget.
Evaluation scope and methodology
The evaluation examined the relevance and performance of the two components of the Program.
The evaluation did not take into consideration activities related to the World Health Organization — Framework Convention on Tobacco Control. The relevance and performance of the Framework were assessed in the Horizontal Evaluation of the Federal Tobacco Control Strategy. As the activities of the PAHO/World Health Organization collaborating centres are not funded through the Program, they were not assessed in this evaluation.
Multiple data collection methods were used in this evaluation, including:
- a literature review
- administrative file and document reviews
- key informant interviews with Public Health Agency staff and external stakeholders
- case studies of one collaborating centre and two projects funded through the Program.
Data from these sources were triangulated to arrive at the evaluation findings.
Evaluation findings and conclusions
Relevance
There is a need for the Health Portfolio to continue to engage with a range of international partners to be able to represent Canadian values and priorities when addressing global health issues. The Program is the main funding mechanism enabling the Health Portfolio to influence global health decisions and to benefit from the work of international organizations.
Globalization has a direct impact on the health and security of Canadians and Canada needs to be engaged internationally to benefit from collective action on global health issues. Memberships in PAHO and IARC allow Canada to influence the policies and strategies of these two organizations. Membership in PAHO enables Canada to benefit from price negotiation and procurement of essential medicines such as vaccines, public health capacity building in Latin America and country-level information on disease surveillance, civil strife and natural disasters, which affect Canadian interests. The implications for Canada of withdrawing its membership from PAHO would be significant and would include reduced ability to achieve the objectives of Canada’s Americas Strategy to influence and contribute to the promotion of Canada’s principles of democratic governance, security and economic prosperity in the hemisphere. It would also include loss of linkages with regional surveillance networks and emergency response centres leading to a reduced capacity to respond to emerging health threats.
The project component provided the ability to meet international commitments and to fund initiatives that supported efforts to address current and emerging health issues of global priority. The project component provided the flexibility to fund health related initiatives with international or Canadian partners involved in the implementation of health projects.
The objectives of the Program were aligned with Government of Canada and Health Portfolio priorities. Canada’s membership in PAHO aligned with Canada’s Americas Strategy, which recognizes the importance of the Americas for Canada and articulates a long-term commitment to engage with the Americas. The Public Health Agency’s Global Health Framework identified PAHO as a core partner. The Program also aligned with Health Canada’s International Health Strategy of the Health Portfolio (2008-2011) which highlighted the continuing process of globalization and its direct impact on the health and security of Canadians.
Evidence indicated that the international activities of Health Portfolio were aligned with federal roles and responsibilities. It was appropriate for Health Canada to be the Government of Canada lead for engagement with PAHO and IARC.
Performance
The PAHO component of the Program made progress toward achieving its objectives. As the lead for Canada’s engagement with PAHO, Health Canada demonstrated that it was able to influence the Organization’s agenda and priorities to promote effective management, exercise oversight, as well as to leverage the strengths and expertise of this Organization to further benefit Canadians.
Evidence demonstrates that efforts were made to have a whole-of-government approach for engagement with PAHO through interdepartmental consultations. However, improvement is still required, specifically with respect to articulating an approach for the Government of Canada’s engagement with the Organization. While Health Canada has identified its own expectations and strategies for working with PAHO through a Framework for Cooperation, signed in May 2010, there is a need to develop, in consultation with all government departments and agencies involved, a whole-of-government approach that articulates Canada’s expectations of engagement with this Organization. Such an approach should take into account the respective mandates and accountabilities of these departments and agencies and enable the reporting of achievements as a result of all Government of Canada activities and funding, as well as assist PAHO in its interface with Canada. As the lead for the Government of Canada, the Office of International Affairs for the Health Portfolio has a role in initiating this process in consultation with other departments and agencies.
In terms of the performance of the project component, small investments in short periods of time led to the development of networks with international and national partners and the leveraging of their expertise. Forty six projects with a total value of $9,636,195 were funded during the evaluation period.
The Program had challenges with respect to optimizing inputs to achieve outcomes. The instability and timing of funding was a key impediment to achieving more strategic results and to ensuring regular funding of smaller projects that connect Health Canada to the research and initiatives of Canadian and international partners. This component has been managed with rigour using fewer resources than anticipated by program authorities.
Recommendations
Whole-of-government approach
While the Public Health Agency (previously Health Canada) funded the membership in PAHO on behalf of the Government of Canada, several federal departments also engage with PAHO. The Framework for Cooperation signed between PAHO and the Department of Health Canada in May 2010 does not replace the need for a whole-of-government approach to ensure maximum benefits for Canada’s investment. Such an approach would offer an opportunity, in the context of the merger of the international affairs function for the Health Portfolio, to review Canada’s dialogue with PAHO and focus on issues of priority to Canada. This whole-of-government approach would have to take into account the roles, mandates and accountabilities of the departments involved.
This is in line with a document that the Office of International Affairs for the Health Portfolio is developing on priorities for Canada’s strategic engagement with PAHO for the next three years, which will be submitted to senior management for approval, both at the Public Health Agenda of Canada and Health Canada in June 2013.
Recommendation 1.
To maximize Canada’s return on investment, the Office of International Affairs for the Health Portfolio should, in consultation with its federal partners, develop a whole-of-government approach to better clarify Canada’s expectations with respect to its engagement with PAHO, as well as define the roles and accountabilities of all involved, including the role of the Office of International Affairs for the Health Portfolio, the Public Health Agency of Canada as the current lead organization and Health Canada as the key Health Portfolio counterpart.
Performance measurement
The absence of performance measurement data was the key barrier to reporting on the successes of the Program. The evaluation found evidence in several documents, pointing back to 2008, of the intention to implement a performance measurement framework for the Program and all international activities. Although the 2008 official program authorities provided a detailed measurement framework for the project component, it was never implemented. A similar framework did not accompany the program authorities for PAHO component. The Government of Canada Policy on Transfer Payments is specific about reporting to Canadians on grants and assessed contributions. Improved monitoring and measurement of the Program’s achievements would provide the basis for all performance reporting.
Recommendation 2.
To be able to report on the results of its international activities, the Office of International Affairs for the Health Portfolio should develop and implement a performance measurement framework for the IHGP that encompasses both the PAHO and project components of the Program.
Project funding
The evidence indicated that there was considerable value to the project component of the Program as it enabled the Health Portfolio to meet international commitments and provided the Health Portfolio with the flexibility to influence and participate in current and emerging federal global health priorities. The project component included grants to meet commitments and obligations to international organizations, including IARC and the World Health Organization — Framework Convention on Tobacco Control. Funding for individual projects occurs yearly but only if there are residual funds from the total budget allocated to the Program, once memberships are paid and commitments and obligations are met. Over the five year period of the evaluation, an average of 5 per cent of the Program budget was allocated to individual projects. However, the instability and tardiness of the funding for the projects, as also identified in previous reviews and evaluations, is a significant impediment to the achievement of project objectives as the availability of funding late in the year limits the ability of the Program to be more strategic in selecting projects.
Recommendation 3.
The Office of International Affairs for the Health Portfolio, in consultation with the Office of the Chief Financial Officer (OCFO), should consider options to address the instability of the funding of projects of the Program to enable more strategic selection of projects.
Management of the projects
While under the responsibility of Health Canada, efforts were invested in developing tools to strengthen the selection and management of the individual projects. For example, a rigorous and inclusive selection process was introduced. While this process may have contributed to the implementation of projects that were relevant and produced outputs, it may have been more onerous than necessary given the size of investment and the low risk of the grants.
Recommendation 4.
For greater efficiency, the Office of International Affairs for the Health Portfolio should review the management and administration of the projects, including its proposal review and selection process, taking into account the size of the investment and the level of risk of the grants.
Page details
- Date modified: