Appendix 3: Evaluation of the Federal Initiative to Address HIV/AIDS in Canada 2008–09 to 2012–13 – Summary of findings

Appendix 3 - Summary of Findings

Rating of Findings

Ratings have been provided to indicate the degree to which each evaluation issue and question have been addressed. The rating symbols and their significance are indicated in the chart below. A summary of ratings is presented in Table 4.

**Table 4: Rating Symbols and Significance**
Rating Symbol	Significance
Achieved	The intended outcomes or goals have been achieved or met.
Progress Made; Further Work Warranted	Considerable progress has been made to meet the intended outcomes or goals, but attention is still needed.
Little Progress; Priority for Attention	Little progress has been made to meet the intended outcomes or goals and attention is needed on a priority basis.
N/A	A rating is not applicable.

Issue 1: Continued Need for the Program	Indicators	Overall Rating
SUMMARY: The evaluation found that despite significant investments over the past decade, HIV/AIDS remains a persistent public health issue for Canada that disproportionately affects vulnerable populations. Additional investments are still needed in many areas of prevention, diagnosis, care, treatment, support and research.
1.1 What is the magnitude of HIV in Canada and globally?	Description of current problem Program activities and reach connected to current needs Views of stakeholders on program connection to needs	Progress Made; Further Work Warranted
1.1.2 In what areas has the HIV and AIDS environment changed most since 2008?	Evolution of problem from 2008 until 2012/emerging trends Views of stakeholders on evolution of problem and areas for additional investment.	Progress Made; Further Work Warranted

Issue 2: Alignment with Government Priorities	Indicators	Overall Rating
SUMMARY: The evaluation found that the Federal Initiative continues to align with the federal government's overall priorities, and the priorities within each of the partner departments. Over the past five to seven years, there have been shifts in government direction relating to some aspects of the Federal Initiative. While the Federal Initiative's goals and objectives continue to align with government priorities, further shifts in government directions should be monitored to assess the continued relevance of current program authorities. The overall rating is currently assessed as achieved.
2.1 Do FI partners' current roles align with federal government and FI partners' priorities?	Identification of federal government priorities Identification FI partner priorities Identification of gaps between priorities & current role	Achieved

Issue 3: Alignment with Federal Roles and Responsibilities	Indicators	Overall Rating
SUMMARY: The evaluation found that the Federal Initiative aligns well with the mandate of each of the federal government partners, but does not provide authority for primary care activities within Health Canada and CSC's mandate. While additional partners are not needed within the Federal Initiative program authority, collaboration with other federal government departments is perceived to be necessary and could be enhanced. While roles and responsibilities are well defined, there is inconsistent understanding of this delineation by stakeholders. The federal government is filling an important and needed national role that is in line with international trends and is supporting stakeholder directions in Canada. Therefore, the overall rating has been assessed as achieved for 3.1 and 3.3, and the overall rating for 3.2 has been assessed as progress made, further work warranted.
3.1 Is the Federal Initiative aligned with the roles and mandates of Federal Initiative partners? Are current Federal Initiative partners the right partners? Do we need others?	Degree of alignment between the mandates, priorities and objectives of the Public Health Agency of Canada, Health Canada, CIHR and CSC and their roles in the Federal Initiative Differences in role between federal, provincial and territorial jurisdictions Identification of gaps in current delivery of HIV/AIDS activities	Achieved
3.2 To what extent are the roles and responsibilities of each partner defined and understood in the Federal Initiative?	Perception of roles and responsibilities in the Federal Initiative Identification of gaps and alignment in partner understanding of role	Progress Made; Further Work Warranted
3.3 How is the Federal Initiative's role different from or similar to the roles of provinces and territories, and non-governmental organizations? Does the Federal Initiative's role complement or duplicate/conflict with roles of other stakeholders? Is there a common understanding of the initiative's roles among stakeholders?	Description of Federal Initiative partners, PTs and NGO roles Differences and areas of alignment between partners, PTs and NGOs Identification of gaps and/or duplication Perception of gaps and/or duplication	Achieved

Issue 4: Achievement of Expected Outcomes (Effectiveness)	Indicators	Overall Rating
SUMMARY: The evaluation found that there is evidence that Federal Initiative support has led to the creation of new and valuable knowledge which has helped us to better understand and combat HIV/AIDS in Canada and abroad. At the same time, the support of the Federal Initiative has helped Canadian researchers to achieve a standing of international excellence. There is also evidence that individual and organizational capacity to address HIV/AIDS has been enhanced through Federal Initiative activities, especially in the context of technical support and community work. Furthermore, there are lots of individual examples of capacity building activities and there is evidence that an increase in capacity is occurring, especially in the research field. Many collaborative activities have occurred, ensuring participation by targeted stakeholders. In addition, there are some examples of successful local or community-based activities to reduce stigma and discrimination, and there are also many examples of improved access to more effective prevention, diagnosis, care, treatment and support as a result of Federal Initiative activities. Finally, there is evidence that federal coherence in the approach to HIV/AIDS is strong despite a lack of annual work plans and priority-setting exercises. However, as the Federal Initiative evolves, improvements to activities should continue. Primarily, given the extensive knowledge that has been developed on the disease and ways to address it, the translation, exchange and application of this knowledge should be supported and enhanced. Additional opportunities persist to address the barriers outlined in the original program authorities (e.g. poverty and lack of housing). Finally, partners in the Federal Initiative should further develop tracking mechanisms to monitor activities and make adjustments based on learnings. As the Federal Initiative is evolving, so should the logic model that explains the program, along with any related performance measurement strategy. Therefore, the overall rating has been assessed as progress made, futher work warranted.
4.1 Is the management accountability and governance for the FI in place and functioning adequately to achieve expected results? What progress, if any, has been made to address the recommendations in the 2008 evaluation?	Demonstration of enhanced coherence and capacity of FI management and governance structure Perception of management and governance ability to function adequately and achieve desired results	Progress Made; Further Work Warranted
4.2 To what extent has progress been made toward the achievement of first level expected outcomes?	Achievement of: Increased knowledge and awareness of the nature of HIV/AIDS and the ways to address the disease Increased individual and organizational capacity (outside of FI partners) Increased Canadian engagement and leadership in the global context Enhanced engagement and collaboration on approaches to address HIV/AIDS	Progress Made; Further Work Warranted
4.3 To what extend has progress been made toward the achievement of second level outcomes?	Achievement of: Reduced stigma, discrimination and other barriers Improved access to more effective prevention, care, treatment and support Internationally informed federal response Increased coherence of the federal response	Progress Made; Further Work Warranted

Issue 5: Demonstration of Economy and Efficiency	Indicators	Overall Rating
SUMMARY: The evaluation found that the prevention of HIV/AIDS is more cost-effective than treating the infection. The Federal Initiative is demonstrating efficiencies in key areas, including laboratory science, research and community-based funding. The integrated approach in these areas of the Federal Initiative may lead to broader impact and greater efficiency of the investment. The governance of the Federal Initiative has clear benefits for partners and assists with the horizontal management of the Federal Initiative. Therefore, the overall rating has been assessed as achieved for the governance of the Federal Initiative, with the remainder assessed as achieved.
4.5.1 Was resource use appropriate for the outputs produced and outcomes achieved? Are appropriate performance data being collected? If so, is this information used to inform senior management and decision makers?	Relationship between expenditures and outputs and outcomes relative to priorities and Demonstration that performance data are used to inform senior management decision making	Progress Made; Further Work Warranted
4.5.2 Is there variance between planned and actual expenditures? If so, what are the implications?	Identification of available financial data on allocations and expenditures Demonstration of planned vs. actual expenditures (Federal Initiative activities) Perception of the implication of variances	Progress Made; Further Work Warranted
4.5.3 Does the horizontal (i.e. interdepartmental) approach for the Federal Initiative add value? What are the added benefits relative to associated costs? Are there any disadvantages to this approach?	Demonstration of added value (integration, benefits and costs of horizontal approach)	Achieved

Evaluation Issue	Achieved	Progress Made; Further Work Warranted	Little Progress; Priority for Attention	Not Applicable
Relevance:
Issue 1: Continued need for the program
1.1 What is the magnitude of HIV in Canada and globally?		X
1.2 In what areas has the HIV and AIDS environment changed most since 2008?		X
Issue 2: Aligned to federal government priorities
2.1 Do FI partners' current roles align with federal government and FI partners' priorities?	X
Issue 3: Program consistent with federal roles and responsibilities
3.1 Is the Federal Initiative aligned with the roles and mandates of FI partners? Are current FI partners the right partners? Do we need others?		X
3.2 To what extent are the roles and responsibilities of each partner defined and understood in the Federal Initiative?		X
3.3 How is the Federal Initiative's role different from or similar to the roles of provinces and territories, and non-governmental organizations? Does the FI's role complement or duplicate/conflict with roles of other stakeholders? Is there a common understanding of the initiative's roles among stakeholders?	X
Performance:
Issue 4: Achievement of intended outcomes (effectiveness)
4.1 Is the management accountability and governance for the FI in place and functioning adequately to achieve expected results?		X
4.2 To what extent has progress been made toward the achievement of first level expected outcomes?		X
4.3 To what extend has progress been made toward the achievement of second level outcomes?		X
Issue 5: Demonstrated economy and efficiency
5.1 Was resource use appropriate for the outputs produced and outcomes achieved? Are appropriate performance data being collected? If so, is this information used to inform senior management and decision makers?		X
5.2 Is there variance between planned and actual expenditures? If so, what are the implications?		X
5.3 Does the horizontal (i.e. interdepartmental) approach for the Federal Initiative add value? What are the added benefits relative to associated costs? Are there any disadvantages to this approach?	X

Page details

Date modified:: 2014-08-08