Executive summary

Context

This report presents the results for the Evaluation of the Adaptation Theme of Canada’s Clean Air Agenda, which was undertaken by Environment and Climate Change Canada’s (ECCC) Audit and Evaluation Branch, in collaboration with its federal partner organizations.

The Government of Canada’s Clean Air Agenda (CAA), in effect until 2015-16, was intended to address climate change and air pollutants via five program themes, one of which was the Adaptation Theme. As of December 2016, the government’s climate change activities fall under the Pan-Canadian Framework on Clean Growth and Climate Change.

The CAA Adaptation Theme was a five-year (2011-12 to 2015-16), $181.62 million initiative that aimed to deliver on the federal role as defined by the Federal Adaptation Policy Framework, approved in 2011^{Footnote 1} . The Theme was coordinated by Environment and Climate Change Canada.

Adaptation Theme programming was organized into four streams, by goal and comprised a total of 10 program elements. Each program element was delivered by one or more of nine federal partners.

Enhance the science foundation to understand and predict climate and assess climate change impacts

• Aquatic Climate Change Adaptation Services Program (ACCASP) – Department of Fisheries and Oceans (DFO)
• Climate Change Prediction and Scenarios Program (CCPSP) – Environment and Climate Change Canada (ECCC)
• Understanding Climate-Driven Ecological Changes in Canada’s North (UCECCN) – Parks Canada Agency (PCA)

Enhance public health and safety

• Heat Alert and Response Systems (HARS) – Health Canada (HC)
• Preventative Public Health Systems and Adaptation to Climate Change Program (PPHSACC) – Public Health Agency of Canada (PHAC)

Build resilience in the North and climate-sensitive Aboriginal communities

• Climate Change and Health Adaptation Program for Northern First Nations and Inuit Communities (CCHAP) – Health Canada (HC)
• Climate Adaptation and Resilience Program for Aboriginals and Northerners (CARPAN) – Indigenous and Northern Affairs Canada (INAC)
• Integrating Adaptation into Codes and Standards for Northern Infrastructure (also known as the Northern Infrastructure Standards Initiative [NISI]) – Standards Council of Canada (SCC) and Indigenous and Northern Affairs Canada (INAC)

Enhance the competitiveness of climate-sensitive economic sectors and systems

• Enhancing Competitiveness in a Changing Climate (also referred to as the NRCan Adaptation Program) – Natural Resources Canada (NRCan)
• Northern Transportation Adaptation Initiative (NTAI) – Transport Canada (TC)

The evaluation is horizontal in scope and examines the Adaptation Theme across the 10 program elements. The evaluation covers the time frame from 2011-12 to mid-2014-15, (representing approximately two-thirds of the funding period) and also, where appropriate, looked at relevant activity in the remainder of 2014-15. The evaluation examined two key areas:

• Relevance: the extent to which the Adaptation Theme addressed a continued need, was aligned with federal priorities and was aligned with federal roles and responsibilities
• Performance: the extent to which the Adaptation Theme achieved intended outcomes and demonstrated efficiency and economy

The current evaluation collected data on nine of the 10 program elements. Transport Canada conducted an independent evaluation of the NTAI in 2014-15 and the results of the Transport Canada evaluation were incorporated into the current evaluation.

Findings and conclusions

Relevance

Evaluation findings indicate that there is a continued environmental, societal and economic need for initiatives to assist Canadians in adapting to climate change and the risks and opportunities it creates. Most key informants reported that climate change needs would not be met effectively in the absence of Adaptation Theme programming.

The Adaptation Theme is aligned with federal priorities related to protecting Canadians, promoting economic growth and development and addressing the sustainability of natural resources and priorities related to the North, including those reported in Canada’s Northern Strategy. The Theme and program elements also align with Canada’s Federal Sustainable Development Strategy (FSDS) and are consistent with federal roles and responsibilities, including those as established by related Acts and the Federal Adaptation Policy Framework. The federal government is viewed by key informants as being well positioned to provide leadership and information and to facilitate collaboration on climate change adaptation.

Performance – efficiency and economy

Overall, program elements comprising the Adaptation Theme appear to be generally well designed and delivered; however, some areas for improvement related to information sharing, increasing engagement and performance measurement were noted.

The different program elements use a variety of program delivery methods to work towards achieving intended outcomes. A majority of key informants indicated that the design of program elements is appropriate for meeting the intended outcomes. Knowledge exchange with stakeholders is identified as a critical aspect of the Adaptation Theme’s program design. While some stakeholder engagement is occurring, more could be done to encourage the participation of existing and potential stakeholders, to increase effectiveness. The products and activities of the program elements are generally perceived to be of high quality and useful to external stakeholders; however, some reported that it was difficult to find them and that activities have not been well advertised.

Governance at the program element level is clear and effective. At the Adaptation Theme level, three committees exist to collectively manage and coordinate the delivery of the Theme activities federally and across jurisdictions. Despite this, there was a strong desire for more collaboration and information sharing among program elements, as well as among federal and non-federal program partners.

Findings suggest that program resources are being used efficiently and economically. Examples of effective cost-savings practices include the use of information technologies to assist with communication, collaboration and outreach and the use of networks to promote activities and share information. Funding for Adaptation Theme activities includes both new and existing funding. Coordinated horizontal financial reporting occurs for the Adaptation Theme; however, the report provides details for new funding only. Some federal partners were unable to provide budget or expenditure figures for their existing funding or details to sufficiently account for variances between new or existing funding and actual expenditures. As a result, it is not possible to present a comprehensive picture of the total budget and expenditures for Adaptation Theme activities over the four-year period from 2011-12 to 2014-15.

A logic model and performance measurement framework (PMF) for the Adaptation Theme are in place; however, the Theme-level PMF does not adequately capture all relevant intended outcomes for all program elements. While some performance data is being collected and reported by program elements through means such as corporate reporting, a number of issues were observed indicating that insufficient data is being collected to support an assessment of  Theme-level results. Program staff and senior managers indicated that performance data is used for reporting at the program element level and provided examples of the use of data in decision making.

Performance – effectiveness

Overall, program elements are making progress towards achieving the Adaptation Theme immediate outcomes. Progress is being made towards recognition by targeted communities and sectors of the need for adaptation and for assessing their risks and opportunities arising from climate change. Adaptation measures are being identified, and awareness of relevant adaptation measures among targeted communities and sectors and collaboration on climate change adaptation are increasing. The evaluation also found early evidence of progress toward intermediate and final outcomes, examples of which include developing research facilities, planning around food security and implementing ground stability systems as part of adapting to climate change. However, the longer term nature of these outcomes means available evidence is still somewhat limited; it is too early to conclude on the extent of progress towards reducing the vulnerability of individuals, communities, regions and economic sectors to the effects of climate change.

Recommendations

Based on the nature of this evaluation focused on the overall Adaptation Theme and the diversity of the different Adaptation program elements, the recommendations reflect observations that were common to most program elements. Accordingly, the recommendations are broadly worded and reflect the understanding that, while generally applicable across all elements, each responsible department will specify through its management responses and action plans how these recommendations should best be addressed for the respective program elements.

Recommendations 1, 2 and 3 are directed to senior management^{Footnote 2}  of each of the nine federal organizations that have been involved in delivering the Adaptation Theme programming. Recommendation 4 is directed to all departments, with the exception of INAC and TC. No areas of concern were found in reviewing the financial information provided by these two departments.

Recommendation 1

Review current adaptation program delivery to identify opportunities to expand engagement among new and existing stakeholder groups and increase awareness of information, products or activities.

Recommendation 2

Review mechanisms for collaboration and information sharing between the Adaptation program elements, as well as with other federal and non-federal program partners, in order to leverage best practices and resources and avoid duplication of effort.

Recommendation 3

Review the current performance measurement framework and data to improve tracking and reporting on progress toward Adaptation-theme level intended results.

Recommendation 4

In collaboration with the departments’ respective Chief Financial Officers, review current processes for opportunities to improve program tracking and reporting of financial information for Adaptation program elements.

Management response

Due to the number of participating departments and the diversity of program elements involved in the Adaptation theme, each partner department is encouraged to develop its own individual management action plan in response to the evaluation’s recommendations in the manner that best addresses its particular circumstances. These responses should be coordinated with other federal Adaptation partners, as appropriate.

Accordingly, the individual departmental management responses are not included in the body of the report. It is the responsibility of each participating department to develop and publish its own management responses under separate cover.