4. Findings

This section presents the findings by evaluation issue (relevance and performance) and by the related evaluation questions. For each evaluation question, a rating is provided based on a judgment of the evaluation findings. The rating statements and their significance are outlined below in Table 1. A summary of ratings for the evaluation questions is presented in Appendix E.

Table 1: Definitions of standard rating statements

4.1 Relevance

4.1.1 Continued need for program

Given the wide-ranging impacts of climate change, there is a continued environmental, societal and economic need for initiatives to assist Canadians in adapting to climate change, as well as the risks and opportunities it creates.

Environmental, societal and economic need

• The combined global average land and ocean temperature increased by 0.85°C from 1880 to 2012, and warming within Canada is occurring at roughly twice the global rate. According to an Intergovernmental Panel on Climate Change (IPCC) 2013 report, most aspects of climate change will persist for many centuries, even if CO₂ emissions are stopped, generating a need for initiatives to assist Canadians to adapt to the effects of rising sea levels, decreasing ice levels, thawing permafrost and more frequent extreme weather events.
• Although economic evidence for Canada is limited, “climate change is expected to impose accelerating costs [in Canada], rising from an average value of $5 billion per year in 2020 to an average value of between $21 billion and $43 billion per year by 2050.” ^{Footnote 1} 
• The effects of climate change are having wide-ranging repercussions resulting from disruption of transportation, loss of natural resources and risks posed to critical infrastructure as well as to health:
  • The forestry sector is a multi-billion dollar industry that supports thousands of jobs and hundreds of communities. Climate change has altered ecological environments which affect forest growth and productivity.
  • The Nunatsiavut area, located along the northern coast of Labrador, has experienced steep increases in temperatures since 1993, leading to unprecedented reductions in snow and sea ice cover. Rising temperatures have caused rapid landscape and biophysical changes in the region, affecting infrastructure, community services and the wellbeing of residents.
  • Permafrost degradation is resulting in increased and more frequent occurrences of potholes, sinkholes, slumping and settlement issues in transportation infrastructure. Failure to address transportation infrastructure vulnerabilities could increase infrastructure maintenance costs in the future and hinder economic development in the North.
  • Changing climate has the potential to affect health status in many harmful ways (for example, related to extreme weather, air quality and illness from food-, water- and rodent-transmitted diseases), which in turn have the potential to lead to a number of economic and social impacts (for example, loss of life, changed welfare, costs to the health care system and impacts on productivity).
• Similar to the documented evidence, key informants also noted examples of climate change impacts on physical health, food security and infrastructure. For instance, emerging diseases such as West Nile virus and Lyme disease affect physical health; food security has been affected by such climate change-induced effects as delayed migrations (for example, the Atlantic salmon run); and changes to permafrost have affected infrastructure in the North. It was also noted that the agri-food industry is vulnerable to climate change, as changes in temperature could impact such things as crop management and the ability to harvest.
• A majority of key informants reported that climate change needs would not be met effectively in the absence of Adaptation Theme programming. Some key informants noted a continued need for specific research and tools such as climate forecasts, risk assessments and capacity building tools and for information on how to adapt Northern and coastal infrastructure.

Furthermore, documented information gaps include data on how climate change has affected public health in Northern communities, information regarding climate change impacts on northern transportation infrastructure and information for business and industry on climate change impacts and adaptation responses.

Complementarity

• Although adaptation work is conducted by a large number of groups at the international, federal, provincial, territorial and local levels, key informants were generally of the opinion that the Theme programming was designed to encourage complementary work, identify cross-cutting issues and common stakeholder needs and provide a central forum to share concerns, data and tools to reduce the duplication of efforts across jurisdictions.
• Key informants identified other specific federal government programs that address issues related to climate change adaptation, beyond those encompassed under the Theme. These programs were considered to complement and not duplicate the Theme’s program elements. Examples include:
  • HC’s First Nations and Inuit Public Health Protection program, which provides environmental public health research, surveillance and risk analysis in the areas of climate change, health adaptation and biomonitoring directed to on-reserve First Nations and Inuit and First Nations living above the 60^th parallel.
  • INAC’s Canadian High Arctic Research Station Initiative, which seeks to improve Canada’s knowledge of the Arctic in order to improve environmental stewardship, economic opportunities and the quality of life of Northerners through the establishment of the Canadian High Arctic Research Station.
  • ECCC’s Climate Change and Clean Air Regulatory Program, which is designed to develop sector-based approaches to regulating air pollutants and controlling greenhouse gas emissions and to promote science-based approaches to inform the development of new standards and regulations.

4.1.2 Alignment with federal government priorities

The Adaptation Theme is aligned with federal priorities related to protecting Canadians, promoting economic growth and development and addressing the sustainability of natural resources, as well as priorities related to the North, including those reported in Canada’s Northern Strategy. The Theme and program elements also align with Canada’s Federal Sustainable Development Strategy (FSDS).

• The 2011 Federal Budget provided funding to continue to protect Canada’s natural environment and address environmental risks, including funding for projects to improve our understanding of climate change impacts. Canada’s Northern Strategy also outlines priorities closely aligned with the Adaptation Theme, specifically those focussed on promoting social and economic development and protecting the North’s environmental heritage. Furthermore, the 2013 Speech from the Throne underlined that investment in the North and Northern communities and infrastructure was a priority.
• FSDS 2013-16 included a climate change adaptation target to “Facilitate reduced vulnerability of individuals, communities, regions and economic sectors to the impacts of climate change through the development and provision of information and tools.” ^{Footnote 2} 
• Adaptation Theme partner organizations recognize the need to address climate change adaptation in their Reports on Plans and Priorities. On a similar note, federal key informants also generally indicated alignment between the Adaptation Theme and federal priorities. Reference was made to aspects related to different elements of the Adaptation Theme, such as: protecting vulnerable communities; reducing the impact of health threats; supporting economic development and prosperity; conserving natural and cultural parks and areas; contributing to the sustainability of natural resources; developing Northern infrastructure; and supporting the Northern Strategy.

4.1.3 Alignment with federal roles and responsibilities

Activities undertaken as part of the Adaptation Theme are consistent with federal roles and responsibilities, including those established by related Acts and the Federal Adaptation Policy Framework. The federal government is viewed as being well-positioned to provide leadership and information and to facilitate collaboration on climate change adaptation.

• Authority for the work undertaken by federal partners involved in the Adaptation Theme is established by a number of different Acts. ^{Footnote 3}  The Canada Transportation Act, the Department of Health Act and the Department of Indian Affairs and Northern Development Act, among others, authorize engagement in activities for the health and benefit of Canadians as they relate to their respective federal partners. Program activity under the Theme is also consistent with federal responsibilities related to geographic areas (for example, oceans), Indigenous people on reserve and interprovincial and international matters. Documentary evidence also supports alignment between the activities being undertaken as part of the Adaptation Theme and the mandates or missions of the involved federal partners.
• The Adaptation Theme is also consistent with the roles of the federal government set out in the Federal Adaptation Policy Framework, which was approved by Cabinet in 2011. The Framework notes that “[t]he Government of Canada is well positioned to mobilize economies of scale to generate and deliver fundamental knowledge and information that can be applied across the country” and that “the federal government is particularly well-positioned to support the development and dissemination of climate change information, guidance and tools that help Canadians to adapt.” ^{Footnote 4} 
• In addition, Canada is a party to the United Nations Framework Convention on Climate Change (UNFCCC), which includes commitments on monitoring and research defined in Articles 4 and 5.
• Findings from the key informant interviews further support the conclusion that the work of the Adaptation Theme is consistent with federal roles and responsibilities. For example, the Theme is generally considered to:
  • align with departmental mandate and missions
  • provide leadership on climate change adaptation as it affects national health and safety
  • be well-positioned to provide national consistency
  • be a trusted source of information, since it employs climate change experts and has the capacity to conduct reliable research and a facilitator of research and tool development
• provide forums wherein stakeholders from provinces, territories, municipalities, Indigenous communities and industries could work collaboratively to approach climate change adaptation through the sharing of ideas and knowledge

4.2 Performance – efficiency and economy

4.2.1 Program design

The 10 program elements within the Adaptation Theme use a variety of program delivery mechanisms, and the design of program elements is generally thought to be appropriate for meeting intended outcomes. Knowledge exchange with stakeholders is identified as a critical aspect of program design. While some stakeholder engagement is occurring, more could be done to encourage the participation of existing and potential stakeholders to increase effectiveness.

• A majority of key informants indicated that the design of program elements is appropriate. They identified a variety of delivery mechanisms that work towards achieving intended outcomes. Examples of different program models include funding projects to develop outputs; conducting training and information sharing with stakeholders; and conducting research and innovation, developing models and projections or developing standards and tools to adapt to climate change while involving experts.
• Survey results also indicate that program element designs are generally appropriate and viewed to be of good quality. Large majorities of survey respondents rated the program elements’ products and activities as very or somewhat credible (97%), up-to-date (91%), comprehensive (89%), timely (84%) and available in an accessible format (82%). The quality of activities received similarly strong ratings, with respondents rating the activities with which they were familiar as somewhat or very informative (95%), well-organized (94%) and effective (94%).
• The usefulness of program outputs was also rated very highly. On average, 92% of respondents rated the information generated or knowledge gained through Theme activities as somewhat or very useful to their organization. Similarly, almost all Adaptation products were, on average, rated as somewhat or very useful to their organization by more than nine in 10 respondents (95%). Across all programs, communication with the federal government regarding program activities or products was rated as somewhat or very easy by at least 75% of respondents. Communication with HC’s CCHAP was rated as “very easy” by 81% of the survey respondents.
• In contrast, survey ratings were lower for the degree to which products and services were rated as easy to find, with 60% indicating the products and services were somewhat or very easy to find, suggesting there may be an opportunity to increase the visibility of program activities and products.
• A key component of this Theme is knowledge exchange with stakeholders in order to “understand which information is needed, how to deliver our knowledge products and in which format” that would best suit the needs of the users. The document review and case studies identified several examples where the program elements engaged with stakeholders to define stakeholder needs and target activities.
  • Selection committees for HC’s CCHAP include community, government and non-government representatives. The committees evaluate proposals and make recommendations regarding funding.
  • PC’s UCECCN conducted workshops with stakeholders to better refine information needs.
  • ECCC’s CCPSP conducted workshops with stakeholders to improve understanding of the supply of and demand for climate information and to improve coordination to ensure that accessible and timely information is available to decision makers.  More broadly, ECCC worked to advance the Federal Adaptation Policy Framework across the federal government (for example, through presentations to the Deputy Ministers Committee on Climate Change, Energy and the Environment) and led interdepartmental policy coordination to improve communications and linkages among program elements.
  • SCC’s and INAC’s NISI conducted interviews with local area experts to assess the information gaps that need to be addressed. The Northern Advisory Committee on Adaptation Codes and Standards, composed of representatives from Yukon, Northwest Territories, Nunavut and Nunavik, was established to guide work related to this program element.
  • Working groups in the Adaptation Platform organized by the NRCan Adaptation Program brought together stakeholders from governments, national industry and professional organizations, industry sector champions and academia to review  particular topics (for example, mining and coastal management) to produce State of Play reports and create programs of work. This work was discussed at biannual plenary meetings. Platform members also had access to a shared online workspace to further their collaboration.
  • The work of TC’s NTAI involved funded projects and related discussions and presentations through networks that scoped potential impacts of climate change on the northern transportation system and identified specific vulnerabilities in the transportation system. ^{Footnote 5} 
  • The case study of DFO’s ACCASP Coastal Infrastructure Vulnerability Index project shows DFO’s Science and Small Craft Harbours experts were engaged to develop a coastal climate change vulnerability index for use by the Small Craft Harbours program. Similarly, NRCan’s Forest Change Initiative case study found that the forest sector helped to gather information on the impacts of climate change on Canadian forests and developed strategies to address these impacts.
• Despite the examples cited above, almost half of all key informants expressed that, while progress was being made, greater efforts were required to improve the level of engagement with existing external stakeholder groups and to reach out to other stakeholders. Identified stakeholder groups included:
  • other federal government departments such as DFO, NRCan, Agriculture and Agri-food Canada, and the National Research Council
  • other levels of government such as provinces, territories, municipalities, and communities
  • industry, academics, non-governmental organizations, emergency preparedness groups, and Indigenous stakeholders
• Benefits of increased engagement with these stakeholders, as identified by interviewees, include:
  • increased potential to leverage external skills and technical capacity
  • increased capacity-building
  • improved knowledge and understanding of target audiences to better understand their needs and enable the development of more tailored deliverables
  • expanded use of adaptation information
  • increased dissemination and greater uptake of new adaptation tools

4.2.2 Program governance and management

Governance for individual program elements appears to be clear and effective. Adaptation Theme-level committees collectively manage and coordinate the Adaptation Theme federally and across jurisdictions and meet regularly to share information on results and best practices. Despite this, a need was identified for improved integration and coordination among program elements and federal and non-federal program partners to promote efficiencies.

• Although governance at the program element level was not a specific focus of this evaluation, several examples of program-level structures, such as steering committees and technical committees, are in evidence. In addition, most of the projects examined in the case studies have formal, documented governance structures. For the most part, key informants generally indicated that, within program elements, decision-making processes are transparent, structures are clearly documented and there is a good understanding of reporting requirements and roles and responsibilities.
• In terms of governance at the Adaptation Theme level, as described in section 2.2, there are three Theme-level committees that focussed on managing and coordinating the delivery of the Theme’s activities federally and across different jurisdictions, including:
  • An interdepartmental Director General Adaptation Policy Steering Committee (DGAPSC), which  advances a coordinated policy approach to climate change adaptation at the federal level, supported by working-level collaboration;
  • An interjurisdictional FPT Adaptation Policy Committee, which promotes improved coordination of adaptation policy across federal, provincial and territorial governments; and
  • An interdepartmental Director General Management Committee (DGMC), which coordinates the Adaptation Theme programs. ^{Footnote 6}
• All three Theme-level committees have Terms of Reference (TOR) and met on a regular basis over the timeframe covered by the evaluation. ^{Footnote 7} 
• A DGMC workshop to share information on results, best practices and lessons learned was held in September 2014. This was viewed positively by interviewees as a method of collaborating and sharing information. However, there was no evidence of other similar mechanisms aimed at facilitating information sharing among program elements. Further, in the view of some interviewees, it was not clear how effective the DGMC was as a mechanism for collaboration and information-sharing.
• Despite the existence of these horizontal committees to coordinate Theme activities, views were mixed regarding the extent to which the committees are having the desired effect.  While several positive references to horizontal meetings were made, a few senior managers and nearly half of key informants across all remaining groups reported a desire for more collaboration and information sharing among program elements, as well as among federal and non-federal program partners. Both interviewees and survey respondents suggested that greater collaboration would help to support effective program delivery and could provide opportunities for increased efficiency such as:
  • potential for reduced duplication of effort from leveraging tools and practices developed or used by other program elements
  • adoption of shared useful practices such as a method of ecosystem classification to standardize ecosystem descriptions and coordinate ecological assessments across areas of interest to multiple program elements
  • efficiencies from coordinating communication among stakeholders within the different program elements who are working on similar projects (for example, organizing joint meetings)
  • sharing similar challenges and discussing potential solutions
• Although structured differently than the overall Adaptation Theme, NRCan’s Adaptation Platform, a component of the NRCan Adaptation Program, was noted as a good example of a mechanism that provides opportunities for communication at both senior and working levels by providing a space to discuss ideas and challenges with others working on the Adaptation Theme, as well as with experts in the field. The Adaptation Platform consists of a plenary body of senior-level representatives and a series of thematic working groups composed of working-level experts from governments, national industry and professional organizations, industry sector champions and academia. ^{Footnote 8}  Involvement in the working groups is based on identified themes, such as a natural resource sector (for example, mining), or a cross-sectoral or regional focus (for example, northern regions).

4.2.3 Program efficiency and alternatives

Findings suggest that program resources are being used efficiently and economically. Examples of effective practices to reduce program delivery costs include regular use of information technologies to assist with communication, collaboration and outreach and the use of networks to promote activities and share information. Due to limitations in the financial data, it is not possible to present a comprehensive picture of the total funding, including both new and existing funding, or expenditures for Adaptation Theme activities over the four-year period from 2011-12 to 2014-15.

• Evidence indicates that program elements encouraged operational efficiencies by leveraging networks to promote activities and share information. For example:
  • HC’s HARS program element uses existing working groups or networks (where they exist) and increased its reach by partnering with The Weather Network.
  • The NRCan Adaptation Platform uses members’ networks to share information and publicize its webinar series.
  • First Nations and Inuit representatives participated in HC’s CCHAP’s project selection process and reached out to networks in their communities and regions to promote the program.
  • ECCC’s CCPSP worked with the Natural Sciences and Engineering Research Council’s Climate Change and Atmospheric Research (CCAR) Initiative, providing support to external researchers whose projects contributed to or improved climate prediction science.
• Documents and performance data indicate that program elements were able to leverage funds through collaborations. For example:
  • PMF performance data indicate that for every dollar that DFO’s ACCASP invested in research projects and adaptation tools in 2013-14, 80 cents of cash and in-kind contributions were provided.
  • Between 2011-12 and 2014-15, NRCan’s Adaptation Program’s Earth Sciences Sector component leveraged $1.23 from non-federal sources for every dollar of federal G&C funding.
  • Twelve of the 37 projects examined in TC’s NTAI evaluation included financial or in-kind support from partners outside of TC, resulting in an estimated $1.05 million in non-federal funding.
• Evidence also shows that program elements make regular use of information technologies to assist with communication, collaboration and outreach, to help to reduce program costs. For example:
  • HC’s CCHAP and SCC’s and INAC’s NISI both developed websites to keep partners and stakeholders informed.
  • The Adaptation Platform component of the NRCan Adaptation Program used an online workspace and teleconference calls to manage and share the knowledge and serve as mechanisms for working group collaboration.
  • HC’s HARS and the NRCan Adaptation Program used webinars to share information and build capacity. PHAC program representatives also noted the use of webinars and conference presentations.
  • Key informants noted that TC’s NISI sometimes used web or conference calls in lieu of in‑person meetings. The savings were then reallocated to developing other standards.
• Key informants generally reported that program operations are efficient, as supported by regular reporting, work plans, process and tracking documents and governance and advisory meetings. Key informants also identified some best practices that program elements currently employ to support efficient and economic program delivery, such as engaging with relevant stakeholders to leverage their capacities. This included working with intermediary regional organizations with the knowledge, skills and connections to distribute information to local end-users; building relevant capacity in Northern communities; and increasing project buy-in and uptake via partnerships and a participatory approach.
• When asked about lessons learned, other key informants suggested ways to further improve delivery, including:
  • increased interaction among those who deliver the programs
  • increased engagement with communities and local experts
  • early engagement with stakeholders overall
  • improved or increased focusing of resources on the highest priority (or most vulnerable) regions, sectors and infrastructure
• A majority of key informants did not identify alternative models that would achieve similar outcomes at a lower cost and a majority of program managers reported that the level of funding was sufficient to reach current program goals.
• Coordinated horizontal financial reporting occurs for the Adaptation Theme. This is provided in a consolidated CAA Spending Overview report, prepared by ECCC’s Corporate Services and Finance Branch and submitted to the Treasury Board Secretariat. This financial report identifies both the planned and actual expenditures approved by the Chief Financial Officer of each organization participating in the Adaptation Theme. However, the CAA Spending Overview report only provides details on new funding.
• As mentioned in section 2.3, total funding for the Adaptation Theme activities included both new and existing funding. New funding for all nine organizations accounted for the majority of total funding (82%). Existing funding accounted for 18% of the total and was distributed across seven of the nine organizations. Financial tables are provided in Appendix B.
• Only three of seven organizations with existing funding were able to provide full expenditure details related to the Adaptation Theme; the remaining four organizations had difficulty providing expenditure details and provided only partial information or no expenditure details at all for existing funding.^{Footnote 9}  In addition, not all organizations provided details to sufficiently account for variances between budgets and expenditures for both new and existing funding. As such, it was not possible to present a comprehensive financial picture of the total expenditures for the Adaptation Theme activities.

4.2.4 Performance measurement

A logic model and performance measurement framework (PMF) for the Adaptation Theme are in place; however, the Theme-level PMF does not adequately identify all relevant intended outcomes for all program elements. While some performance data is being collected and reported by program elements through means such as corporate reporting, a number of issues were observed, indicating that the data collected is inadequate to support an assessment of Theme-level results.

Performance measurement

• Although an Adaptation Theme-level PMF was established, an analysis of the document revealed that it did not adequately identify all relevant intended outcomes for each of the program elements. Immediate outcomes were identified for all program elements, but only five of the 10 program elements had one or more intermediate outcomes listed and only two program elements included a final outcome.^{Footnote 10}  Indicators and targets were specified for outputs and most of the identified outcomes.^{Footnote 11} 
• As of December 2014, all program elements had provided some performance data to report on the progress of indicators identified in the Theme-level PMF. However, several issues were identified with the performance data, such as misalignments between the reported data and the target or outcomes,^{Footnote 12}  missing data^{Footnote 13}  and incomplete data.^{Footnote 14}  Only two of the five program elements with intermediate outcomes reported data on these outcomes.^{Footnote 15}  As a result, there is insufficient data to determine the degree of progress towards meeting the targets for approximately half of the indicators in the PMF.
• A review of the available data indicated that program elements frequently used the approach of reporting output data as a proxy for outcome data in the PMF, despite the fact that outputs were not always clearly related to outcomes. For example:
  • ECCC’s CCPSP provided data on the number of downloads of climate model output datasets for the outcome “Targeted communities and sectors recognize the need for adaptation”.
  • PHAC’s PPHSACC reported presentations delivered to stakeholders as evidence that the outcome “Targeted communities and sectors assess their risks and opportunities arising from climate change” was being achieved.
  • SCC’s and INAC’s NISI provided the number of participants at a workshop for the outcome “Targeted individuals, communities and sectors implement adaptation measures.”
• A few interviewees indicated that performance data is being collected to some degree, while others identified additional types of performance data that would be beneficial. Suggestions included:
  • data on the effects of activities at a community or population level
  • collection of pre and post data to analyze changes in awareness at the targeted community or sector and general public levels
  • analysis of output use and whether use affects climate change adaptation (for example, local applications of tools, changes in responses to extreme events)

Reporting and use of performance data

• Although there is no overarching reporting mechanism for the performance of the Theme, corporate reporting within each of the federal organizations provides information on progress towards achieving outcomes for the individual program elements. However, the structure and detail of reporting of Adaptation information is inconsistent across organizations. Examples of progress reporting in Departmental Performance Reports (DPRs) include program narratives, summaries of progress and lessons learned:
  • HC’s DPR described progress regarding HARS in a narrative format, identifying the number of partner communities and giving examples of the work completed.
  • NRCan’s DPR summarized progress on the Adaptation Platform in a narrative that reports the number of members, new products, amount of funding approved and funds leveraged through partnerships.
• Program staff and senior managers indicated that performance data is used for reporting and also provided some examples of the use of this data in strategic planning and decision making at the program level.

4.3 Performance – effectiveness

8. To what extent have intended outcomes been achieved as a result of the program?

Immediate outcomes

Intermediate outcomes

Final outcomes

The nature and scope of intended adaptation impacts is broad and diverse and the evidence of progress is largely comprised of specific examples linked to different program elements. Overall, program elements are making progress towards achieving the Adaptation Theme’s immediate outcomes. Evidence suggests that targeted communities and sectors have begun to recognize the need for adaptation, assess climate change risks and opportunities and identify adaptation measures, while their awareness of adaptation measures and collaboration on climate change adaptation are increasing.

Despite some evidence suggesting progress towards achieving the intermediate and final outcomes, it is still too early to conclude on progress in implementing adapting measures or on progress towards reducing the vulnerability of individuals, communities, regions and economic sectors.

This section presents the evaluation findings for each intended outcome. A rating statement summarizing observed progress towards achieving each intended outcome is included at the beginning of each of the sub-sections. Ratings were based on available performance data, as well as on other data collection methods such as the interviews, survey and document analysis. Data collection instruments for key informant interviews and the survey were designed such that respondents were asked only about the program elements with which they were most familiar.

Immediate outcome 1: Targeted communities and sectors recognize the need for adaptation^{Footnote 16}  – Acceptable

Program element activities and outputs contributed to the recognition by targeted communities and sectors of the need for adaptation. Progress towards achieving this outcome was made through program outreach and information sharing with stakeholders regarding the necessity of climate change adaptation.

• Documents, performance data and case studies indicate that progress is being made towards targeted communities and sectors recognizing the need for adaptation. Specific examples of activities and outputs include:
  • PHAC’s PPHSACC reported giving 53 presentations on public health and environmental change to stakeholders between 2011-12 and 2013-14.
  • Case study findings indicate that PPHSACC’s Climate Change, Adaptation and Acute Gastrointestinal Illness in the Canadian North project measured and shared self-reported risk factors for gastrointestinal illness in Rigolet.
  • Performance data for HC’s CCHAP for 2012-13 and 2013-14 indicate that all 27 projects funded (15 in 2012-13 and 12 in 2013-14) identified the need for adaptation plans.
  • ECCC’s CCPSP, which tracks both direct data requests and online user access to its information, reported having received 61 direct individual requests over three years for temperature data from all regions of Canada and international institutions. Performance data for 2012-13 indicates that there were 21,298 downloads of climate model output datasets from the Canadian Centre for Climate Modelling and Analysis website.
  • INAC’s CARPAN reported that three regional workshops (Yukon, Northwest Territories and Inuit regions) were delivered to showcase past projects, generate interest and plan for the new program. With the support of CARPAN, diverse methods of information sharing and program outreach were used to contribute to the increased recognition of the need for adaptation. Examples of these methods include regular meetings to advise community members of issues, the development of an internet site focused on adaptation in relation to identified risks and corresponding workshops.
• A majority of key informants agree that the program elements contribute to this outcome through such means as information sharing activities on the effects of climate change (for example, through workshops, a training course or regional dialogues) and data and research related to the effects of climate change. In addition, the majority of the surveyed external stakeholders indicated that program element products (86%) and activities (81%) contributed to a large extent to community or sector recognition of the need for adaptation.

Immediate outcome 2: Targeted communities and sectors assess their risks and opportunities arising from climate change^{Footnote 17}  – Acceptable

Program elements supported targeted communities and sectors to assess risks and opportunities arising from climate change through the development of models, projections and scenarios.

• Overall, the evidence suggests that solid progress is being made towards achieving this outcome, as demonstrated by several specific examples:
  • HC’s CCHAP reported funding multiple First Nation and Inuit projects that assessed climate change and health risks (16 projects in 2012-13 and 12 projects in 2013-14).
  • HC’s HARS reported conducting vulnerability assessments for two communities between 2011‑12 and 2013-14.
  • The NRCan Adaptation Program supported eight risk assessments for coastal communities and five for the energy industry since 2013.
  • The NRCan Adaptation Program’s Forest Change Initiative created an adaptation toolkit to enable stakeholders to identify risks and opportunities and implement adaptation strategies.
  • INAC’s CARPAN supported projects that conducted risk and vulnerability assessments to identify areas of focus for adaptation measures (for example, such projects included the Unama’ki Water and Wastewater Vulnerability Assessment: Eskasoni First Nation).
  • TC’s NTAI funded scoping studies, vulnerability assessments and monitoring/baseline studies through grants and contributions and contracts. These included, for example, projects related to assessing climate change adaptation for transportation in Arctic waters, assessing the vulnerability of the infrastructure and operations of the Port of Churchill and associated shipping routes, identifying potential impacts of climate change on the future streamflow and water levels of the Mackenzie River and assessing permafrost response to climate warming on the Dempster Highway.
  • DFO’s ACCASP Coastal Infrastructure Vulnerability Index project is being developed to enable managers to identify risk and adaptation strategies relative to infrastructure at DFO Small Craft Harbours coastal infrastructure sites.
  • ECCC developed a Departmental Climate Change Risk Assessment Guidance document in 2013. A workshop on this issue held in March 2013 was attended by representatives of 15 federal departments and agencies.
• Most key informants agree that the program elements have contributed to this outcome through such means as community-based risk assessments and community involvement in projects, identifying as examples PCA’s UCECCN and HC’s CCHAP. It was also noted by key informants that program elements contributed to this outcome by developing models, projections and scenarios that demonstrated how climate change would affect certain regions or types of infrastructure. As well, they mentioned that support was provided to communities and sectors to interpret the models and apply the information to local needs. For example:
  • ECCC’s CCPSP provides scenarios and downscaling tools through the Canadian Climate Change Data and Scenarios Network website (formerly known as the Canadian Climate Change Scenarios Network) and has received over 500,000 requests for climate scenarios data between July 2012 and March 2013.
  • PCA’s UCECCN conducted ecotype mapping in five Northern national parks in support of its Climate Change Vulnerability Assessment (CCVA) project.
• On average, roughly three-quarters of survey respondents rated both products and activities as having contributed to a large extent to communities and sectors assessing their risks and opportunities arising from climate change.

Immediate outcome 3: Adaptation measures have been identified to address risks and opportunities arising from climate change^{Footnote 18}  – Acceptable

Evidence reveals numerous specific examples of adaptation measures from a majority of the Theme’s relevant program elements.

• Program documents and performance data provide many examples of progress towards the achievement of this outcome. For example:
  • According to performance data for 2013-14 for PCA’s UCECCN, new monitoring measures for all northern national parks were identified. These new measures were incorporated into each park's long-term operational plan for ecological integrity condition monitoring and data acquired are intended to support future climate adaptation planning.
  • Documents and performance data for 2013-14 show that all of HC’s CCHAP’s projects have identified adaptation measures. Funded projects that identified relevant adaptation measures include, for example, capacity building and knowledge transfer projects and projects focussed on maintaining traditional activities and food security.
  • DFO’s ACCASP reported in 2014 that adaptation tools were being developed for use by program managers in 40% of DFO’s relevant program areas. As well, knowledge was being developed to support future adaptation efforts in 50% of DFO’s relevant program areas.
  • During the timeframe covered by the evaluation, SCC’s and INAC’s NISI was working on developing four new standards intended to guide infrastructure adaptations, with a fifth standard pending.
  • NRCan Adaptation Program projects conducted cost-benefit analyses of adaptation measures in coastal management and the mining sector.
  • The Municipal Water Sub-Project case study from INAC’s CARPAN found the project has identified adaptation measures and opportunities to enhance the water and wastewater infrastructure of Nunatsiavut communities.
  • CARPAN reported in 2013-14 that in the seven projects with completed adaptation plans, 14 communities have identified adaptation measures. One such project is the Climate Change Adaptation Planning within the Chippewas of Georgina Island First Nation Reserve.
• Most key informants agree that the program elements contribute to the identification of adaptation measures. Examples include tools to address chemical changes in fisheries; heat alert and response systems in municipalities and provinces; control methods for vector-borne diseases; community-specific adaptation plans; the Homeowners’ Guide to Permafrost; local food security strategies; infrastructure standards; and ventilation and snow-load management guides.
• Furthermore, a majority of survey respondents reported that program element products and activities have contributed to a large extent to the identification of measures to address climate change risks (86% for both products and activities) and opportunities arising from climate change (65% and 69% for products and activities, respectively).

Immediate outcome 4: Targeted communities and sectors are aware of relevant adaptation measures^{Footnote 19}  – Acceptable

Program elements are generally viewed by key informants and survey respondents to have contributed to awareness of relevant adaptation measures among targeted communities and sectors. Numerous examples of stakeholder outreach and knowledge transfer activities are available to support this view.

• A majority of key informants agree that the program elements contribute to this outcome. This view was generally based on having attended events or having been involved in activities that directly shared tools with communities and industry sectors. Key informants who were not of this view tended to be those who were familiar with program elements that were still developing tools, or had only recently developed them.
• According to key informant interview findings, efforts to increase awareness included communications and social media messaging, workshops, webinars, partnerships and committees and publications. Some specific examples include:
  • HC’s CCHAP’s distributed reports on adaptation measures to targeted communities and hosted Climate Change and Health Adaptation Workshops and knowledge sharing sessions for over 100 First Nation and Inuit community leaders and members from across the Arctic in the fall of 2013.
  • The NRCan Adaptation Program’s held numerous meetings and discussions with Northern and government stakeholders on ongoing research related to the mining sector.
  • Workshops were held as part of the work of PC’s UCECCN in eight northern communities for four northern national parks.
  • A heat and health accreditation course for healthcare professionals was developed by HC’s HARS, which hosted over 200 participants and implemented a distribution strategy to enhance the reach of materials for targeted stakeholders.
• There are also several examples of continuing efforts to raise awareness among targeted audiences that are in the early stages of development or implementation:
  • Case study findings indicated that PHAC’s PPHSACC’s Climate Change, Adaptation and Acute Gastrointestinal Illness in the Canadian North project is working toward identifying specific adaptation measures for northern communities.
  • In response to findings from the NTAI evaluation, TC has committed to finalizing a knowledge transfer plan to communicate the knowledge gained through the NTAI investments to target audiences.
  • The case study of NRCan’s Adaptation Program’s Forest Change Initiative revealed that additional outreach efforts were thought to be needed to bring the models and data developed to the attention of regional partners. Funds were allocated for such efforts to take place in 2015-16.^{Footnote 20}
• A majority of survey respondents reported that program element products (87%) and activities (85%) contributed to a large extent to increased awareness of adaptation measures.
• As discussed in section 4.2.1, more work is needed to increase awareness and engagement of stakeholders in relation to adaptation activities, to contribute more fully to communities and sectors being able to incorporate adaptation planning and activities and reduce vulnerabilities. However, in the context of the immediate outcomes of the program, acceptable progress has been made towards targeted communities and sectors being aware of relevant adaptation measures.

Immediate outcome 5: Increase collaboration on climate change adaptation^{Footnote 21}  – Acceptable

Program elements are generally viewed by key informants to have increased the collaboration among government, community and sector partners around climate change adaptation. Progress in this area is further supported by documented examples, performance data and survey results.

• A majority of key informants agree that the program elements have been effective in increasing collaboration on climate change adaptation. Key informants noted that collaborations have occurred among a variety of partners and stakeholders, including representatives of other government organizations (federal departments, provinces, territories, municipalities), industries, climate research organizations, universities and academics and First Nations and Inuit communities. Such collaborations ranged in complexity from sharing information to active collaboration on program design, priorities, needs assessments, research and tool development.
• NRCan’s Adaptation Platform is flagged by some key informants as a best practice in encouraging collaboration. It brought together over 200 members representing all provinces and territories, eight federal departments, four industries, numerous academics and several other organizations to develop tools and conduct research. At least 16 new collaborations were established between 2011‑12 and 2013-14, including four new working groups in 2013-14.
• TC’s NTAI established two networks of expertise that include stakeholders from provinces, territories, universities, colleges and, to a lesser extent, the private sector: the Network of Expertise on Transportation Infrastructure in Permafrost Regions and the Network of Expertise on Transportation in Arctic Waters. The recent evaluation of the NTAI concluded that these networks are effective venues for building collaboration between territorial governments and researchers.
• ECCC’s CCPSP supports participation in national and international climate science collaborations, as well as contributions to assessments on climate science and climate impacts and adaptation (domestic and international), and met with regional climate consortia to discuss the production and delivery of climate information.
• Through its HARS program, HC has been collaborating with ECCC’s Meteorological Service of Canada, the Ontario Ministry of Health and Long-Term Care, Public Health Ontario and Public Health Units to harmonize HARS across the province.
• Results from the case study of DFO’s ACCASP show that collaboration on the Coastal Infrastructure Vulnerability Index reduced barriers between experts in science, engineering, policy and economics within DFO. This internal collaboration is perceived to have facilitated communication, and the sharing of information and data.
• Case study results indicate that five of the 23 formal collaborations reported by PHAC’s PPHSACC in 2014 were a direct result of the Burden of Illness (BOI) survey project.^{Footnote 22} Local government and community stakeholders consulted throughout the project provided advice on the survey design. Ongoing communication with community members was undertaken to increase awareness of both the project’s purpose and of results-sharing events held in local communities.
• In addition to the examples of collaboration outlined above at the program element level, ECCC facilitated collaboration among federal partners involved in climate change adaptation through its work in chairing governance committees (for example, the Director General Adaptation Policy Steering Committee (DGAPSC) and FPT Adaptation Policy Committee, described in section 2.2).
• A majority of survey respondents indicated that program element products (83%) and activities (82%) contributed to a large extent to increased collaboration on climate change adaptation.
• As discussed in section 4.2.1, more work is needed to improve the design of the Adaptation Theme in order to maximize collaboration with existing and potential stakeholders in relation to adaptation activities, ultimately to contribute more fully to communities and sectors being able to address adaptation in their planning and activities and to reduce vulnerabilities. However, in the context of how the Theme was functioning during the evaluation period, acceptable progress was made towards increasing collaboration on climate change adaptation.

Intermediate outcome 1: Targeted communities and sectors address adaptation in their planning^{Footnote 23} – Acceptable

Some communities and sectors are making progress in addressing adaptation in their planning. Although progress to date has been modest, this is generally thought to be acceptable for the current stage of adaptation programming.

• Survey and key informant feedback suggests that stakeholders think program elements are contributing to this outcome. A majority of survey respondents reported that program element products (81%) and activities (79%) contributed to a large extent to communities and sectors addressing adaptation in their planning. Key informants also reported that program elements are contributing to this outcome at least to some extent, but a few also highlighted that it may be too early to tell and that tools to assess this outcome were not available. It is also acknowledged that while some planning is taking place, it cannot be fully attributed to program work.
• Some examples of adaptation being considered in community and sector planning include:
  • projects funded through HC’s CCHAP that developed community plans to improve food security and the sustainability of housing
  • a few early examples in which TC’s NTAI research has been taken into consideration in infrastructure design and adaptation
  • the development of a blueprint by 41 partner organizations (including local governments, provincial ministries, and owners of major transportation infrastructure) linked with NRCan’s Adaptation Program to adapt to catastrophic river and coastal flooding in British Columbia’s Lower Mainland
  • regions developing plans in response to projections of vector-borne diseases provided by PHAC’s PPHSACC
  • sectors taking the climate change models and projections provided by ECCC’s CCPSP into account when planning future infrastructure
• On a similar note, case study key informants for the NRCan Adaptation Program’s Forest Change Initiative project reported that it is too soon in the project timeline to observe intermediate outcomes.^{Footnote 24}  Case study findings for PPHSACC’s Climate Change, Adaptation and Acute Gastrointestinal Illness in the Canadian North Project show that although Iqaluit and Rigolet have not yet addressed adaptation in their planning, there has been progress towards identifying adaptation measures as a result of the project’s outputs. The outputs have provided Iqaluit and Rigolet with information and tools to inform adaptation planning, but case study key informants noted that community follow-ups would be needed to properly assess progress toward the intermediate outcomes.

Intermediate outcome 2: Targeted individuals, communities and sectors implement adaptation measures^{Footnote 25}  – Unable to assess

While implementation of some adaptation measures has occurred, progress is still in the early stages. As a result, it is too early to adequately conclude on the overall status of this outcome or whether progress to date is in keeping with expectations.

• The evaluation uncovered several documented or reported examples of adaptation measures having been implemented. Examples include:
  • HC’s HARS program successfully piloted heat alert and response systems in four communities and provincial heat alert and response systems that are operational or in development in Manitoba, Alberta and Ontario.
  • TC’s NTAI funded permafrost research related to groundwater and heat transfer, which supported the decision to pilot interceptor ditches to deflect groundwater along a limited section of the Alaska Highway.
  • The HARS program implemented a harmonization project in Ontario, with a pilot project involving public health units in the greater Toronto area during the Pan and Parapan American Games in the summer of 2015.
  • A thermosyphon system^{Footnote 26}  was implemented in the Iqaluit Airport as a result of work completed by SCC’s and INAC’s NISI.
  • A water filtration system was implemented by HC’s CCHAP to ensure that water remains clean during increased storm activity.
  • An ice monitoring network was established to monitor the stability of sea ice and was implemented with support from the CCHAP.
  • Risk maps were developed by PHAC’s PPHSACC for the tracking of Lyme disease, to inform public health decision making in provinces (tools for public health professionals).
• Overall, key informants provided different views regarding this outcome. Although some key informants indicated that the program elements have contributed to communities and sectors implementing adaptation measures, others suggested it was too early to tell. Moreover, a few key informants reported that much of the implementation of adaptation measures has not yet occurred and is only planned for the future.  Case study findings depict a similar picture. For instance:
  • The case study of PHAC’s PPHSACC Climate Change, Adaptation and Acute Gastrointestinal Illness in the Canadian North project found that while information and tools have been developed and provided, community follow-ups are thought to be needed in order to properly assess the degree to which communities are using them to implement adaptation measures.
  • The Municipal Water Sub-Project case study from INAC’s CARPAN found that although the project has developed decision-making and evaluation tools that can be used by local and regional governments in future planning and development endeavours, no evidence of the tools’ use was available at the time of data collection.
  • NRCan’s Adaptation Program’s Forest Change Initiative project case study revealed that case study key informants deemed it to be too soon in the project timeline to observe intermediate outcomes.
  • The DFO ACCASP Coastal Infrastructure Vulnerability Index (CIVI) project case study captured the general perception that it is too early in the development process to evaluate this outcome. Once the index is fully developed, however, the coastal assets of Small Craft Harbours will have a useful adaptation tool for coastal infrastructure.
• The survey found that only a slim majority thinks program element products (64%) and activities (59%) are contributing to a large extent to the implementation of adaptation measures.

Final outcome 1: Reduced vulnerability of individuals, communities, regions and economic sectors to the impacts of climate change^{Footnote 27}  – Unable to assess

Findings from the survey and key informant interviews indicated that program elements have contributed to reducing the vulnerability of individuals, communities, regions and sectors to the impacts of climate change. Some interviewees reported that it is too early to assess progress towards achieving this outcome,^{Footnote 28}  however, and examples generally relate to establishing the conditions to reduce vulnerability, rather than reducing vulnerability itself.

• A majority of survey respondents reported that program element products and activities reduce the vulnerability of communities (78% for products and 82% for activities), regions (78% and 64%, respectively) and individuals (73% and 74%, respectively) to a large extent. A majority (63%) also reported that program activities reduce the vulnerability of economic sectors to a large extent.
• Approximately half of key informants indicated that the program elements have contributed to reducing vulnerabilities. It is important to note that some key informants reported that there was not enough information to assess the program’s contribution to this outcome. As well, while key informants offered a number of examples of actions that should eventually lead to a reduced vulnerability, few actually demonstrate this result. Examples of the kinds of activities that are expected to reduce vulnerabilities include:
  • Raising awareness and increasing knowledge of the risks of climate change (for example, HC’s CCHAP raised awareness of the impacts of climate change in the North).
  • Involving communities and sectors in tool development, thereby better allowing the tools to be directed toward the areas of greatest vulnerability (for example, youth resiliency programs implemented under CCHAP in which youth learn culturally appropriate skills to live in a changing climate).
  • Identifying populations that are most vulnerable to the effects of climate change, such that regions and organizations can develop targeted responses (for example, PHAC’s PPHSACC identified the age groups at most risk for Lyme disease).
  • Developing standards that can be used to determine which infrastructure to invest in and how to prepare infrastructure to withstand the changes brought by climate change (for example, work undertaken by SCC’s and INAC’s NISI).
• Case study findings also suggest that it is too early to conclude on the question of reduced vulnerabilities. PHAC’s PPHSACC Climate Change, Adaptation and Acute Gastrointestinal Illness in the Canadian North project could not conclude regarding progress toward this outcome. Case study key informants noted that community follow-ups would be needed to properly assess progress in this regard. As well, key informants for INAC’s CARPAN Municipal Water Sub-project case study reported that use of the tools in future planning activities will eventually lead to reduced vulnerabilities. They also noted that it is too early in project timelines to conclude on the extent of progress toward this outcome.

Final outcome 2: Increased capacity of individuals, communities and economic sectors to adapt to climate change^{Footnote 29}  – Acceptable

Some progress had been made towards increasing the capacity of individuals, communities and sectors to adapt to climate change through participation in projects, knowledge sharing and enhancing research capacity.

• Documented examples of how the capacities of individuals, communities and sectors have been increased include:
  • HC’s CCHAP supported projects that built research facilities in First Nations and Inuit communities.
  • Through TC’s NTAI, capacity to adapt to climate change is being built among various stakeholder groups, including academic researchers, territorial and provincial policy managers and technical and engineering staff. Network meetings and workshops are noted as key venues for capacity building.
• A majority of key informants agree that the program elements contribute to increasing the capacities of individuals, communities and sectors to adapt to climate change, through such means as the provision of science-based information and tools for community and sector use (for example, risk modeling tools developed by PHAC’s PPHSACC; baseline data from PCA’s UCECCN to identify regional changes due to climate change).
• A majority of survey respondents indicated that program element products and activities contributed to a large extent to increasing individual (59% and 62%, respectively), community (63% and 64%, respectively), regional (60% and 65%, respectively) and the economic sector’s (74% and 75%, respectively) capacities to adapt to climate change.