Evaluation of Freshwater Action Plan: Great Lakes Protection Initiative

List of tables

Table 1: Budget and expenditures of the Great Lakes Protection Initiative (in $000s)

Acknowledgements

The Evaluation Project Team would like to thank the individuals who contributed to this project. In particular, we would like to acknowledge the contribution of the Great Lakes Protection Initiative representatives who provided input and comments to inform the findings and recommendations of this evaluation.

This report was prepared by the Evaluation Division of the Audit and Evaluation Branch, Environment and Climate Change Canada.

This report was approved by the Deputy Ministers on January 31, 2022.

List of acronyms and abbreviations

AAFC

Agriculture and Agri-Food Canada

DFO

Department of Fisheries and Oceans

ECCC

Environment and Climate Change Canada

FY

Fiscal year

FAP

Freshwater Action Plan

G&C

Grants and contributions

GLPI

Great Lakes Protection Initiative

GLWQA

Great Lakes Water Quality Agreement

PIP

Performance Information Profile

LWBP

Lake Winnipeg Basin Program

U.S.

United States

Executive summary

Context

This report presents an evaluation of Environment and Climate Change Canada (ECCC) activities supported by Freshwater Action Plan (FAP) funding for the Great Lakes Protection Initiative (GLPI).

Announced in Budget 2017, the FAP is funded at $70.5 million and provides a framework to advance ECCC’s programming to protect and restore freshwater quality in the Great Lakes and the Lake Winnipeg Basin from fiscal year (FY) 2017 to 2018 to FY 2021 to 2022. The funding is divided between the GLPI ($44.84 million) and the Lake Winnipeg Basin Program (LWBP) ($25.7 million). 3 major river basins (Fraser, MacKenzie and St. John) are also named in the FAP. Because they are funded using existing resources, they are not subject to evaluation. In the Great Lakes context, FAP funds are an addition to ongoing and existing ECCC resources for protection and restoration of the Great Lakes.

The GLPI investments focused on supporting a number of Canada’s commitments under the Canada-United States Great Lakes Water Quality Agreement (GLWQA), with the goal of improving Great Lakes water quality and ecosystem health in 6 particular program areas:

• preventing toxic and nuisance algae
• assessing and enhancing the resilience of Great Lakes and coastal wetlands
• evaluating and identifying at-risk nearshore waters
• reducing releases of harmful chemicals
• engaging Indigenous Peoples in addressing Great Lakes issues
• increasing public engagement through citizen science

Oversight and overall coordination of the GLPI activities are led by ECCC’s Ontario Regional Office of the Strategic Policy Branch. Other ECCC Branches are also involved in the delivery of the GLPI activities.

This evaluation covers the period between FY 2017 to 2018 and FY 2020 to 2021. Activities in 4 of the GLPI program areas, namely preventing toxic and nuisance algae, evaluating and identifying at-risk nearshore waters, reducing releases of harmful chemicals and engaging Indigenous Peoples in addressing Great Lakes issues – were included in the evaluation. In addition, this evaluation examined collaborative governance related to these program areas. The evaluation was conducted concurrently with an evaluation of the LWBP.

Findings and conclusions

Collaborative governance

The evaluation found that the GLPI was aligned with the priorities of the GLWQA and supported the participation of Canadians and Indigenous people in GLWQA governance mechanisms.

To prevent duplication between the governance of the GLPI-funded program areas and ECCC’s existing and ongoing funded program areas for the Great Lakes, the Department integrated the GLPI program areas into its existing governance structures, including the process for administering grants and contributions (G&C). The GLPI benefited from collaborations with many governments and non-government stakeholders for monitoring and restoration activities. Differences in mandates (for example, environmental vs. economy), organizational silos and shifts in environmental policies (at the provincial level) have been barriers that hinder collaboration among some stakeholders. There are opportunities to strengthen the collaboration.

Efficiency: Use of Freshwater Action Plan resources

The GLPI received a $44.84 million budget allocation under the FAP. According to the findings, the GLPI underspent its budget for the period covered by the evaluation, mostly due to delays in staffing. With respect to project management, systems are in place to ensure efficient management of program activities and G&Cs. The GLPI G&Cs administration is embedded in the Department’s G&Cs management system, allowing for an economy of scale.

Collaborations with other stakeholders also led to an efficient use of resources. For instance, environmental data related to the GLPI program areas is provided by provincial and non-government organizations. Overall, ECCC’s GLPI G&Cs have leveraged external funding sources, but at a lower ratio than LWBP G&Cs. For the G&Cs related to preventing toxic and nuisance algae, ECCC’s GLPI G&Cs leveraged $2.13 of external funding (cash or in-kind in comparison with $2.30 for the LWBP. It’s important to note, however, that unlike the LWBP, the GLPI does not require its recipients to secure additional sources of funding.

Use of performance information

The GLPI results are reported against ECCC’s Water Quality and Ecosystem Partnership Performance Information Profile (PIP) indicators and contribute to the GLWQA reporting requirements. Data are gathered from multiple sources, including data from ECCC’s Great Lakes Surveillance Program (in cooperation with the Province of Ontario and local organizations). At the G&Cs level, results are tracked via a reporting form completed by recipients, which is mostly in narrative form. However, there are opportunities to improve G&Cs recipient reporting.

Preventing toxic and nuisance algae

Excess loadings of nutrients, such as phosphorus, are a significant concern, especially in Lake Erie. While the 2019 Canada-United States State of the Great Lakes Report found the status of nutrient concentrations to be “fair,” the situation in Lake Erie is deemed critical. Excess phosphorus loads to the lake from land-based sources in Canada and the U.S. have resulted in significant toxic and nuisance algae blooms. Pursuant to commitments in the GLWQA, Canada and the U.S. have agreed to phosphorus reduction loading targets for the lake. Canada is committed to reducing its loads from Canadian sources by a total of 212 tonnes/year. The GLPI investments improved monitoring and understanding of nutrients and algal blooms. The GLPI also funded projects that reduced phosphorus loadings from Canadian sources by a total of 20 tonnes/year. Despite these GLPI successes, it will take many years of significant concerted effort by all partners to achieve Canada’s phosphorus loading reduction target for Lake Erie.

Evaluating and identifying at-risk nearshore waters

Pursuant to a commitment in the GLWQA, Canada and the U.S. established a Nearshore Assessment Framework in 2016. Using the GLPI resources, ECCC implemented the Nearshore Assessment Framework in the Canadian nearshore waters of the Great Lakes. Through collaborations with many partners, an ECCC team coordinated the delineation and classification of the waters in the nearshore areas of Lake Erie and Lake Ontario (work on other lakes was ongoing at the time of the evaluation). The conditions of these nearshore areas were also assessed to identify areas of high ecological value and those under high stress. The approach used was deemed as both efficient and effective. 1 federal department and some local communities have implemented actions based on the nearshore assessment results. The assessment of the ecological value component has not been completed but will be done for the final integrated report.

Reducing releases of harmful chemicals

Toxic chemicals are 1 of 9 overarching indicators of ecosystem health in the Great Lakes identified in the GLWQA, and are reported on in the State of the Great Lakes Reports. The 2019 Report shows the status of toxic chemicals as “fair” and the trend as “unchanging to improving.” The GLPI investments for addressing the chemicals of mutual concern to both Canada and the U.S. have been modest as there are other funded programs in this area, namely Canada’s Chemicals Management Plan. The GLPI funded, through G&Cs, the development and piloting of innovative approaches to reduce releases of harmful pollutants to the Great Lakes. The work focused on areas not addressed by other ECCC programs, and progress is on track.

Engaging Indigenous peoples in addressing Great Lakes issues

The GLPI supported participation of Indigenous peoples in the GLWQA governance structures and provided support to projects led by Indigenous governments and communities, including awareness-raising projects, monitoring activities and restoration work. While it is too early to assess the impacts of these projects, it was recognized by most Indigenous key informants that they have had a significant positive impact on capacity building.

Overall, Indigenous key informants acknowledged efforts by ECCC to engage with Indigenous governments and community representatives in the development of the new GLPI G&C programming before launching the initial call for proposals. These efforts supported Indigenous capacity to restore and protect the Great Lakes. It was found that eligible projects did not encompass all of the environmental priorities of Indigenous organizations and governments, such as projects about biodiversity. The application process was also deemed overly complex, especially for smaller organizations or communities that need support to apply for funding. The evaluation also found that best practices used in other G&C programs could be considered by the GLPI.

Recommendations

2 recommendations are directed to ECCC’s Assistant Deputy Minister (ADM) of Strategic Policy Branch, as the senior departmental official responsible for the GLPI.

Recommendation 1: identify and implement best practices to enhance and strengthen Indigenous engagement and participation in the Great Lakes Protection Initiative’s G&C programs and projects.

Efforts by ECCC to engage with Indigenous governments and communities have been positively recognized. Through G&Cs, the GLPI provided financial support to projects led by Indigenous governments and communities. Support was also provided to enhance participation of Indigenous peoples in binational and bilateral management tables and subcommittees. While there are notable accomplishments, there remain opportunities to continue strengthening ECCC’s capacity to foster Indigenous engagement. For example, the evaluation found an opportunity for ECCC toprovide more support to G&Cs applicants from Indigenous communities. Continued training and education for ECCC staff on Treaty Rights and cultural awareness was also deemed to be essential. Finally, there is also an opportunity to further engage Indigenous participation at the onset of the application design process in order to inform selection criteria.

Recommendation 2: review G&Cs project monitoring tools to further incorporate quantitative performance indicators where applicable.

The current reporting forms for the GLPI G&C projects mostly include open text fields that are not conducive to consistent and quantitative measures of the expected outputs and impacts of projects. For consistency and improved reporting, it is recommended that the GLPI review its G&Cs reporting strategy to introduce quantitative indicators in its G&Cs reporting forms where applicable.

1. Context

Announced in Budget 2017, the FAP is a framework to advance ECCC programming to protect and restore freshwater quality in the Great Lakes and the Lake Winnipeg basins. The FAP includes funding for both basins amounting to $70.5 million from FY 2017 to 2018 to FY 2021 to 2022, with $44.84 million allocated to the GLPI, and $25.7 million allocated to the LWBP. In the Great Lakes context, the FAP funds are in addition to ongoing and existing ECCC resources for the protection and restoration of the lakes.

This evaluation report presents findings related to ECCC activities supported by the FAP funding for the GLPI for the period between FY 2017 to 2018 and FY 2020 to 2021, related to 4 program areas: preventing toxic and nuisance algae, evaluating and identifying at-risk nearshore waters, reducing releases of harmful chemicals and engaging Indigenous Peoples in addressing Great Lakes issues. This evaluation also examined collaborative governance related to these program areas. The evaluation was conducted concurrently with an evaluation of the LWBP. The evaluation examined 2 issues:

• Effectiveness – the extent to which the GLPI achieved objectives related to preventing toxic and nuisance algae, evaluating and identifying at-risk nearshore waters, reducing releases of harmful chemicals, and engaging Indigenous peoples in addressing Great Lakes issues
• Efficiency – the extent to which activities are governed collaboratively, resources have been used efficiently, and performance information was used to inform decision making

For the purposes of this evaluation, different methodologies were applied, including a review of documents and files, an analysis of financial data and 17 key informant interviews (including 12 with representatives from ECCC and 5 from provincial and other federal organizations). The evaluation was also based on 2 case studies, 1 on Indigenous engagement activities and 1 on nearshore assessment activities. These cases studies included a total of 4 interviews with ECCC staff and 9 interviews with external informants (that is, Indigenous, other federal departments, and U.S. representatives).

Appendix A provides more details on the evaluation approach.

1.1 The Freshwater Action Plan

Since 2015, the mandate letters for the Minister of Environment and Climate Changehave identified the protection and stewardship of freshwater resources, in collaboration with other levels of government, and the protection of the Great Lakes and the Lake Winnipegbasins as priorities.

Announced in Budget 2017, the FAP is funded at $70.5 million and provides a framework to advance ECCC’s programming to protect and restore freshwater quality in the Great Lakes and the Lake Winnipeg Basin from FY 2017 to 2018 to FY 2021 to 2022. The funding is divided between the GLPI ($44.84 million) and the LWBP ($25.7 million). 3 major river basins (Fraser, MacKenzie and St. John’s) are also named in the FAP but are funded using existing resources and are not subject to evaluation. In the Great Lakes context, FAP funds are an addition to ongoing and existing ECCC resources for protection and restoration of the Great Lakes.

The GLPI focused on supporting a number of Canada’s commitments under the Canada-U.S. GLWQA, with the goal of improving Great Lakes water quality and ecosystem health in 6 program areas:

• preventing toxic and nuisance algae
• assessing and enhancing the resilience of Great Lakes and coastal wetlands
• evaluating and identifying at risk nearshore waters
• reducing releases of harmful chemicals
• engaging Indigenous Peoples in addressing Great Lakes issues
• increasing public engagement through citizen science

The FAP is under ECCC Core Responsibility Preventing and Managing Pollution, and contributes to the following departmental results:

• Canadians have clean water
• The Canadian environment is protected from harmful substances
• Canadian communities, economies and ecosystems are more resilient

1.2 Freshwater Action Plan: Great Lakes

For the Great Lakes, the FAP investments were brought under the umbrella term, the Great Lakes Protection Initiative (GLPI). The new resources focused on supporting Canada’s commitments under the Canada-U.S. GLWQA, with the goal of improving Great Lakes water quality and ecosystem health in the following 6 program areas.

Preventing toxic and nuisance algae. Under this area, ECCC was expected to support Canada’s commitments under the GLWQA related to reducing phosphorus loads to Lake Erie from Canadian sources, including:

• the development and implementation of watershed management strategies for nutrient reduction
• science to improve understanding and reporting on progress
• stakeholder actions to reduce nutrient pollution entering Lake Erie

Stemming from a commitment in the GLWQA, the activities were expected to form the federal contribution to the Canada-Ontario Lake Erie Action Plan, whose aim is to reduce phosphorus loading to the lake.

Assessing and enhancing the resilience of Great Lakes coastal wetlands. Recognizing that wetlands are important for preserving biodiversity in the Great Lakes, under this program area and in respect of commitments in the GLWQA, ECCC was expected to:

• conduct science activities to identify the status of coastal wetlands
• assess their vulnerability to projected ecosystem changes
• identify adaptation approaches to enhance resilience and build consensus on priorities for action

Reducing releases of harmful chemicals. In 2016, Canada and the U.S. designated 8 harmful chemicals as chemicals of mutual concern under the GLWQA. ECCC was expected to support the development and implementation of reduction strategies for these chemicals of mutual concern and to engage partners and stakeholders in identifying and implementing innovative approaches.

Evaluating and identifying at-risk nearshore waters. In this program area, ECCC was expected to implement the nearshore assessment framework that was developed under the GLWQA by Canada and the U.S. in 2016, in order to produce the first binational comprehensive assessment of the cumulative environmental effects on Great Lakes nearshore waters. This was to comprise the identification of areas of high ecological value and those under high stress. Results were to be used to identify priorities for action by all levels of government, stakeholders and the public.

Engaging Indigenous peoples in addressing Great Lakes issues. ECCC was expected to support enhanced participation of Indigenous peoples in binational and bilateral management tables and to support community projects that increase Great Lakes awareness and expertise.

Increasing public engagement through citizen science. In this area, ECCC was expected to increase knowledge and participation of Canadians in addressing threats to the Great Lakes by engaging them in citizen science initiatives (for example, water quality monitoring). These activities were expected to have the added benefit of providing useful, low-cost data to support ECCC science.

Oversight of the GLPI activities falls under the Ontario Regional Director General’s Office, which is part of ECCC’s Strategic Policy Branch. Other ECCC Branches are involved in the delivery of the GLPI activities, including the Science and Technology Branch, Canadian Wildlife Service, and Meteorological Service of Canada.

A large number and variety of collaborators and stakeholders are involved in addressing water quality issues in the Great Lakes Basin across program areas funded with the GLPI, as well as program areas funded with ECCC’s existing and ongoing resources. These include, but are not limited to other government departments such as Agriculture and Agri-food Canada (AAFC), Fisheries and Oceans Canada (DFO), Natural Resources Canada, and Global Affairs Canada); the Province of Ontario; the U.S. government; Indigenous peoples and governments (for example, Chiefs of Ontario and Metis Nation of Ontario); transboundary groups (for example, International Joint Commission); scientific and research communities; and non-governmental organizations (for example, conservation authorities and watershed management agencies).

2. Findings

2.1 Progress made on collaborative governance

As recognized in the GLWQA, no single government or agency has the ability to achieve the Agreement commitments alone. Involvement of government and non-governmental organizations is deemed essential. In this context, the GLPI was to be implemented through cross-government collaboration on improving waterquality, biodiversity conservation and sustainable use. The work conducted under the GLPI was also to be aligned with the priorities identified by the GLWQA (see text box).

According to some ECCC key informants, the work conducted by the GLWQA annex subcommittees (governance structures) brought into the fold a large number and variety of other players that have important roles. The GLPI funding was used to support the participation of some of those key players, including Indigenous groups and governments. For example, since 2017, Chiefs of Ontario and Métis Nation of Ontario have received the GLPI funding to support their participation in governance meetings related to the Canada-Ontario Agreement on Great Lakes Water Quality and Ecosystem Health and the GLWQA. The multi-year funding has enabled these organizations to dedicate staff to consistently attend meetings and communicate with communities regarding Great Lakes issues.

Most ECCC key informants agreed that supporting Indigenous representation at governance forums (either for large-scale agreements or local or regional initiatives) was essential to ensure that Indigenous needs, priorities, insights and perspectives are included in discussions and decision-making. However, some key informants from both ECCC and Indigenous communities mentioned that Indigenous governments and communities do not have the same extent of capacity and resources as federal departments like ECCC. Indigenous respondents explained that the federal government expectations with respect to involvement of First Nations and Métis communities can be unrealistic.

Most key informants agreed that there was good participation in various governance committees, including at the GLWQA subcommittee level. According to most key informants, there was a broad agreement on the GLWQA priorities. These include priorities related to chemicals of mutual concern and nutrients which are covered under the FAP resources for the GLPI. For nutrients, for example, it was mentioned that there are specific targets for phosphorous loading reductions to Lake Erie from Canadian sources that had been committed to by Canada, and that a Canada-Ontario implementation team had been established to coordinate actions to achieve those results.

To prevent duplication between governance of the GLPI and existing Great Lakes programming, ECCC integrated the GLPI program areas into existing governance structures, including the process for administering G&Cs. All interviews and case study key informants indicated that ECCC collaborates with many governments and organizations, including federal government departments,provincial government ministries, Indigenous peoples and governments, academia and non-government organizations, including conservation authorities and associations. These collaborations allow for extensive monitoring and restoration activities. For example, efforts to evaluate and identify at-risk nearshore waters were completed as a result of a multiparty team, including ECCC staff, government of Ontario staff (Ministry of Natural Resources and Forestry), the Canadian hydrographic service and DFO staff. Some U.S. agencies representatives (for example, the Environmental Protection Agency) were also involved. Collaboration between the various stakeholders involved was considered a significant success factor by most key informants.

Despite these collaborations, most ECCC respondents identified factors that have created barriers to good collaborations. In particular, differences in mandates (for example, environmental vs. economy), organizational silos and shifts in environmental policies at the provincial level have been barriers to close collaborations among some stakeholders. As well, according to ECCC key informants, there is an opportunity for more collaborative work between the senior managers of federal departments, as well as between the landscape science community (including agriculture) and the water science community (ECCC and DFO).

According to most ECCC respondents, programming to prevent toxic and nuisance algae by reducing phosphorus loadings is not well coordinated between departments. However, AAFC technical staff reviewed and commented on agriculture-related G&C proposals received through ECCC’s GLPI calls for proposals as part of the technical review process. These efforts, in addition to their participation in the established governance processes mentioned above, ensure that ECCC’s G&C’s are focused on phosphorus reduction priorities and are technically feasible.

ECCC key informants said that improved coordination between departments would increase harmonization of decisions to ensure that the best projects obtain support.

There are also departments that fund infrastructure projects that impact nutrients (for example, Infrastructure Canada and Federal Economic Development Agency for Southern Ontario). ECCC was not involved in the design phases of these projects. Involvement of ECCC could help reduce the environmental impacts of these projects, according to ECCC respondents.

2.2 Use of Freshwater Action Plan resources

As shown in Table 1, the GLPI had a budget of $22.7 million for the period from FY 2017 to 2018 to FY 2019 to 2020 (excluding corporate services and employee benefits). Actual expenditures were $21.3 million. The program has underspent its budget (6%), mostly in salaries and O&M. According to program representatives, underspending in salaries occurred mostly in the first year of the GLPI, and was due to delays in staffing positions.

Table 1: Budget and expenditures of the Great Lakes Protection Initiative (in $000s)

Category	Budget	FY 2017 to 2018	FY 2018 to 2019	FY 2019 to 2020	Total
FTE	Planned	26.8	31.8	31.5	N/A
FTE	Actual	29.0	40.1	39.9	N/A
Salaries	Budget	$2,425	$2,912	$2,742	$8,079
Salaries	Expenditure	$1,859	$2,679	$2,887	$7,425
Salaries	Variance (Diff.)	N/A	N/A	N/A	$654
Salaries	Variance (%)	N/A	N/A	N/A	8.1%
O&M	Budget	$3,669	$3,486	$2,126	$9,282
O&M	Expenditure	$3,306	$3,422	$2,011	$8,739
O&M	Variance (Diff.)	N/A	N/A	N/A	$542
O&M	Variance (%)	N/A	N/A	N/A	5.8%
Capital	Budget	$244	$40	0	$284
Capital	Expenditure	$259	0	0	$259
Capital	Variance (Diff.)	N/A	N/A	N/A	$24
Capital	Variance (%)	N/A	N/A	N/A	8.6%
G&Cs	Budget	$695	$2,064	$2,125	$4,884
G&Cs	Expenditure	$645	$2,064	$2,125	$4,834
G&Cs	Variance (Diff.)	N/A	N/A	N/A	$50
G&Cs	Variance (%)	N/A	N/A	N/A	1.0%
Total	Budget	$7,033	$8,502	$6,993	$22,529
Total	Expenditure	$6,069	$8,166	$7,023	$21,259
Total	Variance (Diff.)	N/A	N/A	N/A	$1,270
Total	Variance (%)	N/A	N/A	N/A	5.6%

ECCC key informants provided explanations about why they thought funds were well used. With respect to G&C funding, systems are in place to ensure efficient administration of funding. Projects are assessed and scored against clear evaluation criteria; they are prioritized, with backups; and there are challenge function meetings with management to ensure recommended projects are aligned with program objectives/expected results. Projects are on scope, on budget and on schedule, based on the reporting system indicators.

The administration of the GLPI G&C resources was embedded into ECCC’s existing G&Cs process, which made its delivery efficient. The review of the proposals involves technical review from scientists of various ECCC branches (for example, Science and Technology Branch and Environmental Protection Branch) depending on the nature of the project.

G&Cs activities that are delivered by recipients are also deemed efficient, according to ECCC key informants. For example, water sampling done by partners serves many purposes. Activities of recipients are in line with the science activities of the program. The file review indicates that most activities have been completed as planned, and most projects reported that they are sustainable over time (that is, project activities will continue). A few recipients indicated that some of their activities were cancelled due to the COVID-19 pandemic.

The effort to evaluate and identify at-risk nearshore waters was also deemed efficient mostly because the teams made extensive use of existing information and worked collaboratively with the provincial government, other federal departments and local organizations to gather the data. The ECCC team was also considered well managed: a stable ECCC team was assembled for the purposes of the program, and work plans were developed to keep the project on track.

Calculations based on final reports provided by recipients show that for every FAP dollar spent by ECCC on the GLPI G&C projects, $0.25 was contributed by another stakeholder (cash and in-kind). For GLPI G&C projects related to preventing toxic and nuisance algae $2.13 was leveraged from external organizations (cash and in-kind) for each dollar spent by ECCC. In comparison, the leveraging for LWBP was $2.30 for each ECCC $1. It is important to note, however, that contrary to the LWBP, there is no requirement for recipients to provide additional sources of funding under the GLPI. Program representatives explained that a deliberate decision was made at the early stages of the GLPI not to require applicants to contribute a specific percentage of cash and in-kind support. The rationale was to ensure that a good proposal or project (aligned with the GLPI priorities and objectives) would not be rejected for funding because it did not meet the leveraging target. However, applicants are encouraged to identify any additional partner funding or in-kind project support for their projects. Applications demonstrating other sources of financial or in-kind support are scored favourably when evaluated on value for money.

2.3 Use of performance information

The GLPI activities and results contribute to the delivery of key commitments under the GLWQA, including the development of a triennial binational Progress Report of the Parties and SOGL Reports (highlights and technical reports), as well as the delivery of a triennial Great Lakes Public Forum, to which interested organizations and the public are invited. Results are presented against the GLWQA targets. Each of the GLPI program areas supported has specific performance measures that align with the GLWQA and the ECCC Water Quality and Ecosystem Partnerships PIP under which the FAP is situated. Performance indicators for the GLPI contained in the PIP reflect its key program areas, including total annual reduction (in tonnes) of phosphorus loads from Canadian sources to Lake Erie, number of projects that reduce releases of harmful chemicals, percentage of Canadian nearshore areas assessed and the number of Indigenous organizations and communities participating in Great Lakes decision-making processes and / or projects that restore and protect the Great Lakes. These indicators are aligned with direct, intermediate and final outcomes of the PIP logic model. These indicators are tracked using multiple sources, including data from ECCC’s Great Lakes Surveillance Program (in cooperation with the Province of Ontario and local organizations), and program files.

Reports are based on monitoring activities. For example, nutrient information (phosphorus levels) was collected with the assistance of program partners, and compared against what was targeted. Reports on algae levels are high profile and are published annually. Progress was reported against the 7 GLPI objectives funded through the FAP and expected deliverables were communicated to ECCC senior management on a regular basis.

Recipients of G&Cs are required to complete a reporting form provided by ECCC. In the form, recipients are required to describe results (in narrative form) against planned activities. In addition, recipients are asked to identify challenges they faced, and why some results were not achieved (if that was the case), and why their work plan was changed, if applicable. Recipients are also asked to elaborate on lessons learned, and on whether or not the activities launched through this project will be continued in the future.

Key informants from ECCC said that they were satisfied with the reporting processes and that performance information was used by managers. However, the document review shows that the GLPI G&Cs reporting could be enhanced with more quantitative indicators such as those used for the LWBP. The LWBP G&C reporting form includes precise indicators that measure project impact that can be used to inform ECCC monitoring and reporting processes.

External key informants said that there are opportunities to improve the Sate of the Great Lakes reporting, including narratives of activities and achievements of community groups and non-profit organizations. There could also be more transparency about the models used to report on certain performance indicators (for example, drainage after storms), including the assumptions.

2.4 Preventing toxic and nuisance algae

Maintaining healthy levels of phosphorus is an important part of protecting lakes and rivers. Phosphorus levels that are too high or too low can have harmful impacts on a lake's food web. When phosphorus levels in water become too high, aquatic plant growth can become excessive and harmful. The decay of excess plant material can reduce the amount of oxygen available for fish and other aquatic animals. High nutrient levels can also lead to harmful algal blooms that can kill wildlife that live in or use the water, and affect human health. Conversely, too little phosphorus can result in not enough plant or algal growth to support a lake's food web, which could reduce fish populations and harm local fisheries. In 2019, phosphorus levels were too high in the offshore waters of Lake Erie, too low in the offshore waters of Lake Ontario, Lake Huron and Georgian Bay, and at the right level in the offshore waters of Lake Superior.

Phosphorus loadings in Lake Erie come mainly from “non-point” sources such as agriculture and urban storm water runoff. In 2018, these accounted for 77% of loadings. Other sources include atmospheric deposition, “point sources” such as wastewater treatment plants and industrial effluent, and input from Lake Huron. Algal blooms in Lake Erie are estimated to cause annual costs of $272 million (in 2015 Canadian dollars) to the Canadian economy over a 30‑year period if left unchecked.

Through the GLWQA, Canada and the U.S. have agreed to reduce phosphorus levels entering the western and central basins of Lake Erie by 40% from 2008 levels to decrease the extent of harmful and nuisance algal blooms and zones of depleted oxygen. In 2008, an estimated 9,518 tonnes of phosphorus was entering the western and central basin of Lake Erie. To reduce this by 40%, Canada and the United States agreed to an annual target load of 6,000 tonnes of phosphorus entering Lake Erie in these basins. For Canada, this means a reduction of 212 tonnes of phosphorus loads per year.

According to documentation, nutrients and algae was 1 of 9 overarching indicators of ecosystem health reported in the State of the Great Lakes Reports. The 2019 highlights report shows the status of nutrients and algae in the Great Lakes as fair and the trend as unchanging. This was an improvement from 2017, when the status of nutrients and algae was unchanging to deteriorating. For Lake Erie specifically, the 2019 report shows that the status was poor and unchanging-deteriorating. Between 2008 and 2018, phosphorus input from point sources (for example, industrial plants) has decreased while it has increased from non-point sources (for example, agriculture and urban storm water runoff) in both Canada and the U.S.

The Government of Canada is implementing the Canada-Ontario Lake Erie Action Plan (released in 2018) to reduce phosphorus loads from Canadian sources and address harmful algal blooms and zones of depleted oxygen in Lake Erie. The plan contains more than 120 actions that Canada, Ontario, and their partners will take to work towards achieving the Canada-US binational phosphorus reduction targets for the lake.

All key informants agreed that the GLPI investments supported efforts to reduce nutrient loads in the Great Lakes. For example, investments in research will allow for better monitoring of the loads, that is, a better understanding of the high-load areas, with a focus on 3 watersheds in Lake Erie. Funds are also used to monitor, understand and predict factors causing bacterial blooms, including nutrients, climate change and wildlife (for example, role of mussels).

In addition to ECCC science activities, the GLPI also provided G&Cs for on-the-ground action led by partners, such as Conservation Authorities. Projects focused on the demonstration of innovative approaches or best management practices to reduce phosphorus loadings or on the development of technologies to reduce loadings, such as devices and processes to remove phosphorus from manure and wastewater. The resulting compounds from these processes can then be used as slow-release fertilizer which if applied properly can reduce loadings. Funded partners also evaluated the effectiveness of their projects through the monitoring of phosphorus load reductions. These projects are expected to have impacts in the short and medium terms. Unfortunately, according to some ECCC key informants, some delays occurred due to differing perspectives on priorities between the federal government and the provincial government; however, all projects have been or are expected to be completed on schedule.

With respect to the impact on actual loadings, program information shows that for the period of 2017 to 2020, projects funded by ECCC through the GLPI are expected to reduce Canadian sources of phosphorus by an estimated 20 tonnes per year by 2022, against Canada’s target reduction of 212 metric tons (annually, including permanent and non-permanent reductions). This data indicates that while the GLPI funding program is achieving its objectives, significant attention is still required by all partners to address this area of joint responsibility to achieve Canada’s commitments for the lake. According to 1 key informant and documentation reviewed, agriculture practices also need to change to significantly impact loadings. For example, the spreading of manure during the winter months is still permitted in some conditions (although not recommended^{Footnote 1}), despite the impacts on nutrient loadings.^{Footnote 2}  Some ECCC key informants said that the resources are insufficient to meet the loading reduction targets and that reaching them will take decades. Some said that ECCC funding for nutrients was fairly limited and that significant efforts will be needed to reach stakeholders in the agriculture sector to have an impact. 1 ECCC respondent also explained that, since the targets had been set, some factors have changed and those are accelerating nutrient loadings. For instance, research shows that climate change has led to increased rainfall in the spring and run off of nutrients into the Great Lakes^{Footnote 3 ,Footnote 4}. Phosphorous releases from lake-bottom sediments also increase as the temperature of the lake rises, a phenomenon that will increase if climate change continues to increase Lake Erie’s temperature.^{Footnote 5} 

2.5 Evaluating and identifying at-risk nearshore waters

Using the GLPI resources ECCC was to implement the Nearshore Assessment Framework developed by Canada and the U.S. in 2016. This work was to include the identification of areas of high ecological value and those under high stress. Results were expected to be used to identify priorities for action by all levels of government, stakeholders and the public.

A special team at ECCC was dedicated to coordinate the assessment. The work was completed as a result of a multiparty team, including ECCC staff, government of Ontario staff (Ministry of Natural Resources and Forestry), the Canadian Hydrographic Service and DFO staff. Some representatives from U.S. agencies (for example, Environmental Protection Agency) and consultants were also involved. There was also a group of advisors that participated in the development of the approach.

As per the Framework, the Canadian side of the Great Lakes was delineated and classified into units. External key informants noted that the resolution of the delineation map (units) was appropriate. The choice of units was well grounded in science, reflected the physical properties of the areas, and based on need. Key informants noted that it will be conducive to action.

The exercise also involved assessments of the areas. This includes determining the condition of each unit, including water quality (for example, pH and dissolved oxygen levels), water clarity, aquatic vegetation composition, sediment condition and benthic communities, among other factors, compared to thresholds. The assessments were completed for Lake Erie and Lake Ontario and 2 highlights report have been produced. Assessment of Lake Huron is expected to be completed in fall 2021. An overall integrated report was to be produced once all 4 are produced.

According to most key informants, 1 of the strengths of the approach chosen was that there was an intentional effort to focus on pre-existing information, from ECCC and other sources, to develop the delineation and conduct the assessments. It also reflected the main usages and associated them with each delineated section. The Canadian work differed from the U.S. approach which was more based on statistical sampling principles (versus purposive sampling).

Most key informants, both internal and external, said that the work done was of good quality, despite data limitations and the vast amount of data that was available to analyze. While the reports to date identify areas under high stress, the ecological value component has not been completed and will be done for the final integrated report. It was explained that further work is required to validate the approach to be used to identify these areas. For example, criteria will need to consider what is of value from a social perspective vs. a wildlife perspective.

While completed, the Lake Erie and Lake Ontario reports have not been widely distributed yet. Communications services at ECCC experienced delays due to COVID-19-related restrictions. The assessment results were presented to a number of stakeholders and community groups. As a result of these presentations, a local group in the Niagara region formed a collaborative to take action. The members committed to develop shared solutions and implement actions towards improving coastal health. According to an ECCC key informant, the collaborative managed to secure funding to support action.

DFO will also use the information to inform actions on habitat protection/restoration. Data will also be used to identify navigation routes. The Lake Erie assessment was included in the Canada-U.S. Lake Erie Lakewide Action and Management Plan (LAMP). The Lake Ontario Assessment will not be included in the Canada-U.S. Lake Ontario LAMP due to delays on the U.S. side.

2.6 Reducing releases of harmful chemicals

Toxic chemicals are 1 of 9 overarching indicators of ecosystem health reported in the State o the Great Lakes Reports. The 2019 report showed the status of toxic chemicals as “fair” and the trend as “unchanging to improving”. Concentrations of many chemicals have declined significantly in the Great Lakes. However, concentrations of some toxic chemicals still pose threats to human health and the environment. Concentrations of some compounds, including polychlorinated biphenyl (PCB), still exceed ecosystem-based objectives.

According to some ECCC key informants, the GLPI investments to address chemicals of mutual concern have been modest, given other ECCC programming in this area, but the work and progress were on track. ECCC addresses toxic substances through the national Chemicals Management Plan. The GLPI efforts have involved assessing what needs to be done in the Great Lakes and that has not been initiated through national programs. According to some ECCC key informants, the GLPI team was very well coordinated with the Chemicals Management Plan and addressed gaps. Specifically, according to some key informants, the GLPI helped identify what substances are present in the Great Lakes, in what products and at what levels. Priorities have been identified for actions to reduce legacy toxins (for example, PCBs) as well as emerging ones (for example, polybrominated diphenyl ethers and short-chain chlorinated paraffin). This information will address gaps in the national programs, according to some key informants.

The GLPI funded, through G&Cs, the development and piloting of innovative approaches to reduce releases of chemicals of mutual concern, which continue to enter the Great Lakes despite being heavily regulated in Canada. For example, 1 project measured the effectiveness of filters installed on household washing machines and clothes dryers in collecting certain toxic substances.

2.7 Engaging Indigenous peoples in addressing Great Lakes issues

According to all Indigenous key informants, the Great Lakes are of major significance to Indigenous peoples whose traditional territories are connected to the Lakes. The First Nations and Métis people feel a strong sense of responsibility and stewardship towards the Great Lakes. Several communities are worried about the quality of their drinking water and the impact of poor water quality on food consumption (for example, accumulation of toxins in fish).

Through G&Cs, the GLPI provided financial support to projects led by Indigenous organizations, governments and communities. Support was also provided to enhance participation of Indigenous peoples in binational and bilateral management tables and subcommittees that support the implementation of the GLWQA and the Canada-Ontario Agreement on Great Lakes Water Quality and Ecosystem Health.

With respect to funding, the file review indicates that the GLPI funded 13 projects (to date) under the program area of engaging Indigenous Peoples in addressing Great Lakes issues. 5 projects were completed and the remainder are ongoing until 2022. Most projects included a component of direct community engagement, where project proponents connect with, inform and involve community members. This included festivals and clean-up work with members of the community. Some projects involved research and sample collecting. According to some Indigenous key informants, projects involved a diversity of participants, and community activities were meant to be as inclusive as possible. Indigenous key informants explained that project proponents interviewed elders and community members who could speak to the past and current state of the Great Lakes in order to gather and preserve Indigenous knowledge. Several of the projects also involved youth.

Some ECCC and Indigenous key informants mentioned that the COVID-19 pandemic impacted all projects in FY 2020 to 2021 due to the cancellation of public gatherings and events and the closure of facilities (for example, university labs for processing samples).

Most respondents from ECCC and the communities said that the program allows Indigenous communities to conduct projects that are in line with their desire to act as stewards of the Great Lakes. While it is too early to assess the impacts of these projects, it was recognized by most Indigenous key informants that they have had a significant impact on capacity building. This capacity is expected to help Indigenous communities to play a role in Great Lakes protection through projects that are in line with their priorities.

Many internal and external respondents provided comments on the strengths and limitations of ECCC’s administration of the G&Cs. Indigenous respondents said that the GLPI allowed for multi-year funding and supported a variety of projects. However, all Indigenous key informants pointed out that more funding will be essential to supporting Indigenous engagement around the Great Lakes.

ECCC key informants explained that the GLPI G&C program area for engaging Indigenous Peoples in addressing Great Lakes issues was designed in discussion with Indigenous communities. ECCC held in-person regional meetings, webinars, teleconferences and 1-on-1 discussions to collect insights from Indigenous communities about the direction and parameters of the funding. Potential recipients were provided with contact information for program officers and informed that assistance was available on request. ECCC supported applicants by navigating them through the application process via webinars and 1-on-1 assistance from program officers. All project proponents interviewed indicated that their interactions with the GLPI officers were very positive.

However, while projects were aligned with ECCC’s criteria for G&C funding, project eligibility was not found to encompass all environmental priorities of Indigenous organizations and governments, such as projects about biodiversity. The application process was also deemed overly complex especially for smaller organizations or communities, according to some internal and external respondents. From some of the recipients’ perspective, application forms were complex and cumbersome, with repetitive questions and opaque language.

Some ECCC key informants also felt that some of the rules and processes around the disbursement of funds are too constraining. For example, key informants explained that Indigenous communities often rely on a combination of short-term funding sources to sustain community projects generally. This means they constantly have to write proposals and comply with reporting requirements for several funders.

Support for applicants was identified as important and ECCC may need more capacity to provide enhanced support. ECCC staff would also need expertise, a better knowledge base about the Indigenous context, as well as an increased risk tolerance (for example, consider co‑management approaches which are deemed higher risk). Continued training and education for ECCC staff on treaty rights and cultural awareness was mentioned as important by some Indigenous key informants.

Reflecting on potential improvements, some internal and external key informants mentioned that there was an opportunity to further engage Indigenous participants prior to the actual funding decisions. While interview respondents recognized the efforts to meet with the communities early in the process, there are examples of other federal departments that have engaged Indigenous staff, such as Elders, to assist them directly in their work. Some sit on G&Cs committees, play mentoring roles with Indigenous students and provide guidance to departments on how to implement projects.

3. Conclusions, recommendations and management response

3.1 Conclusions

According to findings, the investment of FAP funding for the GLPI was generally on track to achieve its expected results. New program areas supported through this funding were integrated in the existing governance structures for the Great Lakes and benefited from collaborations from many governments and non-government stakeholders for monitoring and restoration activities. Although the success of the GLPI relies in part on the collaboration of many partners, there are opportunities to strengthen collaborations with other federal departments and other partners.

During the time period of the evaluation, the GLPI underspent its budget, mostly due to delays in staffing. With respect to project management, systems are in place to ensure efficient management of program activities and G&Cs. Collaborations with other stakeholders, such as provincial and non-government organizations, also lead to efficiency by leveraging external resources, expertise and data. As well, the GLPI could review its G&Cs reporting templates to include more quantitative indicators. Otherwise, ECCC managers use the GLPI performance information to support their decision-making.

With respect to performance, the GLPI improved nutrients monitoring and understanding of the nutrients and algal blooms in the Great Lakes, particularly Lake Erie. The GLPI G&Cs supported projects for the purposes of demonstrating innovative approaches to reduce phosphorus loadings, or to developing technologies to reduce loadings, such as devices that remove phosphorus from manure and wastewater. This can then be used as slow-release fertilizer which, if applied properly, can reduce loadings. These projects are expected to have impacts in the short and medium terms. Despite the success of the GLPI in these efforts, it will take many years of significant concerted effort by all partners to achieve Canada’s phosphorus load reduction target for Lake Erie.

The GLPI also supported the implementation of the Nearshore Assessment Framework efficiently and effectively. 2 reports were completed (Lake Erie and Lake Ontario). 1 federal department and some local communities have implemented actions based on the results. The assessment of the ecological value component has not been completed but will be expected to be done for the final integrated report.

The GLPI investments in reducing releases of harmful chemicals were on track. The work focused on areas not addressed by other ECCC programs (including Canada’s Chemicals Management Plan).

With respect to Indigenous engagement, the GLPI investments supported awareness-raising projects, monitoring activities and restoration work. Significant efforts were made by ECCC to collect perspectives from Indigenous government and community representatives on program design. The parameters of the G&Cs could be more aligned with the needs of the communities and governments. ECCC could further improve its programming by considering practices observed in other departments.

3.2 Recommendations and management response

2 recommendations are directed to ECCC’s Assistant Deputy Minister (ADM) of Strategic Policy Branch, as the senior departmental official responsible for the GLPI.

Discussion: Efforts by ECCC to engage with Indigenous governments and communities have been positively recognized. Through G&Cs, the GLPI provided financial support to projects led by Indigenous governments, communities and organizations. Support was also provided to enhance participation of Indigenous peoples in binational and bilateral management tables and subcommittees. While there were notable accomplishments, there remain opportunities to enhance and strengthen ECCC’s capacity to foster Indigenous engagement. For example, the evaluation found a need for ECCC to provide more support during the application process to G&Cs applicants from Indigenous communities. Continued training and education for ECCC staff on Treaty Rights and cultural awareness was also deemed to be essential. Finally, there is also an opportunity to further engage Indigenous participation at the onset of the application design process in order to inform the selection criteria.

Statement of agreement or disagreement: The ADM of SPB agrees with the recommendation.

Management response: The Regional Director General – Ontario will identify best practices to enhance and strengthen Indigenous engagement and participation in the GLPI G&C programs and projects. The Regional Director General – Ontario will implement these best practices in collaboration with the G&Cs Centre of Expertise in the Corporate Services and Finance Branch as required.

Deliverable(s):

1. Enhanced efforts to raise awareness in Indigenous governments and communities of GLPI funding opportunities and application processes.
   1. Timeline: By March 2023, and ongoing
   2. Responsible party: Regional Director General-Ontario
2. Unnecessary burdens and barriers in the application process are examined and eliminated.
   1. Timeline: By March 2023, and ongoing
   2. Responsible party: Regional Director General-Ontario
3. Increased assistance to applicants interested in developing and submitting funding applications.
   1. Timeline: By March 2023 and ongoing
   2. Responsible party: Regional Director General-Ontario
4. Enhanced training opportunities for ECCC staff to build knowledge and awareness of Indigenous cultures, issues and priorities.
   1. Timeline: By March 2022, and ongoing
   2. Regional Director General-Ontario
5. Identification of potential approaches/best practices relevant to the GLPI employed by ECCC and other departments that provide Indigenous groups the opportunity to inform funding decisions. Selection and piloting of at least 1 new approach including, where appropriate, the increased strategic use of directed G&C funding to Indigenous Peoples.
   1. Timeline : By March 2023, and ongoing
   2. Responsible party: Regional Director General-Ontario

Discussion: The current reporting forms for the GLPI mostly include open text fields that are not conducive to consistent and quantitative measures of the expected outputs and impacts of the FAP. For consistency and improved reporting, it is recommended that the GLPI review its G&Cs reporting strategy to introduce quantitative indicators in its G&Cs reporting forms.

Statement of agreement or disagreement: The ADM of SPB agrees with the recommendation.

Management response: The Regional Director General – Ontario will review G&C monitoring tools to further incorporate quantitative performance indicators in the GLPI projects, where applicable. If necessary, the Regional Director General – Ontario will seek changes to the reporting template for recipients.

Deliverable(s)

1. GLPI G&C reporting strategy is reviewed to determine how to supplement existing G&C reporting with improved quantitative indicators.
   1. Timeline: By January 31, 2022
   2. Responsible party: Regional Director General-Ontario
2. New supplemental quantitative indicators are developed to improve tracking of the expected outputs and impacts of the GLPI.
   1. Timeline: By January 31, 2022
   2. Responsible party: Regional Director General-Ontario
3. Supplemental G&C monitoring tool or a modified reporting template with new quantitative indicators and results is piloted during the end-of-year reporting of FY 2021 to 2022 GLPI G&Cs.
   1. Timeline: By March 31, 2022
   2. Responsible party: Regional Director General-Ontario

Appendix A: evaluation scope, methodology and limitations

The evaluation focused on GLPI activities funded under the FY 2017 to 2018 FAP. This work took place concurrently with an evaluation of the FAP investment in the LWBP. The evaluation examined the extent to which planned objectives were achieved/on track in the period between FY 2017 to 2018 and FY 2020 to 2021.

Key questions to be addressed by this evaluation were:

Efficiency

1. To what extent are Great Lakes freshwater activities, supported by the FAP resources, governed collaboratively?
2. To what extent have the FAP resources been used efficiently?
3. To what extent is performance information being used to inform decision-making?

Effectiveness

4. To what extent has the program monitored nutrient loads and supported reductions in nutrient loading?
5. To what extent have funded scientific activities helped to advance the protection of Great Lakes nearshore areas?
6. To what extent are persistent toxic substances, and other substances of concern, being addressed by supported projects?
7. To what extent are Great Lakes freshwater activities under the FAP supporting the engagement of Indigenous peoples?

Approach and methodology

The evaluation used multiple lines of evidence for data collection.

Document review: review of internal documents from ECCC on program management and results; external sources on results, collaborators’ activities and projects; and 2017 evaluation and update on management response

File review:A review of 13 G&C project final reports and other program documents

Financial data analysis: review of departmental reporting documents

Key informant interviews: 17 interviews were conducted with respondents from ECCC’s representatives, other federal departments, the International Joint Commission, and the Government of Ontario.

In the report, the proportion of respondents is indicated using 2 qualifiers: some (less than majority of respondents) and most (majority of respondents). Considering the number of respondents (17, including 12 from ECCC), it was not deemed appropriate to introduce further breakdowns of proportions.

Case studies: 2 case studies examining Indigenous people’s engagement and implementation of the Nearshore Assessment Framework. Each case study involved a document and file review and key informant interviews. For the Indigenous people’s engagement case study, 7 interviews were conducted with Indigenous respondents. For the nearshore engagement case study, 4 interviews were conducted with ECCC staff, and 2 interviews were conducted with external respondents (other federal departments and 1 respondent from the U.S.).

Limitations and mitigation strategies

1. Of 41 G&C projects funded by the GLPI, only 13 projects were completed as of March 2020
   1. Mitigation strategies: The evaluation team mitigated this challenge by incorporating qualitative evidence from interviews and case studies to supplement limited number of project reports.
2. While few key informant interviews were conducted with respondents from other departments and levels of government, most of the interviews were conducted with ECCC.
   1. Mitigation strategies: This limitation should be considered in interpreting study findings. However, evidence from documents and literature were reviewed to supplement and corroborate interview data.