4. Findings

This section presents the findings of this evaluation by evaluation issue (relevance and performance) and by the related evaluation questions.

For each evaluation question, a rating is provided based on an analysis of the evaluation findings. The rating statements and their significance are outlined below in Table 2. A summary of ratings for the evaluation questions is presented in Annex D.

4.1. Relevance

4.1.1. Continued Need for Program

There is a demonstrated need to restore and maintain the Great Lakes ecosystem. The Great Lakes have enormous environmental, social and economic importance and value to Canada. Current and emerging environmental concerns for the Great Lakes require ongoing attention from the Government of Canada. There is evidence of coordination of efforts to avoid duplication among the various jurisdictions involved in restoring and maintaining Great Lakes water quality.

• The Great Lakes ecosystem is important in terms of size and environmental, social and economic value. The Great Lakes are the largest system of fresh surface water on earth, with about 18% of the world's fresh surface water, an ecosystem that supports over 3,500 species of plants and animals, and fishing and shipping industries that inject more than $7 billion annually into Canada's economy.^{Footnote 1} 
• There are a number of environmental issues that continue to be problematic for the Great Lakes, including
• algal blooms and the toxins, habitat degradation, illness and mortality caused by their proliferation;^{Footnote 2},^{Footnote 3} 
  • excess phosphorus^{Footnote 4} from several rural and urban point and non-point sources, such as urban and agricultural runoff, improper manure storage, municipal wastewater effluent, septic systems, and industrial discharges;^{Footnote 5} 
  • invasive species,^{Footnote 6} with a total of 185 aquatic and at least 157 terrestrial non-native species that disrupt the ecosystem, degrade water quality by increasing turbidity,^{Footnote 7} concentrating toxins, and altering nutrient and energy flows,^{Footnote 8} and that cause significant economic damages;^{Footnote 9} 
  • warming of the surface waters of the Great Lakes as a result of climate change;^{Footnote 10} and
  • numerous Areas of Concern (AOCs) that are still categorized as “impaired;”  only 3 of 17 AOCs have been delisted and 2 identified as AOCs in recovery since the program began in 1987.^{Footnote 11} 
• Survey respondents are almost unanimous in indicating that there is continued need for ECCC’s Great Lakes program to meet federal government commitments and to improve the water quality and ecosystem health of the lakes. Similarly, key informants cited emerging and ongoing issues with respect to water quality, pollutants, algae blooms, population growth, industrial development, climate change, and invasive species as evidence of an ongoing need for the research and monitoring supported by the program. They also emphasized the need for coordinated actions to address these concerns, including continued involvement of the federal government in meeting commitments in the Canada–US GLWQA.
• While some key informants identified areas of overlap between the mandates of initiatives in different jurisdictions, they reported that these areas of overlap are appropriate given that Canada, Ontario and the US share many Great Lakes issues in common and have similar responsibilities in their jurisdictions. Key initiatives in other jurisdictions include the US Great Lakes Restoration Initiative and the Province of Ontario's Great Lakes Strategy. Interview respondents also emphasized that there are coordinated efforts by these jurisdictions to avoid duplication. The agreements with Ontario and the US provide the framework for cooperation and collaboration.

4.1.2. Alignment with Federal Government Priorities

The Great Lakes program is aligned with federal government and ECCC priorities. The initiative was included in the 2010 Budget and is consistent with the Government of Canada’s expected outcome of a clean and healthy environment, the Federal Sustainable Development Strategy’s Goal 3 to maintain water quality and availability, and ECCC’s strategic outcome “Canada’s natural environment is conserved and restored for present and future generations.”

• Findings from the document review and key informant interviews indicate that the Great Lakes program is aligned with federal government and ECCC priorities. For example, the program is consistent with the 2014 COA, through which the governments of Canada and Ontario have renewed their commitment to restore, protect and conserve the Great Lakes. In addition, in the renewed COA, the Parties affirmed their commitment to implement the 2012 Canada–US GLWQA. In addition, the program is directly associated with ECCC’s strategic outcome “Canada’s natural environment is conserved and restored for present and future generations.”^{Footnote 12} 
• Investments in this initiative were included in the 2010 Budget. In 2012 the Government of Canada committed $16 million over four years to the Great Lakes Nutrient Initiative to address nearshore water quality and aquatic ecosystem health, and toxic and nuisance algae growth in the Great Lakes. Budget 2013 reiterated the Government of Canada’s commitment to sustain the Great Lakes, including committing to responding to recommendations of the International Joint Commission (IJC) to the Canadian and US governments regarding water levels and flows in the upper Great Lakes.
• The program is also consistent with the Government of Canada’s expected outcome of a clean and healthy environment^{Footnote 13} and Goal 3 of the Federal Sustainable Development Strategy (FSDS),^{Footnote 14} which includes targets to protect and restore the Great Lakes AOCs and aquatic ecosystem. For instance, the most recent Progress Report of the FSDS^{Footnote 15} presents evidence on phosphorus levels in the Great Lakes, including that levels remain above the objectives for the western and central basins of Lake Erie.

4.1.3. Alignment with Federal Roles and Responsibilities

The Great Lakes program is consistent with federal government roles and responsibilities as specified in relevant legislation and international commitments. A federal presence is necessary to ensure coordination among involved parties, which also is consistent with the relevant legislation. Stakeholders believe that delivery of the Great Lakes program is an appropriate role for the federal government to play in the restoration and maintenance of the Great Lakes basin ecosystem.

• The Great Lakes program is consistent with federal legislation related to the environment, including
  • the Department of the Environment Act, which specifies that the powers, duties and functions of the ECCC Minister include the preservation and enhancement of the quality of the natural environment, including water; and
  • the Canadian Environmental Protection Act, which specifies various roles for the Government of Canada related to environmental protection, including taking preventive and remedial measures to protect, enhance and restore the environment and endeavouring to act with regard to the intent of intergovernmental agreements and arrangements entered into for the purpose of achieving the highest level of environmental quality throughout Canada.
• According to the 2013 Federal Sustainable Development Strategy, the federal, provincial and territorial governments are all responsible for achieving environmental sustainability objectives.
• Almost all of the stakeholders consulted through the survey (94%) agree that the work being undertaken as part of the Great Lakes program is an appropriate role for the federal government to play in the restoration and maintenance of the Great Lakes basin ecosystem.
• Similarly, all of the ECCC personnel, partners and stakeholders consulted in interviews agree that the Great Lakes program is consistent with the federal government’s roles and responsibilities. In addition, some federal government interviewees noted that a federal presence and role are warranted given the international dimension of the Great Lakes, and to help coordinate the various parties involved.

4.2. Performance – Efficiency and Economy

4.2.1. Program Design

Overall, the design of the Great Lakes program is appropriate for achieving its intended outcomes: the designs of the GLNI, GLAP and GLSRP are logical; program structures, processes and science are aligned with the new GLWQA; a systematic process exists for re-designating BUIs and delisting AOCs; and Gs&Cs are felt to be an effective mechanism for engaging and supporting a variety of partners to help achieve program results. While the overall program design is sound, some concern was expressed regarding the need for a more clearly defined and timely approach and strategy for delisting AOCs and reallocating resources, as well as the adequacy of the program’s capacity to meet an expected increase in demand for science to support the GLWQA Annex Subcommittees.

• Based on a review of the program documentation and logic models (see Annex B), the design of GLNI, GLAP and GLSRP is logical, science-based and consistent with the objectives of the Great Lakes program.
• Most ECCC personnel, partners and stakeholders believe that the design of the Great Lakes program is appropriate for achieving its intended outcomes. They note that the major issues and priorities are identified in the updated 2012 Canada–US GLWQA and that ECCC program initiatives have been aligned to be consistent with the new structure and processes of the updated agreement.
• There is a systematic process in place for re-designating BUIs and delisting AOCs.^{Footnote 16} However, remediation work for the AOCs consumes a high proportion of program funding, and some ECCC managers suggest that there should be a more clearly defined strategy with targets and timelines, where feasible, for delisting AOCs as soon as possible and reallocating the resources to other priorities, including scientific research. A concern was expressed over the tension between historical BUIs, on which delisting is supposed to be based, and new and emerging problems. Some key informants indicated that community stakeholders resist the delisting of an AOC if new problems emerge, even if the original BUI problems have been adequately addressed. This community stakeholder resistance is thought to lead to some caution and delays in the delisting process, which can slow down the process of addressing lakewide issues not specific to AOCs.
• Grants and contributions are an effective mechanism for engaging and supporting a variety of partners to help achieve program results. Key informants indicated that the projects would not be possible without the financial support from ECCC, which can help to leverage funds and activities from other partners, and many would like to see a greater emphasis on funding for science-based projects, including projects conducted by universities. In addition, large majorities of stakeholders surveyed (over 80%) agree that ECCC should provide grants and contributions to various partners to support their initiatives to improve Great Lakes water quality and ecosystem health.       
• The renewal of the GLWQA has led to a long-term process to improve the alignment of the science supported and conducted by ECCC with the objectives of the 10 GLWQA Annexes, which were developed with input from scientists. ECCC managers generally reported that policy and science related to the Great Lakes are now better aligned in the Department, with good interactions between the two functions to identify what is advisable from a scientific perspective and what is feasible from a policy perspective. Some key informants expressed concern, however, about limitations on the program’s capacity to meet expected increases in demand for science to support GLWQA Annex Subcommittee processes, understand the complexity of interacting factors affecting the lakes, and identify new and emerging risks.
• Survey results indicate that a small majority of ECCC personnel, partners and stakeholders agree that the activities and outputs of GLAP (65%), GLNI (56%) and GLSRP (64%) are adequate and appropriate for improving Great Lakes water quality and ecosystem health.

4.2.2. Program Governance and Management

The management and governance mechanisms established in 2012 for the renewed GLWQA, including the Great Lakes Executive Committee (GLEC) and Annex Subcommittees, have set up binational processes to achieve a variety of objectives consistent with ECCC’s program outcomes and have led to a more open and transparent governance process. Despite these improvements, perceived challenges include communications and collaboration given the number of program initiatives and stakeholder organizations involved, and the fact that roles and responsibilities as defined in the GLWQA are not yet clearly understood by all stakeholders.

Governance Structure

• The evaluation found that program governance appears to have improved with the implementation of the 2012 GLWQA:
  • Under the renewed GLWQA, the Great Lakes Executive Committee and Annex Subcommittees are thought to have improved governance and set up binational processes to achieve a variety of objectives consistent with ECCC’s program outcomes.
  • The updated GLWQA Annexes and Subcommittees are widely considered to be better aligned with the needs and current realities of the Great Lakes than the pre-2012 structure, and key informants note that the work being done is better focused on achieving the expected results, in part due to the establishment of time-based commitments.^{Footnote 17}
• In particular, interviewees perceive that the improved committee structure put in place with the renewal of the GLWQA has led to a more open and transparent governance process, whereby information is more readily available to stakeholders and to the public, and there is an extensive and inclusive engagement and consultation process involving a wide range of stakeholders.
• Survey results also demonstrate fairly strong support for most Great Lakes governance bodies, as more than half of respondents reported that the Remedial Action Plan Committees (72%), COA Executive Committee (58%), GLAP Workplan Review Team (54%) and COA Management Committee (53%) have been largely effective (i.e., responded with a 4 or 5 on a 5-point scale)^{Footnote 18} in managing the delivery of the Great Lakes program. The GLEC is the only committee rated as largely effective by fewer than half of the survey respondents (48%), although interview and case study evidence suggests that it is simply too early to tell whether GLEC will be effective.^{Footnote 19} 
• Key informant and case study results nonetheless identify a number of suggested areas of improvement. For example, key informants variously noted
  • some duplication and bureaucracy associated with the COA Executive and Management Committees, and there may be opportunities for streamlining;
  • the lack of a mechanism for coordination across the 10 GLWQA Annex Subcommittees;
  • that the large number of participants in GLEC may have an adverse impact on its effectiveness; and
  • that it may be difficult to maintain the interest and participation in the GLEC by some stakeholder groups (e.g., Indigenous groups, environmental NGOs) because active participation requires a significant level of commitment and resources.

Clarity of Roles and Responsibilities

• At the level of individual program components, for GLNI, responsibilities of RDGO–Ontario and S&T Branch for the delivery of activities and outputs are clearly specified in the program logic model (Annex B). Also, responsibilities of governance mechanisms associated with GLAP and the COA are specified (see Annex A).
• The overall roles and responsibilities of the Parties and the governance mechanisms are defined by the GLWQA. The GLWQA includes clear statements of the objectives associated with each of the 10 Annexes and the responsibilities of the Parties to achieve and report on progress toward achieving these objectives. For each of the Annexes, the GLWQA specifies that time-limited task teams will be created to focus efforts on priority issues and projects (e.g., the Objectives and Loading Task Team of the Annex 4 Nutrients Subcommittee).
• Survey results reveal mixed opinions on the degree to which roles and responsibilities of the different governments and organizations involved in Great Lakes program initiatives are clear and understood. Less than half of survey respondents (45%) agree that roles and responsibilities are clear and understood. The remaining 55% of respondents indicated that they either disagree (34%) or neither agree nor disagree (21%). Respondents who disagree suggested that there is a need to improve communications regarding roles and responsibilities.
• Key informants reported that different stakeholders are still working out the details of their roles and responsibilities as the work conducted under the governance structure of the new GLWQA progresses. Some external stakeholders suggested that a chart specifying the roles of all players involved in protecting the Great Lakes would be helpful.
• Key informants also reported that they generally believe that the GLWQA and new COA, and their governance structures, are complementary and that there will be no duplication in the work undertaken for each agreement.

Communications, Engagement and Collaboration with Stakeholders

• Most key informants acknowledged that, overall, communications and collaboration are challenging in view of the number of program initiatives and the many stakeholder organizations involved. Despite this, government-to-government communications and collaboration are generally regarded as very good. Documentary evidence also shows that a previous evaluation recommendation^{Footnote 20} to improve coordination among federal government partners involved in GLAP has been addressed through the establishment of the GLAP Five-Year Workplan involving ECCC and its federal partner DFO.
• Views on the effectiveness of communication and engagement with other stakeholder groups and the public are mixed. The greater engagement and participation of various stakeholder groups in the GLEC and Annexes, as well as the enhanced openness and transparency of the process, are viewed very positively by almost all key informants. However, concerns were expressed about the sustainability of engagement by some stakeholders over the long term because of the resource requirements, in people and money, to keep up with a complex and demanding process. Survey results also suggest that respondents have concerns about communications and collaboration among partners and stakeholders, with just over half of respondents (53%) agreeing, and one in ten (12%) disagreeing, that this is effective.

4.2.3. Program Efficiency and Alternatives

A number of factors suggest that the Great Lakes program is efficient and cost-effective. The program has made efficiency improvements to the project application and reporting processes, and the Gs&Cs component has been successful at leveraging approximately three-quarters of project resources from sources other than ECCC. The combined administrative costs of the GLAP and GLNI programs are comparable to those for other ECCC Gs&Cs programs. In addition, most stakeholders consulted generally agree that Great Lakes initiatives are cost-effective.

• In the previous GLAP IV evaluation, it had been recommended that “the RDG Ontario should explore means of streamlining both the GLSF funding approvals process and the GLSF application process.”^{Footnote 21} In an ECCC follow-up report (2011), it was indicated that the program made efficiency improvements in this regard by adopting the application and reporting processes being implemented as part of ECCC’s Action Plan for Gs&Cs Reform. Evidence from the interviews with Gs&Cs recipients confirmed that the application process has generally improved these last few years (e.g., better reporting templates and more timely response from ECCC).
• The Gs&Cs projects have been successful at leveraging other sources of funding and resources. Based on available information in the sample of Gs&Cs project files reviewed, ECCC contributed 26% of project resources, of which 2% was in kind, while 74% of resources were contributed by other sources, of which 53% was in kind.
• While relatively few key informants provided specific comments on the efficiency and economy of the Great Lakes program, most of them viewed the program as being efficient and economical overall, for example, because fiscal restraint has imposed discipline on program spending such that program funds and activities are well targeted and because savings are expected to be generated by project actions (e.g., projects to reduce the problem of aquatic invasive species).
• Over half of respondents to the stakeholder survey (57% to 60%) agree that the Great Lakes initiatives are the most cost-effective ways for the Government of Canada to restore and maintain water quality and ecosystem health in the Great Lakes basin. Of the respondents who do not agree, very few (3% to 6%) disagree that these initiatives are cost-effective while most neither agree nor disagree (33% to 40%).
• When asked if they think changes could be made to deliver Great Lakes activities and outputs at a lower cost, nearly two-thirds (65%) indicated that they did not know. Slightly more more than one in four (27%) think changes could be made to reduce costs while slightly fewer than one in ten (8%) reported that no changes are needed. The key suggestions for delivering the program at a lower cost were to place more focus on funding and cost controls, and to improve coordination and collaboration among the involved parties.

Administrative Ratio

• An indicator of program efficiency is the administrative ratio, which is calculated as direct program costs (salaries and O&M)^{Footnote 22} divided by the Gs&Cs funding disbursed. For the evaluation timeframe, a total of $14.8 million in Gs&Cs was disbursed and the average administrative ratio for the program is approximately 0.15 (see Annex E). This is comparable to the estimated ratios observed in evaluations of other ECCC Gs&Cs programs, which range from 0.15 to 0.25.^{Footnote 23} 
• The ratios for GLNI (starting at 0.55 and reducing to 0.29 over time) are higher than those for GLAP (starting at 0.18 and reducing to 0.13), which may suggest that the former is somewhat less efficient. However, it is important to note that there have been far fewer projects for GLNI than GLAP (2 to 4 versus 45 to 56 per year) and that GLAP is a long-standing program, allowing staff an opportunity to gain experience and improve efficiency in administering Gs&Cs, whereas GLNI is a newer program.

Performance Measurement

The program reports federally and provincially through the COA and binationally through the Canada–US GLWQA on progress in meeting commitments in the agreements. While there are logic models for GLNI, GLAP and the Sediment Remediation projects, there is no formal overall logic model and performance measurement strategy for the Great Lakes program. Program managers and scientists reported that data generally are adequate but that significant improvements could be made. They acknowledge that it is difficult to integrate and aggregate data from the various Great Lakes program initiatives.

• Program performance is documented through various reporting mechanisms, many of which fulfil COA and Canada–US GLWQA reporting requirements. These include reports such as
  • the State of the Great Lakes reports^{Footnote 24} (most recently released in 2011), which deal with the major factors related to water quality, aquatic-dependent life and landscapes and natural processes that have an impact on the state of the Great Lakes ecosystem. The results are presented at the State of the Lakes Ecosystem Conferences co-hosted by ECCC and the US Environmental Protection Agency (most recently held in October 2011);
  • reports related to the 2012 GLWQA issue-specific Annexes, such as Lakewide Action and Management Plan reports for each of the Great Lakes;^{Footnote 25} 
  • the IJC’s Biennial Report on Great Lakes Water Quality,^{Footnote 26} which presents 14 indicators of the chemical, physical and biological integrity of the Great Lakes and two indicators of the performance of government programs in meeting objectives in restoring sites of historic contamination identified as Areas of Concern;
  • Canadian Environmental Sustainability Indicators (CESI) relating to the state of the Great Lakes: restoring the Great Lakes AOCs (in terms of reducing BUIs and restoring beneficial uses) and phosphorus levels in the Great Lakes;
  • various micro-level reporting activities related to ongoing monitoring and modelling work on selected Canadian tributaries and nearshore areas of Lake Erie; data and reports to assess the status of AOCs and BUI indicators; and a performance measurement framework^{Footnote 27} to assess progress in restoring beneficial uses and delisting AOCs, which was developed in response to a recommendation from the 2010 evaluation of GLAP IV; and
  • indicator studies to establish baseline biological and chemical conditions for the Randle Reef Sediment Remediation Project, which will be used subsequently to assess the effectiveness of the project through a comparison with post-remediation conditions.
• Additional reporting is also expected to stem from commitments made in the 2012 GLWQA. This includes a Progress Report of the Parties (Canada–US), first issued in 2016, and an IJC triennial Assessment of Progress Report, to be issued in 2017, which will include an assessment of the Parties’ achievement of a series of general and specific objectives related to Great Lakes water quality.
• In terms of performance measurement in direct support of the Great Lakes program, there is no formal overall logic model or complete performance measurement strategy^{Footnote 28} for PAA sub-program 1.3.4. Logic models have been individually developed for GLNI, GLAP and GLSRP; however, there is currently no systematic collection of indicator data related to many logic model outcomes. Nonetheless, some performance information related to two indicators (i.e., phosphorus levels in the Great Lakes and progress in restoring AOCs in terms of the number of AOCs delisted and number of beneficial uses listed as “impaired” or “requires further assessment”) was collected as part of the Department’s Performance Measurement Framework and reported in Departmental Performance Reports, though not consistently over the five years of the evaluation timeframe.^{Footnote 29} 
• In addition to a lack of performance measurement in support of the various Great Lakes activities, ongoing gaps in performance measurement and reporting for the program were evident in relation to Gs&Cs project reporting. ECCC follow-up to a previous evaluation recommendation to “develop a more regular and robust reporting approach for GLAP projects conducted by federal partners”^{Footnote 30} indicated that the program had established guidelines for monitoring and reporting GLAP funds as well as a GLAP work planning template and associated annual report.^{Footnote 31} However, program management indicated that there is no roll-up of the results information in Gs&Cs project final reports.
• Interviewees, including program managers, scientists, and Gs&Cs recipients, generally indicated that performance data are adequate. ECCC managers in particular reported that monitoring data for nutrients, AOCs and sediment remediation projects have improved and should continue to improve. A number of suggestions were nonetheless offered for improving program performance measurement, including
  • developing a mechanism to monitor the longer-term impacts of Gs&Cs projects (though some recipients have already incorporated project monitoring activities, e.g., pre/post water samples, satellite images and pictures);
  • devising some means of aggregating data from the various Great Lakes program initiatives to show the performance of the overall program;
  • establishing links between interventions and outcomes (e.g., reducing nutrient levels and water quality outcomes);
  • differentiating the impacts of new interventions and legacy effects;
  • isolating the impacts of land-based activities on nutrient levels (e.g., agriculture, winter run-off);
  • assessing the quality of data (e.g., phosphorus levels) contributed by various stakeholders such as funded project proponents, and integrating all relevant data;
  • improved data archiving; and
  • various other suggestions for improving performance data, including improved coordination among federal partners, better documentation, and adopting adaptive management^{Footnote 32} for ongoing monitoring and assessment.

4.3. Performance – Effectiveness

Evidence of outcomes achieved^{Footnote 33} was obtained through all lines of evidence and results are presented in this section by the intended outcomes specified in the logic models for GLNI and GLAP.

The survey of stakeholders provided ratings of the extent of achievement for all immediate, intermediate and longer-term outcomes. Results are presented for the percentage of survey respondents who perceived that a particular outcome has been achieved to a large extent (i.e., responded with a 4 or 5 on a 5‑point scale) or to some extent (i.e., responded with a 2 or 3 on the scale). It should be noted that because many committees and subcommittees were still actively working on key deliverables for GLNI, including targets to be set, it was too early to assess the effectiveness of some aspects of this program. According to the interview evidence, this may explain modest survey ratings of outcomes achievement, as much of the work is still in progress.

4.3.1 Great Lakes Nutrient Initiative

Objectives/Immediate Outcomes

Due to the fact that there are a large number of immediate/direct outcomes (12) in the GLNI logic model, the findings in this section are presented for the five GLNI objectives with which the immediate outcomes are associated. As detailed below, the findings indicate that the work being conducted as part of GLNI generally is on track and progress has been made on each of the five objectives.

Immediate Outcome 1: Measurement of phosphorus loads from urban and agricultural sources to identify and assess phosphorus discharges, inform decision making, and track and report progress toward achievement of phosphorus reduction targets – Acceptable.

Acceptable progress has been made toward achieving outcomes in the areas of science and monitoring for the measurement of phosphorus loads, including the development of inventories, models and baseline information about nutrients, and phosphorus reduction targets have been developed.

• Changes and resource alignments within ECCC’s Water Science and Technology Directorate resulted in certain delays at the end of Year 1 (2012–2013) of the Great Lakes Nutrient Initiative. Consequently, this led to adjustments to some of the monitoring work and setbacks in the analysis of data.^{Footnote 34} However, the 2013–2014 DPR indicates there has been progress toward outcomes in the areas of science and monitoring, including the development of inventories, models and baseline information about nutrients. Documentation^{Footnote 35} also indicates that various sampling devices were installed to provide data on phosphorus loading^{Footnote 36} and information for forecasting models.
• Based on the work of the Annex 4 Nutrients Subcommittee, phosphorus reduction targets for the western and central basins of Lake Erie were announced by ECCC and the US EPA in February 2016, as planned.
• According to the IJC, both Canada and the United States have made important contributions to improving our understanding of the sources of excess nutrients and developing successful methods for managing them.^{Footnote 37} 
• A majority of stakeholders completing the survey indicated that GLNI has achieved to a large extent the sub-outcomes of having critical data for input to forecasting models to derive the phosphorus load reduction targets for Lake Erie (59%), and having an updated and improved understanding of Canadian phosphorus loadings to Lake Erie (56%). All remaining respondents indicated that these sub-outcomes have been achieved to some extent.

Immediate Outcome 2: Improved understanding of the influence of aquatic invasive species and other factors contributing to algae production, and of the impacts of toxic and nuisance algae on water quality and ecosystem health – Acceptable.

Research documented through GLWQA Annex Subcommittee reports has led to a better understanding of nutrient loadings as well as invasive species and other factors that affect water quality, ecosystem health and algae growth.

• Progress has been made by the Annex 4 Nutrients Subcommittee to improve understanding of invasive species and other factors affecting water quality. The 2012 amendment to the GLWQA contains several new Annexes responding to current and emerging challenges, including aquatic invasive species (AIS). ECCC contributed to the binational Biodiversity Conservation Strategies (BCS), which included an assessment of the threat of AIS, and assembled key regional partners to create a coordinated action plan for the common reed and other priority terrestrial invasive species. Many commentators on the COA were supportive of the current commitments under this agreement, particularly those related to early detection and rapid response initiatives, improved understanding and tools to respond to AIS, and engagement of the Great Lakes community to prevent, detect, respond to and manage AIS.^{Footnote 38} 
• The Government of Canada has been working collaboratively with Ontario to address AIS, and COA commitments reflect efforts to work together and with others to protect the waters of the Great Lakes from AIS. These collaborative efforts include planning and delivering early detection, response actions when and where such responses are possible, and science to improve understanding and develop new tools.^{Footnote 39} As a key partner, Fisheries and Oceans Canada also delivers science that contributes to the understanding of the state of the ecosystem and progress toward a range of GLWQA 2012 objectives, with specific focus on AIS and habitat and species in the Great Lakes.^{Footnote 40} Project file information indicates that some local projects involve the monitoring of invasive species, such as zebra mussels and shoreline plants.
• Close to half of stakeholders in the survey (44%) indicated that GLNI has achieved to a large extent the sub-outcome of an improved understanding of the connection between phosphorus concentrations and algae production, and 37% provided the same positive rating for the sub-outcome of an improved shared understanding of factors contributing to toxic algal production. All remaining respondents indicated that these outcomes have been achieved to some extent. These findings are not surprising given that efforts closely related to this outcome are ongoing as part of the Annex 4 Nutrients Subcommittee work.
• Based on the work of the Objectives and Loading Task Team of the Annex 4 Nutrients Subcommittee,^{Footnote 41} key informants generally believe that understanding of the factors contributing to algae production and its impacts has improved. They also recognize that the science is complex, the situation is dynamic, and ongoing research is required.

Immediate Outcome 3: Establish binationally agreed upon, science-based phosphorus load reduction targets – Acceptable.

The Annex 4 Nutrients Subcommittee has summarized the research on nutrient loadings in Lake Erie from Canadian tributaries and recommended phosphorus loading targets for Lake Erie.

• As outlined in the COA, there is a commitment to develop science-based phosphorus concentration and load reduction targets for Lake Erie by 2016. According to a 2014 report,^{Footnote 42} agreement on a binational approach to monitoring and reporting on phosphorus concentrations and loads will be sought at that time.
• The Objectives and Loading Task Team has been established by the Annex 4 Nutrients Subcommittee of GLEC. The Task Team is conducting an in-depth analysis using existing science and modelling tools, and measurements of phosphorus in different zones of Lake Erie to develop models and establish phosphorus targets for the lake. Recommendations for targets were published^{Footnote 43},^{Footnote 44} and, as noted earlier, binational targets were announced in February 2016.
• The Annex 4 Nutrients Subcommittee is on track to complete the work to develop a binational Phosphorus Reduction Strategy for Lake Erie.^{Footnote 45} 
• ECCC personnel, partners and stakeholders indicated that the work being conducted as part of GLNI generally is on track. While they believe that there still are knowledge gaps, they observed that research has led to a better understanding of phosphorus loadings, and progress has been made in developing phosphorus load reduction targets for Lake Erie.

Approximately half of survey respondents indicated that the GLNI has achieved the following to a large extent: updated and scientifically defensible phosphorus targets for management purposes, to meet key milestones in the Canada–US GLWQA (50%); and ecosystem objectives and phosphorus targets to guide domestic and binational phosphorus management decisions (47%). Approximately half also indicated some extent of achievement for each outcome. Modest ratings of the degree of achievement of these outcomes are not unexpected considering that work in support of these outcomes was ongoing at the time of the survey, and that the survey asked whether the outcomes had already been achieved.

Immediate Outcome 4: Develop and assess policy options for reducing phosphorus discharges from agricultural and urban point and non-point sources – Acceptable.

ECCC has completed an evaluation of policy options for reducing phosphorus discharges to Lake Erie, and efforts to further assess and refine these options are expected to continue as part of the development of Canada’s Domestic Action Plan.

• ECCC is leading an evaluation of policy options and best practices for reducing phosphorus discharges from both point and non-point sources to support decision making by all levels of government and the private sector for the control of toxic and nuisance algae levels in the Great Lakes.^{Footnote 46} 
• Most respondents indicated that the development of policy options for achieving phosphorus reduction has been achieved to some extent (57%) or a large extent (29%).
• Key informants noted that adaptive management, an approach that seeks to improve resource management by assessing and learning from the outcomes of management interventions, is a central theme of the Phosphorus Reduction Strategy, to reinforce the concept that the lakes are complex dynamic environments that are not fully understood and that are expected to change as we change our impacts on them. There is a consensus that the Phosphorus Reduction Strategy will serve as a road map for a domestic action plan and watershed phosphorus reduction implementation plans.
• Work on the development and assessment of policy options to reduce phosphorus discharges from Canadian sources to Lake Erie is mostly completed,^{Footnote 47} and efforts to further assess and refine these policy options are expected to continue as part of the development of Canada’s Domestic Action Plan, to be finalized in 2018.

Immediate Outcome 5: Develop a binational nearshore assessment and management framework – Acceptable.

Acceptable progress has been made by the Annex 2 Lakewide Management Subcommittee to develop a binational nearshore assessment and management framework. A draft framework was completed in February 2016.

• The Annex 2 Lakewide Management Subcommittee has established a Task Team to develop an integrated framework to identify priorities and guide management of nearshore areas of the Great Lakes. Implementation of the nearshore framework is expected to advance lakewide management.
• The work to complete the nearshore framework is well underway; a binational progress report was completed and accepted in December 2014.^{Footnote 48} Subsequently, a draft framework was completed in February 2016, and a revised draft was posted for public review in May 2016.
• Key informants reported that the preliminary work to develop a nearshore assessment and management framework has been more complex than expected. For example, the focus of the Ontario provincial government is on priority areas (e.g., beaches), while the federal government is looking at the lakes more systemically. Both the provincial and federal governments agree on a focus on non-point sources of pollution.
• Four in five (80%) survey respondents reported that the sub-outcome related to “coordinated binational management of nearshore areas to benefit Great Lakes water quality and aquatic ecosystems” has been achieved to some extent, which is consistent with the fact that the nearshore framework is still under development. Fewer than one in ten (7%) feel that this outcome has been achieved to a large extent.

Intermediate Outcomes

Almost all stakeholders who completed the survey believe that at least some progress has been made toward achieving all GLNI intermediate outcomes, and few or no respondents think that progress has been made toward these outcomes to a large extent. Additional evidence for each intermediate outcome is presented below.

Intermediate Outcome 1: Implement pollution prevention and control measures to reduce phosphorus loadings – Acceptable.

Acceptable progress has been made in developing phosphorus reduction targets, and measures are currently being developed in 2016 to achieve these targets.

• As noted, a Domestic Action Plan to reduce phosphorus loadings in Lake Erie to achieve binational targets is currently being developed in 2016, and is expected to be completed in 2018.
• Evidence from the key informant interviews indicates that the program’s Gs&Cs component has contributed to this outcome by helping communities to reduce sewage spills in the lakes. For example, in the file review, one project focused on diagnosing the state of a sewage system.

Intermediate Outcome 2: Minimize the occurrence and impacts of toxic and nuisance algae – Opportunity for Improvement.

Although some progress has been made over the past decades in reducing algae in Lake Erie, there is research evidence indicating that recent algae blooms are a result of excessive nutrient loadings and that there has been a levelling off or reversal of earlier reductions in nutrient loadings.

• Continual improvements to lakewide management of Lake Erie are being made,^{Footnote 49} but the results to date are mixed in terms of the impact these measures are having on algae. A study conducted by IJC (2013) using seven chemical indicators illustrates that there have been mostly favourable or stable results since 1987, reflecting the success of policy changes implemented in both Canada and the US after the original 1972 Agreement.^{Footnote 50} The same study, however, also concludes that recent (2013) algal blooms seen in the Great Lakes are a result of excessive nutrient loadings.^{Footnote 51} There are also signs of levelling off or even, in some cases, a reversal of reductions in nutrient loadings in the past decade and earlier. According to a 2014 IJC study, total phosphorus loads have not declined appreciably in many watercourses draining agricultural areas, and the highly bioavailable dissolved fraction of total loads (dissolved reactive phosphorus) has been steadily increasing.^{Footnote 52} 
• A major focus of the efforts of ECCC and its partners is to reduce the amounts of excess phosphorus entering the Great Lakes, Lake Erie in particular, which contributes to algae growth.

Intermediate Outcome 3: Canada and US establish and meet phosphorus load reduction targets for all Great Lakes – Unable to Assess.

It is premature to assess the degree to which phosphorus reduction targets are being met because targets for Lake Erie have only recently been developed.

• As noted, binational phosphorus reduction targets for the western and central basins of Lake Erie were announced by ECCC and the US EPA in February 2016. It is too early to assess the degree to which targets will be met.
• GLNI’s focus is on Lake Erie, but the science and policy approaches developed are eventually expected to be transferable to other Great Lakes.

Intermediate Outcome 4: Improved understanding and management of the Great Lakes nearshore – Acceptable.

Understanding of the nearshore has improved through the process of developing a nearshore assessment and management framework, and efforts to manage the nearshore will intensify once the framework has been approved in 2016.

• Key informants familiar with the work of the Annex 2 Lakewide Management Subcommittee reported that much has been learned and documented (e.g., in progress reports^{Footnote 53} ), through both science and stakeholder engagement, in the process of developing the nearshore assessment and management framework.
• Efforts to manage the Great Lakes nearshore will intensify following completion and approval of the nearshore assessment and management framework in 2016. As noted, a draft framework was completed in February 2016 and a revised draft posted for public review in May 2016.

Intermediate Outcome 5: Reduction in cumulative impacts of human activities in nearshore areas – Unable to Assess.

It is premature to assess this outcome because the nearshore assessment and management framework has not yet been finalized.

• More systematic measurement of the cumulative impacts of human activities on the Great Lakes nearshore^{Footnote 54} will be possible following completion and approval of the nearshore assessment and management framework in 2016.

4.3.2 Great Lakes Action Plan

Immediate Outcomes

Immediate Outcome 1: More effective/better integrated remedial actions in Areas of Concern – Acceptable.

Remedial actions in AOCs are being implemented through the development of Remedial Action Plans and implementation of funded projects focused on BUIs and AOCs with the involvement of local partners.

• Remedial Action Plans are developed for each AOC and, based on the file review, all Gs&Cs were targeted toward AOCs and BUIs and have involved various local partners, including municipalities, First Nations communities and other local organizations. Projects include
  • restoration initiatives, such as restoration of buffer zones, wetlands, forests and riverbank or stream habitats;
  • lake habitat projects involving the planting of trees and plants, and protected or enhanced wetland habitat for species at risk and other species;
  • monitoring projects including geomorphology assessment projects, water sampling and analysis projects, and deployment of monitoring mechanisms; and
  • education and awareness projects targeting landowners of available resources and general public awareness campaigns.
• Few project files include impact measurements, such as the impacts on water quality of these various funded activities.
• All survey respondents indicated that the work undertaken through GLAP has achieved this outcome to some (46%) or a large extent (54%).

Immediate Outcome 2: Improved identification of environmental problems and progress in Areas of Concern – Acceptable.

Identification of environmental problems and progress in AOCs is being achieved through the conduct of science and monitoring of ecosystem health as well as the preparation of Remedial Action Plans, Annual Workplans and Progress Reports for AOCs.

• The document review shows that Remedial Action Plans, Annual Workplans and Progress Reports are prepared to identify problems and track progress in AOCs.
• O&M funds are utilized for science and monitoring, which is integral to the identification and implementation of remedial actions in AOCs.^{Footnote 55} 
• Almost all survey respondents reported that the work undertaken through the GLAP has achieved this outcome to some (48%) or a large extent (48%).

Immediate Outcome 3: Improved management and coordination of efforts to restore and maintain the Great Lakes Basin ecosystem – Acceptable.

Management and coordination of efforts to restore the Great Lakes ecosystem have improved with the governance structure and processes of the 2012 GLWQA.

• There is a strong consensus among key informants that the Annex subcommittee structure and processes (described in Annex A) were strengthened for the 2012 GLWQA, compared to the previous agreement, and have greatly improved the management and coordination of efforts to achieve Great Lakes program outcomes, including restoring and maintaining the Great Lakes Basin ecosystem.
• Efforts within ECCC are reasonably well coordinated. Interview findings indicate that there were already long-standing distinct units within the department working on most of the applicable 2012 GLWQA Annexes (excluding the two Annexes led by other departments).
• All survey respondents indicated that the work undertaken through GLAP has achieved this outcome either to some extent (54%) or a large extent (46%).

Intermediate Outcomes

Intermediate Outcomes 1 and 2: Pollution from identified sources is minimized or eliminated and habitats are restored in Areas of Concern – Acceptable.

Since 1987, the environmental quality in Canada’s 17 AOCs has improved, with three AOCs delisted and two more designated as being in the recovery stage (one of which was so designated in 2011, within the evaluation timeframe). Projects in AOCs have had a number of benefits, including reducing pollution and restoring habitats.

• The environmental quality in Canada’s 17 Great Lakes Areas of Concern has improved since the restoration program began in 1987. Although no AOCs were delisted in the evaluation timeframe, to date three out of 17 AOCs have been delisted (i.e., all restoration activities have been implemented and delisting criteria met): Collingwood Harbour and Severn Sound, both in Lake Huron (in 1994 and 2003 respectively), and Wheatley Harbour in Lake Erie (in 2010). Two more AOCs have been designated as being in the recovery stage: Spanish Harbour in Lake Huron (in 1999) and Jackfish Bay in Lake Superior (in 2011).^{Footnote 56} 
• ECCC reports that by 2019, required remedial actions are expected to be completed in five more Canadian Areas of Concern: Nipigon Bay, Niagara River, the Bay of Quinte, St. Lawrence River (Cornwall) and Peninsula Harbour.^{Footnote 57} 
• According to the ECCC Great Lakes Areas of Concern web page, projects in AOCs have enhanced water quality, restored fish and wildlife populations and habitats, improved management of municipal waste waters, and investigated and developed options to manage contaminated sediments in the AOCs.^{Footnote 58} 
• To date, over 900 restoration projects have been completed by ECCC and community partners in the AOCs.^{Footnote 59}
  • According to file review evidence, about one Gs&Cs project out of five in the sample included habitat restoration work, such as planting and restoration of buffer zones, wetlands, forests and riverbank or stream habitats. The program’s own internal analysis of all projects funded by the GLSF over the past five years indicates that roughly two in five projects (38%) were related to habitat restoration.   
  • Key informants indicated that the work conducted under GLAP has led to concrete results, for example, through Gs&Cs-funded projects that have reduced various types of pollution and improved habitats in AOCs. During the evaluation timeframe, BUI 14 on loss of fish and wildlife habitat has been re-designated as “not impaired” in the Nipigon AOC and is up for re-designation in the Bay of Quinte and St. Lawrence AOCs.
  • Just under half of stakeholders in the survey perceived that the work undertaken through GLAP has to a large extent achieved the outcomes of minimizing or eliminating pollution from identified sources (42%) and restoring habitats (48%). All or most of the remaining respondents (58% and 48% respectively) indicated some extent of achievement. There are no CESI indicators to provide empirical data on pollution levels or habitat restoration in the Great Lakes, however.

Intermediate Outcome 3: Activities of federal partners and stakeholders advance remedial actions, monitoring, outreach and engagement in each Area of Concern – Acceptable.

Activities of federal partners and stakeholders help to advance remedial actions in AOCs, for example, through their engagement on Remedial Action Plan (RAP) committees.

• GLAP governance mechanisms related to work on AOCs engage federal partners and stakeholders. The GLAP Workplan Review Team conducts an annual review of five‑year work plans submitted by federal departments, in consultation with the GLEC. In addition, Remedial Action Plan (RAP) committees, each with a federal project lead, maintain working-level links to community and provincial stakeholders to ensure that environmental needs at the AOC level are addressed.
• The file review indicated that funded projects targeted at AOCs have engaged various local partners, including municipalities, First Nations communities and other local organizations.
• Close to two-thirds of stakeholders completing the survey (62%) reported that this outcome has been achieved to a large extent, with the remaining 38% indicating to some extent.

Long-term Outcomes

Long-term Outcome 1: Beneficial uses are determined to be unimpaired and Areas of Concern are delisted – Acceptable.

In the evaluation timeframe, 17 beneficial uses were restored. Overall, since 1987, 54 beneficial uses out of a total of 146 across all 17 AOCs have been restored to “not impaired” status as of 2015. No AOCs were delisted during the five-year evaluation timeframe; however, in 2011 one AOC was designated as being in the recovery stage.

• FSDS target 3.3 (from November 2013) is to take federal actions to restore beneficial uses for delisting five Canadian AOCs and to reduce the number of impaired beneficial uses in the remaining AOCs by 25% by 2018.
• In the evaluation timeframe (from 2010 to 2015), 17 beneficial uses in AOCs were restored. Overall, since 1987, 54 out of 146 previously identified^{Footnote 60} beneficial uses across all 17 AOCs have been restored to “not impaired” status as of 2015.^{Footnote 61} The number of impaired uses has increased in only one AOC (Detroit River).^{Footnote 62}
  
• No AOCs were delisted during the five-year evaluation timeframe. However, as noted, one AOC was designated as being in the recovery stage in 2011 (Jackfish Bay in Lake Superior),^{Footnote 63} and ECCC has reported that remedial actions are expected to be completed in five more AOCs by 2019.
• Over one-third of stakeholders completing the survey (39%) indicated that this outcome has been achieved to a large extent, and 61% indicated to some extent.

Long-term Outcome 2: Canada’s international commitments related to Great Lakes Areas of Concern are met – Acceptable.

ECCC is meeting its commitments under the GLWQA, for example, through its GLAP work in AOCs, leadership in the implementation of the Agreement, preparation of 2014 Lakewide Management and Action Plan reports, and finalization of a Canada–US Biodiversity Conservation Strategy for Lake Superior.

• ECCC has continued its collaboration with the US and met commitments related to the GLWQA. For example, in addition to GLAP work in AOCs, the Department has continued to lead implementation of the GLWQA, prepared 2014 Lakewide Management and Action Plan reports, finalized a Canada–US Biodiversity Conservation Strategy for Lake Superior, and co-led the identification of the first set of proposed Canada–US chemicals of mutual concern.^{Footnote 64} 
• Key informants noted that the roles of the Parties are clearer under the 2012 GLWQA and that the indicators for BUIs are more specific.
• With respect to the 2007–2012 COA, ECCC met 176 of 189 commitments and continued efforts on the remaining commitments.^{Footnote 65} Similar findings on the 2014 COA are not yet available.
• Just over half of stakeholders completing the survey (52%) indicated that Canada's international commitments related to Great Lakes AOCs are being met to a large extent, with 44% indicating to some extent.

4.3.3 Great Lakes Sediment Remediation Projects: Progress to Date

As it is premature to assess the degree of outcome achievement for GLSRP, progress to date on the major project^{Footnote 66} at Randle Reef was examined. Design and planning for this project is now complete, with preliminary work on the seven-year project beginning in the fall of 2015. Construction of the containment facility in the first phase began in the spring of 2016, with final capping and completion of the project scheduled for 2022. The federal government is contributing $46.3 million of the total project cost of $138.9 million.

Randle Reef is an area of highly contaminated sediment located on the south shore of Hamilton Harbour in the western end of Lake Ontario, and is considered to be the largest and one of the more complex and highly contaminated sediment sites in the Great Lakes. Sediment remediation is required to reduce the environmental impacts of contaminants, including the polycyclic aromatic hydrocarbons (PAHs) and heavy metals located at this site. A shared responsibility model has been adopted with the Government of Canada, the Government of Ontario and the local community, participating equally in the design and implementation of a solution. This legacy site is a priority for remediation in the Hamilton Harbour Remedial Action Plan and under the COA.^{Footnote 67} 

The objective of the Randle Reef Sediment Remediation Project is to contribute to the improvement of environmental conditions in Hamilton Harbour and to assist in the delisting of the harbour as an Area of Concern. The development of the project has been ongoing since 1992, when the Remedial Action Plan was first submitted to the IJC.^{Footnote 68} Since then, the initiation of the project’s core activities has experienced a series of unanticipated delays related to technical studies, funding disputes, a change in the project lead (from the Hamilton Port Authority to ECCC), legal concerns, and finalizing the project design and estimated cost.^{Footnote 69}, ^{Footnote 70} These delays resulted in moderated goals and rescheduled timelines to address the project objectives in a feasible manner. The seven-year remediation project is now planned for completion in 2022. The project will be conducted in three phases: (1) construction of a 7.5‑hectare engineered containment facility (ECF) over the most highly contaminated sediment containment box (two years); (2) dredging and placement of additional contaminated sediment within the steel-walled ECF (two years); and (3) constructing a cap and isolation of remaining sediment on the containment structure (three years). A total of 675,000 cubic metres of sediment is being managed through the project.

A total of $138.9 million has been allocated to the project. This includes $46.3 million from each of the federal and Ontario governments, $14 million from the City of Hamilton, $14 million from US Steel Canada, $14 million from the Hamilton Port Authority, $2.3 million from the City of Burlington and $2 million from Halton Region.

The design and planning phase for the Randle Reef Remediation Project has been completed. ECCC issued a request for proposals for the first phase to build the steel containment walls in February 2014. The planned work was delayed by a year when the initial bids came in over budget, and the request for proposals was reissued in the spring of 2015. An ECCC official reported that preliminary work to reconstruct a harbour wall along the shoreline began in the fall of 2015. The shoreline wall is required to allow dredging of sediments in the second phase. The first phase of the ECF construction started in the spring of 2016.