4. Findings

This section presents the findings of this evaluation by evaluation issue (relevance and performance) and by the related evaluation questions.

For each evaluation question, a rating is provided based on an analysis of the evaluation findings. The rating statements and their significance are outlined below in Table 2. A summary of ratings for the evaluation questions is presented in Annex D.

Table 2. Definitions of Standard Rating Statements
Statement Definition
Acceptable The program has demonstrated that it has met the expectations with respect to the issue area.
Opportunity for Improvement The program has demonstrated that it has made adequate progress to meet the expectations with respect to the issue area, but continued improvement can still be made.
Attention Required The program has not demonstrated that it has made adequate progress to meet the expectations with respect to the issue area and attention is needed on a priority basis.
Not Applicable There is no expectation that the program would have addressed the evaluation issue.
Unable to Assess Insufficient evidence is available to support a rating.

4.1. Relevance

4.1.1. Continued Need for Program

Evaluation Issue: Relevance Rating
1. Is there a continued need for the program? Acceptable

There is a demonstrated need to restore and maintain the Great Lakes ecosystem. The Great Lakes have enormous environmental, social and economic importance and value to Canada. Current and emerging environmental concerns for the Great Lakes require ongoing attention from the Government of Canada. There is evidence of coordination of efforts to avoid duplication among the various jurisdictions involved in restoring and maintaining Great Lakes water quality.

4.1.2. Alignment with Federal Government Priorities

Evaluation Issue: Relevance Rating
2. Is the program aligned with federal government priorities? Acceptable

The Great Lakes program is aligned with federal government and ECCC priorities. The initiative was included in the 2010 Budget and is consistent with the Government of Canada’s expected outcome of a clean and healthy environment, the Federal Sustainable Development Strategy’s Goal 3 to maintain water quality and availability, and ECCC’s strategic outcome “Canada’s natural environment is conserved and restored for present and future generations.”

4.1.3. Alignment with Federal Roles and Responsibilities

Evaluation Issue: Relevance Rating
3. Is the program consistent with federal roles and responsibilities? Acceptable

The Great Lakes program is consistent with federal government roles and responsibilities as specified in relevant legislation and international commitments. A federal presence is necessary to ensure coordination among involved parties, which also is consistent with the relevant legislation. Stakeholders believe that delivery of the Great Lakes program is an appropriate role for the federal government to play in the restoration and maintenance of the Great Lakes basin ecosystem.

4.2. Performance – Efficiency and Economy

4.2.1. Program Design

Evaluation Issue: Performance – Efficiency and Economy Rating
4. Is the program design appropriate for achieving its intended outcomes? Acceptable

Overall, the design of the Great Lakes program is appropriate for achieving its intended outcomes: the designs of the GLNI, GLAP and GLSRP are logical; program structures, processes and science are aligned with the new GLWQA; a systematic process exists for re-designating BUIs and delisting AOCs; and Gs&Cs are felt to be an effective mechanism for engaging and supporting a variety of partners to help achieve program results. While the overall program design is sound, some concern was expressed regarding the need for a more clearly defined and timely approach and strategy for delisting AOCs and reallocating resources, as well as the adequacy of the program’s capacity to meet an expected increase in demand for science to support the GLWQA Annex Subcommittees.

4.2.2. Program Governance and Management

Evaluation Issue: Performance – Efficiency and Economy Rating
5. To what extent is the management and governance of the Great Lakes program clear, appropriate, and efficient for achieving expected results? Opportunity for Improvement

The management and governance mechanisms established in 2012 for the renewed GLWQA, including the Great Lakes Executive Committee (GLEC) and Annex Subcommittees, have set up binational processes to achieve a variety of objectives consistent with ECCC’s program outcomes and have led to a more open and transparent governance process. Despite these improvements, perceived challenges include communications and collaboration given the number of program initiatives and stakeholder organizations involved, and the fact that roles and responsibilities as defined in the GLWQA are not yet clearly understood by all stakeholders.

Governance Structure

Clarity of Roles and Responsibilities

Communications, Engagement and Collaboration with Stakeholders

4.2.3. Program Efficiency and Alternatives

Evaluation Issue: Performance – Efficiency and Economy Rating
6. Is the program undertaking specific activities and delivering products at the lowest possible cost? How could the efficiency of the program’s activities be improved? Are there alternative, more economical ways of delivering program outputs? Acceptable

A number of factors suggest that the Great Lakes program is efficient and cost-effective. The program has made efficiency improvements to the project application and reporting processes, and the Gs&Cs component has been successful at leveraging approximately three-quarters of project resources from sources other than ECCC. The combined administrative costs of the GLAP and GLNI programs are comparable to those for other ECCC Gs&Cs programs. In addition, most stakeholders consulted generally agree that Great Lakes initiatives are cost-effective.

Administrative Ratio

Performance Measurement

Evaluation Issue: Performance – Efficiency and Economy Rating
7. Are performance data being collected and reported? If so, is this information being used to inform senior management / decision makers? Attention Required

The program reports federally and provincially through the COA and binationally through the Canada–US GLWQA on progress in meeting commitments in the agreements. While there are logic models for GLNI, GLAP and the Sediment Remediation projects, there is no formal overall logic model and performance measurement strategy for the Great Lakes program. Program managers and scientists reported that data generally are adequate but that significant improvements could be made. They acknowledge that it is difficult to integrate and aggregate data from the various Great Lakes program initiatives.

4.3. Performance – Effectiveness

Evaluation Issue: Performance – Effectiveness Rating
8. To what extent have intended outcomes been achieved as a result of the program?
  • GLNI
  • GLAP
  • GLSRP
Acceptable
Acceptable
Unable to Assess

Evidence of outcomes achievedFootnote 33 was obtained through all lines of evidence and results are presented in this section by the intended outcomes specified in the logic models for GLNI and GLAP.

The survey of stakeholders provided ratings of the extent of achievement for all immediate, intermediate and longer-term outcomes. Results are presented for the percentage of survey respondents who perceived that a particular outcome has been achieved to a large extent (i.e., responded with a 4 or 5 on a 5‑point scale) or to some extent (i.e., responded with a 2 or 3 on the scale). It should be noted that because many committees and subcommittees were still actively working on key deliverables for GLNI, including targets to be set, it was too early to assess the effectiveness of some aspects of this program. According to the interview evidence, this may explain modest survey ratings of outcomes achievement, as much of the work is still in progress.

4.3.1 Great Lakes Nutrient Initiative

Objectives/Immediate Outcomes

Due to the fact that there are a large number of immediate/direct outcomes (12) in the GLNI logic model, the findings in this section are presented for the five GLNI objectives with which the immediate outcomes are associated. As detailed below, the findings indicate that the work being conducted as part of GLNI generally is on track and progress has been made on each of the five objectives.

Immediate Outcome 1: Measurement of phosphorus loads from urban and agricultural sources to identify and assess phosphorus discharges, inform decision making, and track and report progress toward achievement of phosphorus reduction targets – Acceptable.

Acceptable progress has been made toward achieving outcomes in the areas of science and monitoring for the measurement of phosphorus loads, including the development of inventories, models and baseline information about nutrients, and phosphorus reduction targets have been developed.

Immediate Outcome 2: Improved understanding of the influence of aquatic invasive species and other factors contributing to algae production, and of the impacts of toxic and nuisance algae on water quality and ecosystem health – Acceptable.

Research documented through GLWQA Annex Subcommittee reports has led to a better understanding of nutrient loadings as well as invasive species and other factors that affect water quality, ecosystem health and algae growth.

Immediate Outcome 3: Establish binationally agreed upon, science-based phosphorus load reduction targets Acceptable.

The Annex 4 Nutrients Subcommittee has summarized the research on nutrient loadings in Lake Erie from Canadian tributaries and recommended phosphorus loading targets for Lake Erie.

Approximately half of survey respondents indicated that the GLNI has achieved the following to a large extent: updated and scientifically defensible phosphorus targets for management purposes, to meet key milestones in the Canada–US GLWQA (50%); and ecosystem objectives and phosphorus targets to guide domestic and binational phosphorus management decisions (47%). Approximately half also indicated some extent of achievement for each outcome. Modest ratings of the degree of achievement of these outcomes are not unexpected considering that work in support of these outcomes was ongoing at the time of the survey, and that the survey asked whether the outcomes had already been achieved.

Immediate Outcome 4: Develop and assess policy options for reducing phosphorus discharges from agricultural and urban point and non-point sources Acceptable.

ECCC has completed an evaluation of policy options for reducing phosphorus discharges to Lake Erie, and efforts to further assess and refine these options are expected to continue as part of the development of Canada’s Domestic Action Plan.

Immediate Outcome 5: Develop a binational nearshore assessment and management framework Acceptable.

Acceptable progress has been made by the Annex 2 Lakewide Management Subcommittee to develop a binational nearshore assessment and management framework. A draft framework was completed in February 2016.

Intermediate Outcomes

Almost all stakeholders who completed the survey believe that at least some progress has been made toward achieving all GLNI intermediate outcomes, and few or no respondents think that progress has been made toward these outcomes to a large extent. Additional evidence for each intermediate outcome is presented below.

Intermediate Outcome 1: Implement pollution prevention and control measures to reduce phosphorus loadings Acceptable.

Acceptable progress has been made in developing phosphorus reduction targets, and measures are currently being developed in 2016 to achieve these targets.

Intermediate Outcome 2: Minimize the occurrence and impacts of toxic and nuisance algae Opportunity for Improvement.

Although some progress has been made over the past decades in reducing algae in Lake Erie, there is research evidence indicating that recent algae blooms are a result of excessive nutrient loadings and that there has been a levelling off or reversal of earlier reductions in nutrient loadings.

Intermediate Outcome 3: Canada and US establish and meet phosphorus load reduction targets for all Great Lakes Unable to Assess.

It is premature to assess the degree to which phosphorus reduction targets are being met because targets for Lake Erie have only recently been developed.

Intermediate Outcome 4: Improved understanding and management of the Great Lakes nearshore Acceptable.

Understanding of the nearshore has improved through the process of developing a nearshore assessment and management framework, and efforts to manage the nearshore will intensify once the framework has been approved in 2016.

Intermediate Outcome 5: Reduction in cumulative impacts of human activities in nearshore areas Unable to Assess.

It is premature to assess this outcome because the nearshore assessment and management framework has not yet been finalized.

4.3.2 Great Lakes Action Plan

Immediate Outcomes

Immediate Outcome 1: More effective/better integrated remedial actions in Areas of Concern – Acceptable.

Remedial actions in AOCs are being implemented through the development of Remedial Action Plans and implementation of funded projects focused on BUIs and AOCs with the involvement of local partners.

Immediate Outcome 2: Improved identification of environmental problems and progress in Areas of Concern Acceptable.

Identification of environmental problems and progress in AOCs is being achieved through the conduct of science and monitoring of ecosystem health as well as the preparation of Remedial Action Plans, Annual Workplans and Progress Reports for AOCs.

Immediate Outcome 3: Improved management and coordination of efforts to restore and maintain the Great Lakes Basin ecosystem Acceptable.

Management and coordination of efforts to restore the Great Lakes ecosystem have improved with the governance structure and processes of the 2012 GLWQA.

Intermediate Outcomes

Intermediate Outcomes 1 and 2: Pollution from identified sources is minimized or eliminated and habitats are restored in Areas of Concern Acceptable.

Since 1987, the environmental quality in Canada’s 17 AOCs has improved, with three AOCs delisted and two more designated as being in the recovery stage (one of which was so designated in 2011, within the evaluation timeframe). Projects in AOCs have had a number of benefits, including reducing pollution and restoring habitats.

Intermediate Outcome 3: Activities of federal partners and stakeholders advance remedial actions, monitoring, outreach and engagement in each Area of Concern Acceptable.

Activities of federal partners and stakeholders help to advance remedial actions in AOCs, for example, through their engagement on Remedial Action Plan (RAP) committees.

Long-term Outcomes

Long-term Outcome 1: Beneficial uses are determined to be unimpaired and Areas of Concern are delisted Acceptable.

In the evaluation timeframe, 17 beneficial uses were restored. Overall, since 1987, 54 beneficial uses out of a total of 146 across all 17 AOCs have been restored to “not impaired” status as of 2015. No AOCs were delisted during the five-year evaluation timeframe; however, in 2011 one AOC was designated as being in the recovery stage.

Long-term Outcome 2: Canada’s international commitments related to Great Lakes Areas of Concern are met Acceptable.

ECCC is meeting its commitments under the GLWQA, for example, through its GLAP work in AOCs, leadership in the implementation of the Agreement, preparation of 2014 Lakewide Management and Action Plan reports, and finalization of a Canada–US Biodiversity Conservation Strategy for Lake Superior.

4.3.3 Great Lakes Sediment Remediation Projects: Progress to Date

As it is premature to assess the degree of outcome achievement for GLSRP, progress to date on the major projectFootnote 66 at Randle Reef was examined. Design and planning for this project is now complete, with preliminary work on the seven-year project beginning in the fall of 2015. Construction of the containment facility in the first phase began in the spring of 2016, with final capping and completion of the project scheduled for 2022. The federal government is contributing $46.3 million of the total project cost of $138.9 million.

Randle Reef is an area of highly contaminated sediment located on the south shore of Hamilton Harbour in the western end of Lake Ontario, and is considered to be the largest and one of the more complex and highly contaminated sediment sites in the Great Lakes. Sediment remediation is required to reduce the environmental impacts of contaminants, including the polycyclic aromatic hydrocarbons (PAHs) and heavy metals located at this site. A shared responsibility model has been adopted with the Government of Canada, the Government of Ontario and the local community, participating equally in the design and implementation of a solution. This legacy site is a priority for remediation in the Hamilton Harbour Remedial Action Plan and under the COA.Footnote 67 

The objective of the Randle Reef Sediment Remediation Project is to contribute to the improvement of environmental conditions in Hamilton Harbour and to assist in the delisting of the harbour as an Area of Concern. The development of the project has been ongoing since 1992, when the Remedial Action Plan was first submitted to the IJC.Footnote 68 Since then, the initiation of the project’s core activities has experienced a series of unanticipated delays related to technical studies, funding disputes, a change in the project lead (from the Hamilton Port Authority to ECCC), legal concerns, and finalizing the project design and estimated cost.Footnote 69, Footnote 70 These delays resulted in moderated goals and rescheduled timelines to address the project objectives in a feasible manner. The seven-year remediation project is now planned for completion in 2022. The project will be conducted in three phases: (1) construction of a 7.5‑hectare engineered containment facility (ECF) over the most highly contaminated sediment containment box (two years); (2) dredging and placement of additional contaminated sediment within the steel-walled ECF (two years); and (3) constructing a cap and isolation of remaining sediment on the containment structure (three years). A total of 675,000 cubic metres of sediment is being managed through the project.

A total of $138.9 million has been allocated to the project. This includes $46.3 million from each of the federal and Ontario governments, $14 million from the City of Hamilton, $14 million from US Steel Canada, $14 million from the Hamilton Port Authority, $2.3 million from the City of Burlington and $2 million from Halton Region.

The design and planning phase for the Randle Reef Remediation Project has been completed. ECCC issued a request for proposals for the first phase to build the steel containment walls in February 2014. The planned work was delayed by a year when the initial bids came in over budget, and the request for proposals was reissued in the spring of 2015. An ECCC official reported that preliminary work to reconstruct a harbour wall along the shoreline began in the fall of 2015. The shoreline wall is required to allow dredging of sediments in the second phase. The first phase of the ECF construction started in the spring of 2016.

Page details

Date modified: