4. Findings

This section presents the findings of this evaluation by evaluation issue (relevance and performance) and by the related evaluation questions.

For each evaluation question, a rating is provided based on the evaluation findings. The rating statements and their significance are outlined below in Table 2. A summary of ratings for the evaluation questions is presented in Annex B.

4.1. Relevance

4.1.1. Continued need for program

There is a demonstrated need to address water quality issues in the Lake Winnipeg Basin and to better understand emerging issues that are impacting water quality and the lake’s response to interventions.

• Lake Winnipeg is the world’s 10th-largest freshwater lake and the world’s third largest reservoir^{Footnote 1} . More than 7 million people live in the Lake Winnipeg basin which covers 1,000,000 km2 between Canada and the United States^{Footnote 2}. The lake plays a critical role in tourism, recreation, commercial and sport fisheries, and hydroelectric generation in Manitoba.
• The basin is an important and complex watershed with significant water quality challenges related to excessive nutrients which have a negative impact on the ecosystem and economy. Available research and literature^{Footnote 3} clearly demonstrate the water quality challenges in the Lake Winnipeg Basin, including the following:
  • Eutrophication^{Footnote 4} is a key issue for this ecosystem. Excessive nutrients contribute to the growth of huge tracts of algae, which rob the lake of oxygen, clog fishing nets, foul beaches, and can produce harmful toxins.
  • There are multiple domestic and transboundary sources of nutrients, including agriculture sources and wastewater treatment plants, among others. The loss or degradation of wetlands, which are instrumental in filtering nutrients that flow off the land and into the lake, is exacerbating the issue.
  • Excessive nutrient levels have a negative impact on fish and fish health, as well as recreation and tourism. The Lake Winnipeg ecosystem supports an annual $50 million freshwater fishery and a $110 million recreation and tourism industry^{Footnote 5}.
• Interviewees agreed that there is a continued need to address persistent and ongoing water quality issues in the Lake Winnipeg Basin. The threats to the lake are evolving and not fully understood, requiring further knowledge and understanding of interactions and the impacts of interventions.
• In terms of stewardship funding, according to LWBSF funding applicants there are few alternative sources of funding. Most surveyed funding recipients (64%) did not believe their project would have been carried out if their organization had not received funding from the LWBSF.

4.1.2. Alignment with federal government priorities

Evaluation issue: relevance	Rating
2. Is the program aligned with federal government priorities?	Acceptable

LWBI objectives align well with federal government priorities to protect freshwater as a ”precious resource” and with the strategic outcome of ECCC to conserve and restore Canada’s natural environment.

Federal priorities

• The objectives of the LWBI are aligned with federal government priorities related to water quality and sustainability as outlined in federal policy and budgetary commitments, as well as commitments made in the Federal Sustainable Development Strategy.^{Footnote 6}
• The 2015 Ministerial Mandate Letter sets out departmental priorities and includes a focus on protecting the country’s freshwater sources as “a precious resource that deserves protection and careful stewardship.” The Minister of ECCC is mandated to work in collaboration with the Minister of Fisheries, Oceans and the Canadian Coast Guard to renew the federal commitment to protect the Lake Winnipeg Basin.

ECCC strategic outcomes

• LWBI is aligned to the strategic outcome of ECCC to ensure that Canada’s natural environment is conserved and restored for present and future generations. The LWBI falls under Program 1.3 Sustainable Ecosystems (Sub-Program 1.3.7) which “aims to sustain Canada’s ecosystems over the long term by providing Canadians, their governments and the private sector with the environmental information and tools required to incorporate social, economic and environmental considerations into decision-making and action.”

4.1.3. Alignment with federal roles and responsibilities

Evaluation issue: relevance	Rating
3. Is the program consistent with federal roles and responsibilities?	Acceptable

Water quality is an area of shared federal, provincial and territorial responsibility. Federal involvement in Lake Winnipeg is consistent with the transboundary nature of the basin and the national importance of Lake Winnipeg as a large and significant freshwater body. ECCC’s provision of science expertise fills an important role not addressed by other stakeholders. While the shared jurisdiction and involvement of multiple organizations in Lake Winnipeg introduces the potential for duplication, there are mechanisms in place that help mitigate this issue.

• The federal government has a legislated role in the management of transboundary waters that are of national interest. As Canada’s 6th largest freshwater body, Lake Winnipeg is a freshwater resource of national importance. National leadership for water management is assigned to the Minister of the Environment in the Department of the Environment Act. While Lake Winnipeg itself resides fully within the Province of Manitoba, the Lake Winnipeg Basin covers four provinces and four states and over half of the nutrients that flow into Lake Winnipeg originate from outside Manitoba’s borders. ECCC, with the support of the International Joint Commission, is responsible for delivering on the federal obligation under the International Boundary Waters Treaty^{Footnote 7} to support the management of the basin. The federal government is currently a signatory/participant in key inter-jurisdictional agreements pertaining to Lake Winnipeg including the CA–MB MOU and the Lake Friendly Accord.
• ECCC is also responsible for administering and enforcing the pollution prevention provisions under the Fisheries Act, which prohibit the deposit of deleterious substances into fish habitat.
• There are a variety of organizations that focus on issues in Lake Winnipeg, including universities (e.g., the University of Manitoba), other federal departments (e.g., AAFC, DFO), the provinces, multi-jurisdictional organizations (water management boards) and NGOs (e.g., the Red River Basin Commission, RBC Bluewater). Key informants acknowledge that the policy and program landscape is complex, but they do not view activities conducted by the LWBI as duplicating the work of other organizations. Coordination mechanisms such as the Lake Friendly Accord, the LWRC, inter-jurisdictional water management boards and the CA–MB MOU help support coordinated action and mitigate duplication.

4.2. Performance – effectiveness

4. To what extent have intended outcomes been achieved as a result of the program?

Direct Outcomes

Evaluation issue: performance – effectiveness	Rating
i. Effective actions by individuals and organizations	Acceptable
ii. Groups are aware of and apply for funding from the LWBSF	Opportunity for improvement
iii. Increased stakeholder collaboration and coordination of federal actions	Acceptable
iv. State of the Lake Indicators are established and tracked	Opportunity for improvement
v. Use of predictive capability to model nutrient scenarios	Acceptable
vi. Increased scientific knowledge and data for decision-making	Opportunity for improvement

Intermediate Outcomes

Evaluation issue: performance – effectiveness	Rating
i. Reduced nutrients in Lake Winnipeg basin	Attention required
ii. Manitoba/transboundary bodies establish nutrient objectives	Acceptable

Final Outcome

Evaluation issue: performance – effectiveness	Rating
Improvement of the ecological health of Lake Winnipeg	Unable to Assess

Direct outcome 1 : Effective action by individuals, farmers, communities, and organizations to manage nutrients – Acceptable

The program has measures in place to prioritize funding to LWBSF projects which, based on available knowledge, are likely to have the greatest impact on managing nutrients. The majority of completed projects contributed to some level of sustainability over time through sharing of results and continued work with partners.

Selection and Funding of Effective Actions

• In an effort to maximize the effectiveness of projects funded by the LWBSF, the majority of LWBSF funds (70%) are directed to ‘on the ground’ actions.
• Further, LWBSF project proposals are assessed against clearly established priorities and criteria that are based on available science. Examples of efforts aimed at increasing the effectiveness of actions funded by the LWBSF include:
  • Targeting locations known to be a source of significant, direct nutrient loading to Lake Winnipeg. The majority of LWBSF project funds (70%) are directed to actions or research activities in the Red River/Assiniboine watersheds since the Red River is the highest contributing tributary in the Basin, accounting for one-half of nutrient inputs into Lake Winnipeg.
  • Focussing on addressing point sources and wetlands and shoreline restoration. Agricultural and major urban sources, together with spring flooding, are known to be key factors responsible for the increase in phosphorus loading in the lake^{Footnote 8}. Wetland creation, restoration and preservation play an important role in both intercepting runoff from the land before it reaches open water and in flood protection. The majority of projects (two-thirds) addressed issues related to water retention on the land/wetland restoration, while a minority were related to wastewater treatment and point sources. For the 29 projects completed between 2012-2013 and 2015-2016, LWBSF records identified:
    • 1,376 hectares of wetlands restored or protected;
    • 11,135 metres of shoreline restored or protected;
    • 102,340 trees/shrubs planted
    • 9,404 hectares of land with surface water retained; and
    • 3,597 head of livestock restricted from accessing streams/shoreline
• Both internal and external interviewees agreed that the technical review and PAC review of funding applications is an effective process for selecting projects, as it is science-driven, focuses on priority areas, and is fair and transparent.

Sustainability of projects beyond the funding period

• Another consideration in the assessment of project proposals is whether the project will provide ongoing benefits to the lake and watershed. Most LWBSF funding recipients indicated that their projects are having broader impacts beyond the stewardship action for which they were funded. For instance, funding recipients agreed or strongly agreed that results from the project (to date) have been shared with other groups or organizations (93%) and partnerships developed during the project are continuing with another project or next-phase activities (86%).
• Most funding recipients whose projects were completed at the time of the survey (9 out of 14) indicated that some or all project activities continued after the funding from LWBSF ended. Examples of the sustainability of impacts identified in the file review included
  • ongoing awareness and education initiatives such as promotion of project activities in the media, events or scientific publications or integration of material in a school curriculum; and
  • continuation of project activities through integration into an existing initiative or replication of the project in another setting or with new partners.

Direct outcome 2: Lake Winnipeg groups are aware of and apply for funding from the LWBSF – Opportunity for improvement.

The program is well known in the Lake Winnipeg Basin and there is a strong demand for funding to conduct high-quality stewardship projects. However, Indigenous groups are not well represented as LWBSF funding recipients or partners in these projects.

• The LWBO conducts outreach to potential LWBSF applicants via mainstream media, social media, email and trade shows/conferences, although close to half of surveyed applicants (49%) reported first becoming aware of the LWBSF through word of mouth.
• All funds for Phase II of the program have been fully subscribed. Over three funding rounds under LWBSF Phase II, 178 Letters of Intent and 137 applications were received (excluding applications that were withdrawn or deemed ineligible) of which 49 were funded (success rate of 37% or $4.4 million allocated out of $20.8 million applied for).
• Almost all program applicants (89%) agreed that information about the LWBSF application process was easy to find. Applicant organizations are typically university/research organizations, ENGOs, and conservation districts/watershed associations.
• Although 15 First Nations reside around Lake Winnipeg, LWBI program representatives noted that the engagement of Indigenous groups in the LWBI is in the early stages and communities are not yet well represented among LWBSF funding recipients and partners

Direct outcome 3: Increased partner and stakeholder collaboration at basin and sub-basin levels / Increased coherence and coordination of federal actions relative to Lake Winnipeg and its basins — Acceptable.

ECCC’s leadership and participation in the LWBI has fostered increased levels of collaboration and coordination among stakeholders participating in each of the program’s three pillars.

Science

• A recent management review of the CA–MB MOU^{Footnote 9} concluded that the priorities for science activities had been collaboratively determined and activities within federal and provincial departments were better coordinated as a result of the MOU. ECCC interviewees confirmed that LWBI annual science workplan objectives and priorities are informed by many factors including Manitoba’s research activities, science in the US and universities, issues identified by local stakeholders, and the federal mandate.
• LWBI scientific collaboration benefits greatly from the work of the LWRC, a non-profit research organization with a mandate to coordinate research in the Lake Winnipeg Basin. ECCC scientists are active members of the LWRC, which also includes representatives from government, educational and other agencies and organizations. The LWRC facilitates sharing of results among scientists working in the basin through annual science meetings and posting of publications.

Stewardship

• The LWBSF selection criteria and requirement for matching funding (at least one-third of total project costs must come from alternate sources) encourage collaborations at the project level. An analysis of the project files and survey data confirm high levels of partner collaboration:
  • On average, projects in each funding round have involved four to six partners (consistent with the program’s target of five partners, including ECCC).
  • Partners are drawn from all sectors, but have a focus on provincial and territorial governments, individuals and conservation districts/watershed associations, municipal governments, and academic institutions and ENGOs. Just 4% of projects included Indigenous groups or communities.
  • Most projects (61%) led to the development of partnerships that were new.
  • A large majority of funding recipients (74%) agreed or strongly agreed that the project contributed to greater collaboration among stakeholders in the Lake Winnipeg Basin.
• To increase partner and stakeholder collaboration, the program has held two LWBSF symposiums during the evaluation timeframe that bring together project recipients to discuss and share their work.^{Footnote 10}

Transboundary partnerships

• As described in section 2.2, the focus of the program’s work under this pillar is on supporting collaboration and ensuring a coherent, coordinated approach among the various jurisdictions involved in the Lake Winnipeg Basin. Examples of the program’s contributions in this area include the following:
  • Engagement of the province and federal departments (AAFC, INAC, DFO) in twice yearly CA–MB MOU Steering Committee meetings. The review of documents and interviews with program representatives and external stakeholders confirmed that meetings are held regularly, are productive and have a high level of engagement;
  • LWBI representation and advocacy within international water governance bodies such as the Red River Water Board and Souris River Water Board; and
  • ECCC was one of the first signatories of the Lake Friendly Accord, an initiative of Manitoba’s Department of Sustainable Development and the South Basin Mayors and Reeves. The Lake Friendly Accord document is a declaration of support by signatories to take action to reduce nutrient loading and improve water quality in the Lake Winnipeg basin. ECCC staff participated in the Lake Friendly Steering Committee overseeing the implementation of the Accord, as well as working groups related to Governance and Science and Research.
• The transboundary partnerships were universally seen by internal and external interviewees to be useful and important, leading to beneficial information sharing and opportunities for collaboration and coordination of efforts. For instance, interviewees confirmed the value of the CA–MB MOU and the MOU Steering Committee which were both viewed very positively and described as a “strong piece of work”, and “effective.” This is consistent with the 2014 management review of the MOU, in which most Steering Committee members agreed that the MOU has been effective and improvements in information sharing were noted.
• The benefits of positive relationships developed through the transboundary partnerships pillar extended beyond the scope of LWBI planned activities. For instance, collaborations during the study period allowed the LWBI to draw on the experience and expertise of stakeholders to develop a strategy to address the emergence of zebra mussels in Lake Winnipeg.
• Ensuring representation of the LWBI in transboundary water governance bodies was viewed as important by key informants, in order to raise the profile of the issues specific to Lake Winnipeg within these organizations that have a broader mandate.

Direct outcome 4: State of the Lake indicators are established and progress against them is tracked — Opportunity for improvement.

Development of State of the Lake Indicators aimed at providing greater insight into the status and trends of Lake Winnipeg’s water quality and aquatic ecosystem health remains in the early stages and is delayed from target timelines identified in the CA–MB MOU Science Subsidiary Arrangement.

• In 2011, the State of the Lake report was released by ECCC and Manitoba^{Footnote 11}. With growing concerns about water quality in the lake, the report was the first attempt to summarize scientific knowledge about the aquatic ecosystem of Lake Winnipeg as it pertains to eutrophication. The report reflects changes in the physical, chemical and biological characteristics of Lake Winnipeg and also discusses current issues such as algae blooms and toxins, aquatic species at risk, and risks posed by climate change^{Footnote 12}. An update of the State of the Lake Indicators was a commitment of the CA–MB MOU Science Subsidiary Arrangement that set out a plan to “coordinate federal and provincial science to identify, test and validate appropriate indicators of aquatic ecosystem health for Lake Winnipeg and its Basin.” Appropriate indicators were to be adopted to monitor the environmental, social and economic factors of interest to the general public and stakeholders and allow the efficacy of current nutrient objectives to be evaluated. A report was to be submitted in 2016 on the status and trends in water quality and aquatic ecosystem health in the Lake Winnipeg Basin.
• Development of the State of the Lake Indicators is delayed due to a lack of capacity at the provincial level for this work during the period under study. However, some recent progress is notable, including creation of an Indicator Steering Committee with staff from ECCC and Manitoba’s Department of Sustainable Development; development of draft indicators and a process for further developing and publishing indicators for Lake Winnipeg and its watershed; and a draft table of contents and reporting plan (technical and summary reporting, eventually available online).
• According to key informants, monitoring data to assess progress on nutrient loading are in the early stages and there is likely a need for more data to support this requirement.

Direct outcome 5: Use of predictive capability to model nutrient scenarios — Acceptable.

ECCC predictive modelling of nutrients in the Lake Winnipeg Basin has been enhanced and is used to support work by Manitoba to understand optimal nutrient reduction scenarios in key tributaries.

• The Water Quality Analysis Simulation Program (WASP) model has been used by scientists for several decades to set nutrient target loads for various water bodies. Since 2010, ECCC has applied the WASP model to develop a eutrophication model for Lake Winnipeg that is calibrated for conditions in the basin, including major nutrient and algal dynamics. An iteration of the Lake Winnipeg eutrophication model was delivered to the Manitoba Department of Sustainable Development in fall 2013. The model was extended by ECCC and Manitoba scientists to incorporate three additional years of water quality data up to 2010 to strengthen the simulations by capturing more variability and water residence cycles of the lake. ECCC and Manitoba scientists used the model to simulate nutrient reduction scenarios and to develop draft nutrient loading targets for Lake Winnipeg’s main tributaries^{Footnote 13}. The optimal nutrient reduction scenario established an annual total phosphorus load target for Lake Winnipeg (4,850 tonnes (4,850K kilograms) of total phosphorus per year) with targeted loads for each of the main four tributaries (4,250 tonnes) and smaller tributaries (600 tonnes). This would require a decrease of approximately 37% from current loadings based on the most recent estimates of average annual phosphorus loads available^{Footnote 14}. Parallel targets were also established for nitrogen.^{Footnote 15}
• According to documentary sources and the views of ECCC and Government of Manitoba key informants, the WASP eutrophication model for Lake Winnipeg is the best available predictive model to accurately model eutrophication in a complex aquatic ecosystem such as Lake Winnipeg. External users agreed that the work is of high quality and essential to the development of nutrient loading targets by the province. However, there are known limitations of the model.^{Footnote 16} Specifically, there is a lack of certainty about the relative contribution of various factors to nutrient loading (and, therefore, challenges in determining the most effective strategies for managing nutrients), and continued work is required to refine the model in order to account for evolving conditions, including invasive species and climate change–related impacts on seasonal flooding.

Direct outcome 6: Increased scientific knowledge and data to inform and support decision making^{Footnote 17} — Opportunity for improvement.

Good progress is being made on scientific commitments identified for Phase II and ECCC scientists have published numerous articles in peer-reviewed journals about issues facing Lake Winnipeg, however, there is a continued need to better understand the effectiveness of nutrient management practices in the basin to identify actions that will have greater impacts. While scientific collaboration and some sharing of data and scientific knowledge occur, more could be done to improve the dissemination of research findings and water quality data.

• Documents reviewed indicate that there has been good progress toward the scientific commitments identified under the LWBI Phase II. The Science Plan for Phase II had the objective of “focusing on watershed research and monitoring to help further target actions to improve the lake and to measure the performance of actions taken to improve water quality.” Key science projects have included in-lake research activities to support development of predictive eutrophication models for Lake Winnipeg to inform ecologically relevant nutrient objectives, and monitoring research to understand the lake and watershed conditions and factors influencing nutrient loading.
• While progress is being made toward an improved understanding of the strategies for reducing nutrients in the basin, many internal and external key informants noted that additional evidence is needed to better understand the effectiveness of nutrient management practices in the basin and help identify actions that will have more significant impacts.
• Scientific findings are disseminated through science workshops hosted by the LWRC, in presentations to scientific audiences, and through professional networks. Water quality monitoring data collected by ECCC are shared with the International Red River Board, other water management boards, Manitoba, and internal ECCC researchers. ECCC scientists also publish their findings in peer-reviewed journals. In total, according to program performance measures, twenty-nine articles related to issues facing Lake Winnipeg were published by ECCC scientists in scientific journals during the four years under study.
• The case study on the LWBIN identified a need to improve the dissemination of data and scientific research about Lake Winnipeg. LWBIN, the web-based data and information portal designed by ECCC, is intended to facilitate sharing of water quality data and other research related to the basin. The LWBIN was transferred from ECCC to the University of Manitoba during the study period, with the provision of funding to continue to enhance the functionality of and access to the portal. While no data are available on the use of the portal, the case study of the LWBIN and the views of interviewees on the effectiveness of the portal as a dissemination channel were mixed. The LWBIN is not yet fully operational and there are some logistical challenges (e.g., restrictions on access to some data sources) which limit the extent to which researchers are able to populate their findings and data in the portal. ECCC water quality monitoring data are available through the LWBIN, however, the most current data available in the portal are from 2011 (but more recent data can be accessed by direct request to the department).
• ECCC program representatives agreed that the LWBI could be more proactive about disseminating scientific findings to other stakeholders in the basin, particularly to stakeholder groups who develop regulations related to nutrient release or who are responsible for the implementation of recommended management practices.

Intermediate outcome 1: Reduced nutrient loading in the Lake Winnipeg Basin — Attention required.

Information is not available to assess the degree to which the actions of various stakeholders are impacting nutrient loadings in the basin. While estimates of reductions from LWBSF projects are on track to meet their targets and are substantially greater than in Phase I, the level of nutrient reductions they deliver is extremely small in relation to total phosphorus estimated to be entering the lake, with total loading reductions from all Phase II projects over five years estimated at less than 1% of annual nutrient loads.

• Recent information on the level of nutrient loadings in the Lake Winnipeg Basin is not available, and as a result it is not possible to understand the degree to which efforts by other stakeholders in the basin may be having an impact on annual nutrient loadings. The most current information available on loadings comes from the State of Lake Winnipeg Report: 1999–2007, which was published in 2011, and indicated that the total phosphorus load for Lake Winnipeg was, on average, 7,655,000 kilograms annually (with variability each year depending on precipitation and associated river flows).
• The LWBSF funded projects include a focus on delivering nutrient reductions, and phosphorus reduction estimates for the projects are calculated using a desk top method provided by ECCC^{Footnote 18}. While the nutrient reductions achieved through the LWBSF are important and a step in the right direction, they are extremely small in relation to the annual phosphorus loading of the lake. The estimate of total LWBSF reductions achieved over five years will represent less than 1% of one year of loadings.
• During the first four years of Phase II of the LWBSF (2012–2013 to 2015–2016), funded projects are estimated to have reduced or diverted a total of 37,200 kilograms of phosphorus. This amount is on track to meet or exceed the LWBSF target of 10,800 kilograms/year (or 54,000 kilograms in total) when the remaining projects are completed in 2016–2017. The amount of phosphorus reductions delivered by Phase II projects is substantially greater than those for Phase I, which were estimated to be 6,500 kilograms in total over four years (average of 1,600 kilograms per year).
• Note that the estimates of nutrient reductions to date have come from just 10 out of the 29 projects that were funded and completed during the period under study, with over one-half of the reductions being contributed by one project (decommissioning of a municipal sewage lagoon site).

Intermediate outcome 2: The province of Manitoba/transboundary management bodies establish nutrient objectives — Acceptable.

While not directly the responsibility of ECCC, draft nutrient objectives for Lake Winnipeg have been established by the Province of Manitoba, and work on the development of nutrient objectives is being carried out by two transboundary water management bodies that impact the basin.

Province of Manitoba

• As Lake Winnipeg lies within provincial boundaries, the Province of Manitoba is the jurisdiction responsible for establishing the lake’s nutrient objectives. Transboundary water management bodies are responsible for establishing nutrient objectives for shared waters such as the Red River.
• Provincial nutrient objectives for Lake Winnipeg are based on scientific research, including the historical research conducted by Bunting et al. (2011)^{Footnote 19} which examined historical water quality conditions in Lake Winnipeg since the early 1800s. The research documented increases in phosphorus concentrations in the south basin of the lake to approximately 0.05 micrograms/litre (μg/L) between 1900 and 1990 and further acceleration of eutrophication between 1990 and 2006 with phosphorus concentrations reaching 0.10 μg/L. The authors recommended a 50% reduction in phosphorus influx to pre-1990 levels to avoid future toxic impacts. Based on this research, Manitoba’s (unpublished) phosphorus reduction goal is to reduce phosphorus concentrations in the lake to pre-1990 levels of approximately 0.05 μg/L.^{Footnote 20}

Transboundary water management bodies

• Two of the four transboundary water management bodies in the Lake Winnipeg Basin have made progress on nutrient objectives related to tributaries within the basin:
  • The Prairie Provinces Water Board (PPWB) ratified Interprovincial Water Quality Objectives^{Footnote 21} in 2015 that included thresholds for nutrients – including water quality parameters for phosphorus and nitrogen – for 12 transboundary rivers in Alberta, Saskatchewan and Manitoba.
  • The IRRB has established water quality objectives for five variables,^{Footnote 22} although none of these relate to nutrient levels. Objectives related to nutrient levels are currently under development and are expected to be finalized in 2017 or 2018.

Final outcome: Improvement to the ecological health of Lake Winnipeg, namely: Reduction in the magnitude and extent of harmful algal blooms; Reduced beach advisories and improved water quality for recreation; Restoration of the ecological integrity of Lake Winnipeg; A sustainable fishery — Unable to assess.

There are insufficient data to fully assess improvements to the ecological health of Lake Winnipeg, There is widespread agreement, however, that the ecological integrity of the lake and the basin has not improved significantly during the period under study based on efforts to date and the impact of a number of factors such as invasive species, weather events linked to climate change and the recirculation of nutrients released from lake sediment.

• Currently available indicators and supporting data to measure long-term improvements in the ecological health of Lake Winnipeg are limited due in part to delays in establishing State of the Lake Indicators. While the annual number of algal advisories issued for Lake Winnipeg beaches decreased over the four years under study (from 10 advisories in 2012-13 to 1 in 2015-16), data on other indicators such as the prevalence of toxic blooms and an indicator on ecological integrity of the lake are not available.
• The evaluation evidence suggests that the LWBI is unlikely to have resulted in detectable improvements to the ecological health of Lake Winnipeg. Scientific research indicates significant and negative impacts on water quality due to factors such as the presence of invasive species, weather events/seasonal flooding linked to climate change and recirculation of nutrients (nutrients that are released from the sediment within the lake).
• At this point in time the program is having only a very minimal impact in terms of the actual nutrient reductions realized through on-the-ground activities and investment in science and transboundary partnerships is only likely to translate into improvements in water quality in the longer term.

Unintended outcomes: Have there been any unintended (positive or negative) outcomes? — Acceptable.

Unintended/unexpected outcomes of the LWBI are few, but mostly positive.

• The review of project annual reports revealed an example of an unexpected positive non-environmental outcome. Namely, one funding recipient noted that project activities improved profitability for the livestock sector, which increased investment in infrastructure to improve pasture management and sustainability more broadly.
• The case studies identified positive unexpected outcomes, particularly regarding capacity building:
  • Support for the operation of the MV Namao for offshore work helped create the capacity and experience within the LWRC to also add the operation of a nearshore vessel as part of the research and monitoring plan on the lake.
  • The information sharing and collaborative network of the LWBIN has helped to add a community monitoring element, including citizen science groups and GIS mapping.

4.3. Performance – efficiency and economy

4.3.1. Program design

Evaluation issue: performance – efficiency and economy	Rating
5. Is the program design appropriate for achieving its intended outcomes?	Opportunity for improvement

The science, stewardship and transboundary partnerships pillars remain relevant, and the interrelationships among the organizations delivering the program components are effective for establishing a foundation to address issues in the lake. While the current LWBI program design is generally consistent with program objectives, there appears to be a disconnect between the program design and the program’s expected outcomes as portrayed in the logic model, as the program’s current activities fall short in terms of delivering any significant actions that address the logic model’s Intermediate outcome of “reduced nutrients in the Lake Winnipeg Basin.”

• The design of the LWBI is logical, science-based and consistent with the objectives of the program as identified in its foundational documents which emphasize:
  • establishing a long-term, collaborative and coordinated approach to support the sustainability of the lake; and
  • improved science-based understanding of the dynamics of Lake Winnipeg and its Basin for more informed decision-making.
• Internal stakeholders agreed that the science, stewardship and transboundary partnership pillars continue to be appropriate areas of focus for the LWBI. The three pillars each play an important role in establishing the foundation needed to address the longstanding water quality issues in the lake. Further, there appear to be beneficial interrelationships among the pillars. The development of a science plan to identify and address gaps in data is leading to relevant scientific evidence to inform funding of stewardship projects; stewardship projects also play a role in contributing to scientific knowledge and an increased awareness among stakeholders of the issues impacting the basin; and transboundary partnerships encourage the collaboration necessary to coordinate both science and stewardship activities.
• While the current LWBI program design is generally consistent with program objectives, there appears to be a disconnect between the program’s design and its expected outcomes as portrayed in the logic model. The program’s design is not fully reflected in the logic model and it falls short in terms of delivering any significant actions that address the logic model’s Intermediate outcome of “reduced nutrients in the Lake Winnipeg Basin.” This in turn limits visible progress toward the longer term outcomes of improved ecological health for the lake. The final outcome represents an important aspirational objective, but requires the collective efforts of other stakeholders and jurisdictions or introduction of more direct measures and program alternatives to be achieved.
• Most internal key informants were comfortable with the allocation of resources among the three program pillars.^{Footnote 23} However, some interviewees cautioned against any further reductions to the science pillar (as occurred between Phase I and Phase II),^{Footnote 24} particularly given that many important questions remain concerning the effectiveness of management practices and the impact of emerging issues such as zebra mussels. Currently, the level of funding to science within the LWBI is proportionately higher than other ECCC freshwater programs.^{Footnote 25} Others in the external stakeholder community, however, were of the opinion that LWBSF funding for stewardship is under-resourced.

4.3.2. Program governance and management

Evaluation issue: performance – efficiency and economy	Rating
6. To what extent is the governance structure clear and appropriate?	Opportunity for improvement

The LWBI leads or participates in multiple governance structures and roles and responsibilities of the various players are clear. Engagement of Indigenous groups in governance is in the early stages, but more could be done to ensure their consistent and coordinated participation.

• As described in section 2.3, the LWBI governance structure involves multiple internal federal (the LWBI Executive Committee, Transboundary Connections Calls) and multi-jurisdictional committees (CA–MB MOU Steering Committee). Additionally, the LWBSF engages scientific experts and other stakeholders in Technical and Public Advisory Committees to evaluate applications for funding.
• Internal and external interviewees indicated that the structures are effective for coordinating the various stakeholders working on the issues surrounding the Lake Winnipeg Basin. The roles and responsibilities, including for the management of G&C and science resources, are generally understood by all relevant parties. Stakeholder diagrams produced by the LWBO help provide clarity of relationships.
• Many interviewees stated that information sharing is the greatest benefit of collaboration among stakeholders. This includes access to the expertise of other departments such as best management practices on agricultural land from AAFC and knowledge of invasive species from DFO (given the emergence of zebra mussels in the basin).
• Although the Lake Winnipeg Basin is expansive and involves many different stakeholder groups, for the most part, the structures appear to involve most relevant stakeholders, and include cross-membership among the committees. As previously noted, one exception is the participation of Indigenous groups, whose input is not currently integrated in a coordinated manner. The recently established Lake Winnipeg Indigenous Collective^{Footnote 26} was identified as a positive step forward in Indigenous stakeholder engagement.
• Collaboration with internal ECCC partners (such as other freshwater lake programs) and other relevant federal departments (DFO, AAFC, INAC) is working well according to most internal key informants, although some would like to see more formal involvement and coordination from these other federal departments.

4.3.3. Program delivery

Evaluation issue: performance – efficiency and economy	Rating
7. Is the program implemented in an efficient and economical manner?	Acceptable

The evaluation found that the LWBI is a well-managed program and includes a number of practices that contribute to efficient activities. LWBSF funding applicants were generally satisfied with the delivery of the program.

• Here are some examples of ways the program enhances the efficiency of delivery:
  • use of collaborative mechanisms such as the LWRC and the annual LWBSF symposium to avoid duplication, create synergies and ensure that future investments are purposeful;
  • leveraging learning within ECCC across freshwater lake scientific programs; and
  • leveraging support from other organizations for LWBSF-funded projects, including volunteer efforts (for LWBSF projects, partner contributions are leveraged at a 3:1 ratio to program funds, and the Technical Review and Public Advisory Committees leverage federal expertise and volunteer time).
• LWBSF applicants were generally satisfied with the delivery of the program; three-quarters of applicants or more agreed that the LWBSF Project Officer provided satisfactory support during the project (93%), the application form was easy to complete (82%), and eligibility criteria were easy to understand (76%). The supporting reference tools are also regarded as satisfactory, although fewer agreed that the program's funding decisions were received in a timely manner (60%).
• As might be expected, unfunded applicants had lower levels of satisfaction with the program. Additionally, most unfunded applicants (71%) did not agree that they were provided with sufficient information on the reason(s) their LWBSF funding application was unsuccessful.
• The efficiency of the program was also assessed through an analysis of financial information for the LWBSF. Considering salary and O&M costs to deliver the LWBSF G&C program, the proportion of administrative overhead for the program during the period under study was approximately 14%.^{Footnote 27} This is lower than in Phase I of the program (22% overhead) and lower than other ECCC G&C programs such as EcoAction (22%–25% overhead).

4.3.4 Performance Measurement

Evaluation issue: performance – efficiency and economy	Rating
8. Are performance data being collected and reported? If so, is this information being used to inform senior management / decision makers?	Opportunity for improvement

A Performance Measurement Framework was recently developed, is diligently populated, and is used to monitor and inform decision making about the program. Additionally, performance data is being captured to measure the impacts of projects funded under the LWBSF. Weaknesses were identified in the program’s logic model in terms of presenting an accurate depiction of the program and a realistic progression to achieving final outcomes.

• Although missing indicators and data to track progress on final outcomes, the program has developed and implemented a Performance Measurement Framework (PMF) that includes performance measures to monitor outputs and report on many intended outcomes. Targets are established and historical values are being tracked for the majority of indicators.
• ECCC interviewees reported that performance data have been used to inform senior managers of outcomes, support Departmental Performance Reports, and aid in the selection of LWBSF projects.
• Performance data is captured electronically on the environmental outcomes of funded LWBSF projects and is readily accessible for analysis. Through the survey, the majority of funding recipients (86%) indicated that the reporting template allowed for meaningful reporting on the results of their project. However, the calculation of project environmental outcomes has some challenges.
• The LWBIN case study found that performance measures are being reported as per the requirements of the Contribution Agreement. After the network is fully established, it will be important to ensure meaningful performance measures are in place for this LWBSF project to understand the effectiveness of this dissemination channel.
• As described in section 4.3.1, although a key Intermediate outcome of the evaluation is reduced nutrient loadings, the program as currently designed does not include activities which deliver significant levels of nutrient reductions. While the program’s PMF identifies a target value for this indicator that is in line with the program’s actions, the current level of nutrient reductions are not in their own right likely to make significant contributions to the program’s final outcomes related to improved ecological health of the lake. Furthermore, the two other identified Intermediate outcomes (which both relate to establishing nutrient objectives) are also not likely in their own right to lead to the program’s final outcomes. The logic model appears to be missing an outcome related to greater uptake and actions on the part of other important stakeholders in the basin.

• Previous
• Table of contents
• Next