Evaluation of the National Hydrological Services

List of tables

• Table 1. Partners’ assessment of efficiency
• Table 2. Distribution of spending from 2018-2019 to 2021-2022
• Table 3. Planned vs. actual spending from 2018-2019 to 2021-2022
• Table 4. Partners’ satisfaction with ECCC’s NHS administration
• Table 5. Budget 2018 investments

List of figures

• Figure 1. Staff on cableway (Penticton, BC)
• Figure 2. Water level measurement using reference height markers (Penticton, BC)
• Figure 3. Castor River Station (Russell, ON)
• Figure 4. Castor River with radars anchored on the bridge and shore markers (Russell, ON)
• Figure 5. Example of image velocimetry reading on Rideau River (Ottawa, ON)

Acknowledgements

The Evaluation Division would like to thank the individuals who contributed to this project. In particular, we would like to acknowledge the contribution of program representatives, partners and interested parties who provided input and comments to inform the findings and recommendations of this evaluation.

This report was prepared by the Evaluation Division of the Audit and Evaluation Branch, Environment and Climate Change Canada.

This report was approved by the Deputy Minister on May 9, 2023.

List of acronyms and abbreviations

ADM

Assistant Deputy Minister

CESI

Canadian Environmental Sustainability Indicators

ECCC

Environment and Climate Change Canada

IJC

International Joint Commission

IM/IT

Information Management / Information Technology

ISO

International Organization for Standardization

LWCB

Lake of the Woods Control Board

MSC

Meteorological Service of Canada

MRBB

Mackenzie River Basin Board

NAT

National Administrator's Table

NHP

National Hydrometric Program

NHS

National Hydrological Services

O&M

Operations and Maintenance

ORRPB

Ottawa River Regulation Planning Board

PPWB

Prairie Provinces Water Board

P-T

Provinces and Territories

QMS

Quality Management System

SPB

Strategic Policy Branch

WSC

Water Survey of Canada

Introduction

This report presents the findings from the evaluation of Environment and Climate Change Canada’s (ECCC) National Hydrological Services (NHS). This summative evaluation covers the 4-year period from 2018-2019 to 2021-2022 under the following themes: relevance and responsiveness; efficiency; capacity to deliver on the mandate; results achieved; and use of performance information. The evaluation was conducted as required by the 2016 Treasury Board Policy on Results and puts forward recommendations to address opportunities for future improvements.

1.1 Program overview

The objective of the NHS is to support water management decisions that protect the health and safety of Canadians and ecosystems. In partnership with the provinces and territories (P-Ts), the NHS is responsible for delivering the National Hydrometric Program (NHP), which involves monitoring water quantity and flow across Canada.^{Footnote 1}  While P-Ts provide support for the NHP network, the federal government is responsible for operating water-monitoring stations on rivers and lakes across Canada and for consolidating water-quantity data to provide an integrated view of the country’s surface-water resources with the exception of some cost arrangements with non-ECCC operated stations. NHP stations are co-managed with P-T partners through the National Administrator’s Table (NAT), established under the 1975 bilateral agreements and subsequently updated, and comprises representatives from federal, provincial, and territorial governments.

The NHS activities are divided in two main streams:

• Monitoring water levels and flows (the NHP)
  • Design, develop and implement hydrometric procedures, methods and technology
  • Identify potential solutions by monitoring the evolution of procedures and instrumentation
  • Acquire hydrometric data
  • Produce and disseminate hydrometric data, including in support of the Canadian Environmental Sustainability Indicators (CESI) program.
  • Gather, analyze and respond to stakeholder feedback
  • Negotiate partner agreements
  • Prepare engineering studies and reports
  • Manage the apprenticeship program for hydrometric technologists
• Support to inter-jurisdictional water management
  • Participate on domestic and international water management boards and related tasks and study groups by providing water resources science and engineering expertise
  • Host and provide administrative oversight for domestic water board secretariats
  • Host, manage, and participate in water management boards under the International Joint Commission (IJC)
  • Provide analysis and assessment of monitoring data towards water resources management issues
  • Coordinate actions with stakeholders
  • Manage licensing under the International River Improvements Act
  • Provide support in the areas of hydrology and hydraulics for environmental assessments
  • Provide support to Natural Resources Canada and Public Safety in terms of hydrology and hydraulics expertise related to floodplain mapping.

1.2 Governance and organization

Accountability for the program rests with the Assistant Deputy Minister (ADM), Meteorological Service of Canada (MSC). Accountability is also dictated by Treasury Board Secretariat and ECCC through the Departmental Results Framework. The Hydrological Services program within the DRF falls under the Core Responsibility “Predicting Weather and Environmental Conditions”. National Hydrological Services support the achievement of the Core Responsibility with the goal of “Improving efficiency, timeliness and access to hydrometric data”.

Within the MSC, four Directorates have important roles in the delivery of the NHS. The roles and responsibilities of each directorate are as follows:

Monitoring and Data Services Directorate is responsible for data collection, management and quality assurance, as well as network design and operation. Monitoring and observation of environmental and meteorological conditions over Canada’s vast and diverse landmass and shoreline is enabled through the MSC’s extensive monitoring and observation networks. Data are also received from international sources and through partnerships with provincial and territorial (P-T) agencies.

Canadian Centre for Meteorological and Environmental Prediction Directorate is the operational hub where numerical weather and environmental prediction models are developed, tested and operated, and observation data is ingested into the numerical models to provide the foundation for Canada’s weather forecasts and warnings.

Policy, Planning and Partnerships Directoratecaptures the evolving weather and environmental service needs of Canadians and integrates these into MSC’s program planning, strategic directions, and partnerships.

Prediction Services Directorate is responsible for developing weather and related environmental services and information dissemination to meet the needs of a variety of audiences, notably providing weather warnings, forecasts and information to the general public, 24 hours per day, 365 days per year.

Administration of the NHP

MSC administers and operates the NHP through collaborative governance and cost-recovery arrangements with the P-Ts. Under formal bilateral cost-sharing Ministerial Agreements with the P-Ts, MSC serves as the principal operator of the national hydrometric (water quantity and flow) monitoring network, operating approximately 2200 of 2800 water monitoring stations on rivers and lakes across Canada. These Ministerial Agreements set out the scope, principles, collaborative management and implementation approach to support informed decision-making on water-related issues, effective and efficient operations and determine their respective contributions.

The National Hydrometric Program (NHP) component of the NHS is co-managed by the NAT and the Coordinators Committee, both consisting of members responsible for the administration of hydrometric monitoring agreements in each province or territory and one national administrator designated by Canada, which is the Director General at ECCC who has responsibility for the NHS.

International and domestic inter-jurisdictional water management

ECCC contributes water resources science and engineering expertise to the management of international and domestic transboundary water by carrying out the orders of the International Joint Commission (IJC) under the International Boundary Waters Treaty Act and participation in domestic water boards in partnership with the provinces and territories. Domestic inter-jurisdictional boards include the Mackenzie River Basin Board (MRBB), the Prairie Provinces Water Board (PPWB), the Lake of the Woods Control Board (LWCB) and the Ottawa River Regulation Planning Board (ORRPB). The NHS houses the secretariats for the LWCB and the ORRPB. Both the PPWB and the MRBB are managed by the Strategic Policy Branch of ECCC.

1.3 Resources

Annual expenditures managed for the NHS averaged $50.7M between 2018-2019 and 2021-2022. Similarly, human resources to deliver on the NHS program averaged 324 employees. Additional information is provided in section 2.3 of the report.

Budget 2018 announced an $89.7M investment to transform the NHS, which until then had not benefited from any significant investment in over 25 years. Funding targeted four specific streams: 1) Develop capacity to predict water quantity; 2) Address critical failing infrastructure; 3) Strengthen engineering and technical capacity; 4) Evaluate and test innovations in measurement technology and data integrity. The funding was limited to five years, sun setting on March 31st 2023.

1.4 About the evaluation

The evaluation of the NHS is part of ECCC’s Audit and Evaluation Plan 2022 to 2027. This summative evaluation covers ECCC’s role over the 4-year period from 2018-2019 to 2021-2022 and examines the following themes: relevance and responsiveness; efficiency; capacity to deliver on the mandate; result achieved; and use of performance information.

The evaluation matrix for this evaluation can be found in Annex A.

Multiple lines of evidence were used, including:

• Document and File Review: The primary focus of the document review was NHS program specific documents and Government of Canada policy and program documents, complemented by other publicly available sources.
• Analysis of administrative, performance and financial data: Project and performance data on program outputs and outcomes was reviewed.
• Key Informant Interviews: Over 16 interviews were conducted with program representatives of ECCC that play a role in the administration of the NHS.
• Partner and Stakeholder Survey: A survey of partners representing P-T governments, other federal or international organisations was sent to 40 recipients. 18 recipients (45%) completed the survey.
• Case Studies: An in-depth review of documents related to two water boards: International Osoyoos Lake Board of Control and the Ottawa River Regulation Planning Board to better understand the functions of the boards, their roles, and limitations.
• Site Visits: Two site visits were conducted to water stations in the Okanagan River basin and the Ottawa River Basin to get an in-depth understanding the operations of the stations and innovation solutions currently being tested.

Findings

2.1 Relevance

Requirements

Water crosses political and geographical boundaries. Within Canada, under the Constitution Act (1867), the provinces are “owners” of the water resources and have a wide range of responsibilities in their day-to-day management.

The Canada Water Act established a framework for collaboration among federal, provincial and territorial governments in matters relating to water resources, with each level of government having a different role and many areas of shared responsibility.

The International River Improvements Act provides for licensing of activities that may alter the flow of rivers flowing into the United States.

Finally, the Department of the Environment Act assigns the national leadership for water management to the Minister of the Environment. For international waters, the Boundary Water Treaty provides principles for Canada and the United States to follow in using the waters they share through the International Joint Commission.

The 2021 ECCC Minister’s Mandate Letter also emphasized the need to invest in the Meteorological Service of Canada to upgrade infrastructure, including information technology, to ensure it continues to effectively perform its vital functions of monitoring changes in the weather, climate, water, ice and air quality, and predicting weather and environmental conditions. Within these established requirements, there is clear evidence that there is a continued need for both core elements of NHS activities: 1) Monitoring water levels and flows; 2) Support to inter-jurisdictional water management.

Climate Change understanding, predictability and mitigation

• There is a need for quality water data that the NHS produces and supports. It is recognized that water is the vehicle that delivers many climate change impacts to society and the environment. Precipitation has increased across all regions by an average of 20% since 1948. Precipitation is falling more often as rain rather than snow, especially in the spring and fall. There are changes in the nature of extreme events, and detectable changes in the nature of flood events in some areas, with more rain-on-snow events, rainfall-driven events and earlier spring flooding. This was evident in events such as the 2013 Calgary flood, the 2014 Assiniboine flood and the 2017 and 2019 Ottawa floods (Canada in a Changing Climate: National Issues Report, 2021).
• In its Building a Resilient Canada Report, the Council of Canadian Academies notes the need to improve the quality and availability of climate and disaster data. In particular, localized, high-resolution data that is applicable to specific decision contexts is largely absent, and the understanding of extremes is relatively weak. The NHS, along with other MSC groups, have a role to play in that regard.
• Budget 2018 also recognized this need by providing funding of $89.7M to adapt Canada’s water services to climate change with an objective to revitalize water stations, improve services for long-range water forecasts, test and implement new technologies and expand technical and engineering capacity.

Community support

When surveyed for the evaluation about whether there is continued need for federal government involvement in water management in Canada, 85% of program partners indicated that yes, there is a continued need, citing the need for coordination, consistency and sharing of expertise across the country. Some partners also brought forward the need to advance the work on numerous fronts, including prediction, flood forecasting and related public safety areas, and climate change.

Other considerations

When interviewed, ECCC personnel also recognized the continued need of the NHS program, with the Budget 2018 investments a core example of the value of the program to allow for a continued national leadership in supporting the P-T partners. While not part of the evaluation, the upcoming creation of the Canada Water Agency is also a recognition of a need to put water at the forefront of government priorities, with important and broad policy initiatives needed to be tackled; for example, in the space of social, economic, environmental and climate change policies.

2.2 Responsiveness

The evaluation sought to assess how the program is responding to numerous challenges it has faced since 2018, some of which were anticipated by the program (i.e., technology, climate change), and others that were unforeseen (COVID-19).

COVID-19

As with all organisations, COVID-19 was an unforeseen situation, and prompt adjustments were required to allow for the continuation of NHS key services:

• While some employees continued to work in offices, the switch to telework was rapid, and as a result of the decentralized nature of the organization and its teams, tools and practices were already in place that allowed for the continuation of work.
• A pivot was required as travel was limited, and some planned field work could not happen. Resources were diverted to advance projects, such as studies, and the switch of Aquarius^{Footnote 2}  to the cloud-based version for data acquisition and production.
• Adjustments to normal program delivery were required to comply with ever-changing health guidelines. The NHS quickly developed occupational health and safety protocols for critical services to be carried out, and also reached out to provinces and territories to exchange on which critical stations were to be prioritized.
• As a result of the initial uncertainty in 2020, there were some impacts with regard to hiring of staff, field work that could not take place, and delays in contracting and supply chains that that impacted the construction, maintenance and decommissioning of infrastructure. This impacted half of the infrastructure spending for 2020, or roughly $4M. As a result, some of the funding was redirected to other projects or reprofiled to future years.
• When asked, 84% of the program’s partners indicated they felt ECCC was either very or somewhat responsive in the face of COVID-19. Of all the survey questions to partners about ECCC’s responsiveness, this is the area where respondents felt ECCC was most responsive.

Evolving technology

As in any environmental field, there are different levels of technologies available to support required data collection. In the NHS’ case, a variety of options are available, from using decades old, proven ways to collect and validate data (i.e., manual measures using cableways) to more modern technologies (using shore-operated technology).

Figure 1. Staff on cableway (Penticton, BC)

Figure 2. Water level measurement using reference height markers (Penticton, BC)

Figure 3. Castor River Station (Russell, ON)

Figure 4. Castor River with radars on the bridge and shore markers (Russell, ON)

• Currently, the program has technologies in place that allow for the collection of data that is disseminated online in real-time for the majority of water stations. However, to ensure the integrity and continuity of the data, there are different methods in place, some requiring the usage of cableways. As the program evolves, the program is trying to modernize this data collection while maintaining quality, as the examples below show.
• A site visit in the Ottawa area allowed the evaluation team to witness different technologies being tested, such as Image Velocimetry (using fixed-camera and drone based images) and surface velocity radar (discharge and stage) sensors. These are promising technologies that allow for safer collection of data for the program’ staff (a very important consideration with regard to occupational health and safety, especially in high flow conditions) and have the potential to provide immediate data feeds of the river and even can allow for images of the waterbody. However, it does include some limitations, such as additional infrastructure and technological cost, and limited use in winter when the river is frozen. It was clearly stated that there is not a one size-fit-all technological solution to gather water data, with thoughtful consideration being required, based on the local waterbody.
• In interviews, management indicated that funding obtained in 2018 was essential to allow the program to test all of these innovations. However, the program is now at a juncture where it needs to have an understanding of upcoming funding and then decisions will be made about what types of tested innovations can move to the implementation stage. At the time of the evaluation, testing was still ongoing and future funding unknown.
• When asked, 68% of the program’s partners indicated in the survey that they felt ECCC was either very or somewhat responsive to evolving technologies, while 21% felt they were somewhat (or very) unresponsive. In written comments, some have noted ECCC has been slow to adopt new technologies, as well as delays in approving final datasets.
• The evaluation found that integrating an innovation component as part of the funds received in Budget 2018 generated multiple positive outcomes and is considered a good practice for program design and evolution that should be leveraged by the Department. The innovation-funding component allowed program officials to take stock of the current situation, adapt to change, and prepare for the future. Furthermore, it enabled officials to leverage other countries’ experiences in managing hydrometric programs. The use of the innovation funding has supported the identification of ways to increase the quality and safety of operations while reducing costs and reliance on expensive traditional infrastructure.

Different needs and circumstances of specific population groups/regions

In a country as large and diverse as Canada, everyone recognizes that in the environmental field, what works well in southern Ontario might not work in the Rockies or in the Arctic.

• The way the NHS is structured supports the different needs of the Canadian populations and its regions. The program is decentralized, having staff located in most provinces and territories,^{Footnote 3}  allowing for presence and understanding of the waterbodies in each area. As a result, this fosters collaboration with local authorities, as the evaluation witnessed during a site visit in British Columbia when the provincial dam operator adjusted his schedule to accommodate finalization of the measurements by the NHS.
• Further, the administration of the hydrometric program is structured around individual long-standing agreements with provinces and territories, for which some components (financials, investments, etc.) are negotiated on a yearly basis. Overall this allows both parties to support informed decision making on water related issues, establish a cooperative framework for an effective and efficient hydrometric monitoring program and set out respective parties’ contributions. The program indicated this yearly approach is essential in the context of changing circumstances, whether they are extreme weather events or changed budgetary realities.
• Similarly, the administration of the inter-jurisdictional water boards is supported by formal agreements and/or orders by the parties, with clear membership on these boards. For example, on the International Osoyoos Lake Board of Control, there are 10 members (5 Canadian, 5 American), which include a NHS staff, a mayor of a local community, a regional water board, a US Indigenous group, and a provincial member.
• Senior management have indicated that there is a lot of variation in the needs of the provinces and territories, as the landscape is much different. However, they have felt the responsiveness of the NHS has been very good, especially with recent catastrophic events such as the floods in Fort McMurray, Ottawa or in British Columbia where the NHS was prompted to provide assistance in their field of expertise. Further, the National Administrators Table, where all P-T partners sit several times a year, provides another good venue to exchange and recognize these different needs.
• When asked in the survey, 58% of the program’s partners have indicated they felt ECCC was either very or somewhat responsive to the different needs and circumstances of specific population groups/regions, while 16% felt they were somewhat unresponsive.

Adaptation to climate change

The Government of Canada, with its Federal Adaptation Policy Framework for climate change, set its vision that “Canada is resilient to a changing climate by successfully adapting to the challenges and opportunities, and ensuring the health, safety, and security of Canadians and Canada’s environmental, social, and economic wealth in a long term and sustainable manner.” The work on adaptation to climate change by the NHS is on two broad levels:

• First, there is a need to ensure that the NHS infrastructure is able to withstand the impacts of climate change. The events of November 2021 in British Columbia illustrated this when due to the atmospheric river several stations were destroyed as a result of landslides and high water levels. To that end, the program now looks at available data, such as floodplain mapping, to ensure new infrastructure is built to allow more resiliency and mitigate these potential impacts, which will enable continuity in data collection.
• Secondly, the data produced by the NHS is a key information source in understanding how climate change impacts the country’s waterbodies and its landscape. Ensuring the continuity of this data is essential, as this allows the provision of key products further down the line such as floodplain mapping, flood forecasting and warnings.
• When asked in the survey, 63% of the program’s partners have indicated they felt ECCC was either very or somewhat responsive to the climate change adaption, while 16% felt they were somewhat unresponsive. In written comments, three individuals noted the need to adapt the hydrometric network to be resilient and adapt to climate change.

Integration of Indigenous knowledge and perspectives

The Government of Canada recognizes the need to work with Indigenous partners and incorporate their knowledge and perspectives.

• In its section about water governance, Canada in a Changing Climate: National Issues Report (2021) recognized that Indigenous knowledge is also becoming more explicitly recognized in water research, and in the mandates and practices of water organizations.
• As part of the case studies, we found unequal engagement of Indigenous groups by ECCC. For the Okanagan River, as part of the broader Okanagan Basin Water Board’s 2022 Water Supply Webinar, we saw that involvement of the Okanagan Nation Alliance in their work to ensure fisheries habitat is part of the decision making, with features to ensure proper movement and spawning grounds. Further, the International Osoyoos Lake Board of Control created in September 2015 four new Board member positions for added representation of local community and Indigenous knowledge and interests. However, at the moment, the only representative for Indigenous groups is from the US Confederated Tribes of the Colville Reservation. For the Ottawa River Regulation Planning Board (ORRPB), there is limited integration of Indigenous groups, with participation from some Indigenous communities in the Upper Ottawa River Committee to engage prior and during spring freshet.
• In an internal network assessment report, there is recognition of a gap with regard to Indigenous people and lands for the NHS network current water station coverage, noting it as one of the five priority areas when investing resources.
• Senior management noted that there are challenges engaging Indigenous communities effectively, since currently there is no integrated water governance programming in place for the federal government. The Canada Water Act is silent about Indigenous engagement. The NHS’ operations are about gathering the data and working through the water boards’ structure, with a lot of variation between the different water boards, where some have more Indigenous involvement than others. Management recognizes it is something that needs to be improved, with more relationship building needed, especially in light of the creation of the Canada Water Agency.
• We believe the timing is ideal and presents an opportunity for tangible improvements in Indigenous engagement. Furthermore, the 2021 ECCC Minister’s Mandate Letter directed the Minister to advance the modernization of the Canada Water Act to reflect Canada’s freshwater reality, including climate and indigenous rights. Leveraging departmental resources beyond the NHS program will allow improved engagement and collaboration with Indigenous groups.
• When asked in the survey, 42% of the program’s partners have indicated they felt ECCC was either very or somewhat responsive to the integration of Indigenous knowledge and perspectives, while 42% felt they were neither responsive nor unresponsive. A further 16% felt ECCC was somewhat unresponsive.

Recommendation 1: The Assistant Deputy Minister of the Meteorological Service of Canada should consider opportunities for the NHS to improve engagement and collaboration with Indigenous groups in the context of reconciliation.

2.3 Efficiency

The evaluation sought to assess efficiency on three different aspects. The first centered on the efficiency of delivery in domestic and international partnerships of the NHS. The second aspect concerns internal organisation and mechanisms (such as governance, internal coordination and roles and responsibilities) to support efficient delivery. The last aspect centers around overall efficient use of resources.

Partnerships

The core activities of the NHS are foundationally based on partnerships. For the core activity of providing hydrometric information, through the National Hydrometric Program, agreements are in place with each province and territory that clearly set out, on a yearly basis, the operation of the network, investments to be made, and detailed cost sharing. With that, there is very strong reporting in place that benefits transparency and openness about the program’s management. All parties have the information in front on them and are therefore able to make decisions together for their areas of concern, with opportunities of exchanges and finding of efficiencies and innovation.

A few disadvantages of this approach can however be noted. First, it is onerous in terms of resources to establish, secure approvals, and then report on – a partner has noted important delays on that latter front. Secondly, it can be challenging to secure funding, as both parties (the federal and the P-Ts) need to have these resources available to them, for the same goal. In different operational contexts and political environments, it can be difficult to achieve at times. Finally, in the same vein, it can be difficult to align both budgetary processes, which can create delays before being able to do the actual infrastructure work.

For the second core activity, supporting inter-jurisdictional water management, these are also set in agreements with either the International Joint Commission or the P-Ts. For example, in the case of the International Osoyoos Lake Board of Control, it was established by Order of the IJC on September 12, 1946. In the case of the Ottawa River Regulation Planning Board , a domestic board, it was established through the 1983 Agreement Respecting Ottawa River Basin Regulation, signed by the Governments of Canada, Ontario and Quebec. In contrast to the NHP agreements noted above, the establishment of such boards have less drivers towards efficiency, but rather aim to ensure decision-making on shared areas of interest, and to allow local partners involvement. The evaluation has not identified any specific observations with that regard.

In the survey conducted for the evaluation, we asked the program’s partners to indicate their agreement to statements related to the efficiency of the agreements.

Table 1. Partners’ assessment of efficiency

N=18 (respondents)	Agree/somewhat agree	Neither agree or disagree	Somewhat disagree/disagree
The current governance supports an efficient delivery of the NHS.	61%	17%	22%
The roles and responsibilities are clear and optimized to allow for efficient delivery.	61%	28%	11%
Financial agreements are established in a way that maximizes efficiency.	61%	33%	6%

When asked to further elaborate on areas that need to be improved, partners indicated:

• a need for better roles and responsibilities at the project level (n<5);
• a need for long term financial and policy planning between P-Ts and the federal government in order to reduce surprises (n<5);
• more timely annual reporting (n<5);
• better engagement and collaborative decision making (n<5), and;
• better alignment between budgeting cycles of P-Ts and the federal government (n<5)

A few (n<5) noted recent improvements in increased collaboration between the P-T partners and the federal government. Overall, these comments are also in line with the results of the annual surveys administered by NHS program with National Administrators’ Table members and the Water Board Members.

Internal organisation and mechanisms

The governance of the NHS program stems from decades of ECCC’s leadership in water management, even pre-dating the creation of the ECCC, with the Water Survey of Canada (WSC) established in 1908 as the federal water measurement and information service. As with all historical programs, there have been shifts in organization, with the latest major one being in 2014, when the NHS moved away from regional leadership to a centralized one, under the leadership of one executive director for the NHS program. The goal was to help uniformity of practice in the administration of the program, reduce silos and ensure better integration between all of the regions.

However, we have noted that for both the Mackenzie River Basin Board and the Prairie Provinces Water Board, the responsibilities currently fall with the Strategic Policy Branch (SPB), more in line with the regional approach noted above. Management within SPB have indicated that for both these boards, there are additional important responsibilities with regard to water quality issues, as well as strong Indigenous engagement required. Further, having regional leadership with broader environmental responsibilities, beyond water quantity, is also recognized as a benefit when dealing on these two boards.

Yearly work plans are also established by different types of core activities, such as regional hydrometric operations, engineering, international water boards, domestic water boards, etc.. Each work plan provides key activities, deliverables, metrics, and risks of the program. Similarly, individual water boards also produce work sheets or work plans, depending on the nature of the activities, which ultimately guide upcoming work.

As with all national programs, there are still noted improvements that need to be made to ensure better sharing of best practices. Further, management have noted differences in the way each water boards is run and administered, which can be expected given individual members bring local flavors to their management.

On the Budget 2018 investment delivery, a very thorough governance structure was established with a Director General Advisory Committee, a Directors’ committee, and individual committees for some of the four initiatives. There are also bi-monthly dashboards which report the evolution of the delivery and allow senior management to remain updated. Further to that, there is reporting out to the National Administrators’ Table so the P-T partners are aware of the progress on NHS renewal.

Similar to the above paragraphs about roles and responsibilities, the evaluation has not found issues with regard to roles and responsibilities of staff. Staff have indicated that there are clear roles and responsibilities, especially in light of the agreements with P-Ts to administer the hydrometric network, and similarly, in the establishment of the water boards.

Likewise, coordination between NHS groups has been said to be working well, given the numerous committees in place that allow for regular communications to take place. In interviews, staff indicated cooperatively thinking about solutions and working towards the common goals. A few areas were identified as more challenging, such as Information Management / Information Technology (IM/IT), hiring of human resources, and procurement. In these cases, more regular meetings have been set up to ensure prompt support to deliver on these Budget 2018 commitments that put increased demand for support.

Efficient use of resources

A particularity of the NHS program resides in the cost-shared approach of its work in the administration of the hydrometric network. As a result, roughly 35% of the expenses administered by the NHS are funded by the P-Ts and are managed as a vote netted revenue. For the evaluation period (2018-2019 to 2021-2022), the program administered, on average, $30.7M of salary, $16.6M of operations and maintenance (O&M), $2.9M of capital, and $0.4M of contributions, for a total average of $50.7M. Of these expenditures, on average, the federal contribution was $32.5M, while the P-Ts contributed $18.2M. The table below shows the detailed numbers over the evaluation period.^{Footnote 4}

Table 2. Distribution of spending from 2018-2019 to 2021-2022

Actuals ($)	2018-2019	2019-2020	2020-2021	2021-2022
Salary	26,168,858	29,836,483	33,564,583	33,413,759
O&M	16,431,928	17,338,559	15,368,176	17,134,525
Capital	2,350,757	3,077,168	2,800,068	3,535,176
Contributions	95,000	425,000	425,000	655,000
Grants	0	0	0	0
Total NHS expenditures	45,046,543	50,677,210	52,157,827	54,738,460

For the previous five-year period (2013-2014 to 2017-2018), the total expenditures were lower and more stable, averaging $41.3M per year. However, the influx of the Budget 2018 saw a stable increase of funding.

Table 3. Planned vs. actual spending from 2018-2019 to 2021-2022

($)	2018-2019	2019-2020	2020-2021	2021-2022
Planned	42,145,897	55,716,558	62,469,728	64,659,495
Total expended	45,046,543	50,677,210	52,157,827	54,738,460
Variance ($)	2,900,646	-5,039,348	-10,311,901	-9,921,035
Variance (%)	6.9%	-9.0%	-16.5%	-15.3%

It is also important to assess, per the table above, the variance between the planned and actual spending over the evaluation period. The inability to spend all of forecasted sums can be attributed to numerous factors:

• First, it was challenging at the onset of the funding to spend the funds, as they were provided in October 2018, which was late in the fiscal year.
• Further, there were challenges associated with getting the necessary enablers' support, for example human resources hiring, procurement, and IT, as well as getting the P-Ts on board with increased funding. For the latter point, there are requirements in agreements with the P-Ts that funding be matched for infrastructure projects.
• Finally, for the last two years, COVID-19 severely affected numerous projects, especially on the innovation and infrastructure front, since travel was severely constrained, which impacted both staff and contractors on these projects. Further, global supply chain issues resulted in delays and cost increases in securing materials required for innovation and infrastructure projects.

2.4 Capacity to deliver on mandate

In late 2017, an eleven-member Blue Ribbon Panel was established to review the efficiency and effectiveness of the broad range of functions, activities and mechanisms associated with delivering the NHS’s mandate. The panel was comprised of highly knowledgeable water experts from academia, the private sector, key federal and provincial agencies, international organizations and other advanced National Hydrological Services from around the world. The report was thorough and provided nineteen recommendations. Of particular interest for this evaluation question, it noted capacity shortfalls and insufficient funding as being a limiting constraint for the program.

While conducted in parallel, several areas of improvement the Blue Ribbon Panel identified were targeted with the 2018 funding, such as increased resourcing for transboundary management and monitoring of waters, pursuing prediction work, and infrastructure funding, among others.

In interviews, some staff noted that pre-2018, the situation was grim, with decisions being entertained about shutting down as many as 200 federal stations due to a variety of reasons (capacity, maintenance deficit and occupational health and safety risk for personnel and the public). P-T partners were not supportive of that possibility, as they relied on that data. For the period between 2016 and 2018, there had been an average of 288 employees to deliver on the NHS programs. This number was increased to 338 employees by 2021 (2019 with 303, 2020 with 331). Funding wise, notwithstanding the changes in yearly allocations due mainly to COVID-19 delays, the Budget 2018 funding was set to represent an increase of 40% compared to past levels of NHS expenditures.

Overall, senior management have indicated that Budget 2018 was critical in turning around what was an underfunded program and get back on track with infrastructure’s condition as well as limiting ECCC’s liabilities, provide better support to water boards, and begin testing innovation, all elements that the program was unable to do prior to 2018. The majority of key results highlighted in section 2.5 are the direct consequence of this funding. However, it was noted that if there isn’t an adjustment in funding moving forward, negative impacts on the program are expected, such as no uptake of the innovation technologies tested, loss of technical and engineering capacity, and a likely return of rusted out infrastructure. Management felt this investment brought it back to an acceptable level, but there is now a need to maintain a similar level of funding to ensure the sustainability of the program, as otherwise some gains made with the funding would likely be lost. Further, both senior management and some P-T partners feel this level of service is a baseline level that needs to be maintained. Also, as severe weather events continue to occur with increasing frequency and intensity as a result of climate change, there will be additional pressure on the program to provide support to P-Ts and increased risk on infrastructure. Finally, predictability of future funding would be beneficial to ensure results for Canadians, and allow for continued leadership with the provincial and territorial partners.

Another interesting area that was brought forward by staff that impacts capacity is the fact that this is a highly specialized field with very specific technical knowledge. As a result, it takes some time for new staff to be trained, whether on the technical side or the engineering side. The field of water technicians and engineers is relatively small in Canada. Further, for the technical side, it was explained that it takes several years for a new technologist to be fully trained with a return on investment expected at around 3 to 5 years mark, where staff would be completely comfortable and knowledgeable about the different types of environment, conditions and situations they can face. This was evident during a site visit, where several different skill sets are required to work with different equipment and tools, dealing with the public, and needing to trouble-shoot sophisticated equipment.

While no specific questions were asked in the survey to program partners about capacity, three indicated they have noticed improvements due to an increased capacity as a result of the new funding.

2.5 Results achieved

The NHS is centered around two core activities: 1) providing hydrometric information; 2) supporting inter-jurisdictional water management. The associated departmental result is that Canadians use authoritative weather and related information to make decisions about their health and safety. There are a dozen indicators associated with the program, some of which have linkages to International Organization for Standardization (ISO) Quality Management System (QMS) objectives^{Footnote 5}  and MSC services standards, and most of which are also reported on through the Treasury Board Secretariat’s Departmental Results Framework. Indicators focus on elements such as dissemination of data on the web, new field technologies, satisfaction with services by selected partners, usage by Canadians.

Over the evaluation period, there have been minor shifts in the overall results framework to align to the program’s business and be as efficient as possible. For example, the Departmental Results Indicator changed in 2021-2022 from “Percentage of Canadians that use ECCC information to address water-related impacts on health, safety, economy and environment”, with a target of 80% to “Percentage of program partners and clients satisfied with ECCC’s Hydrological Services program”, with a similar target of 80%. This is in line with the information the program collects and the program’s main users, being P-Ts and water board members. Furthermore, the program doesn’t have all the levers to ensure increased use of that type of information by Canadians. The official DRI results have always been reported using the percent program satisfaction (see table 4, below), since survey information on the percent of Canadians using water data was not available.

For the metric that is set to officially become the DRI in 2021-2022 (i.e., percentage of program partners and clients satisfied with ECCC’s Hydrological Services program), there is historical data that supports its assessment, as the program historically collected information from both the P-Ts and the water board members through annual surveys. Historically, these have relied on a small sample size (i.e., response rate below 50%). While the target is an 80% satisfaction rate, it would only have been met for 2021-22, as the table below demonstrates:

Table 4. Partners’ satisfaction with ECCC’s NHS administration

Partner	2018-2019	2019-2020	2020-2021	2021-2022
P-Ts	56.0%	50.0%	60.0%	100.0%
Water Boards	85.0%	95.0%	70.0%	90.0%
Average	70.5%	72.5%	65.0%	95.0%

Another set of good performance information that the program tracks as part of its ISO 9001 QMS and to report on the public MSC Service Standards, are the following three metrics centers around accessibility, timeliness and accuracy, for which data has been tracked over the years. These are: 1) 95 % of hydrometric data is accessible to the public 24 hours, 7 days a week; 2) 90% of all hydrometric provisional real-time data is available online within 6 hours of occurrence; 3) 80% of data are generated according to Standard Operating Procedures (SOPs).

With regard to the Budget 2018 funding, there were four streams with associated objectives and budgets, as summarized in the table below:

Table 5. Budget 2018 investments

Components	Objectives	Funding over 5 years
1) Develop capacity to predict water quantity	Develop a national approach for predicting changes in water flows and availability, with a new system to forecast water levels and flows in five of Canada’s major water basins.	$20M
2) Address critical failing infrastructure	Investment in existing infrastructures and innovative monitoring technologies and equipment.	$39M
3) Strengthen engineering and technical capacity	Augment engineering and technical capacity to address chronic issues around staffing, training and retention.	$16M
4) Evaluate and test innovations in measurement technology and data integrity	Strengthen national data systems, work with other sectors to test innovative new technologies, improve the ability to integrate promising new technologies into operations, and keep pace and innovate in a rapidly evolving technological environment.	$16M

As of the Fall of 2022, all of the four components have seen key deliverables achieved. However, COVID-19 created some important disruptions, especially related to the movement of staff, supply chain disruptions, temporary hiring freeze, and inflationary pressures. As a result, an additional year (2023-2024) was granted to spend funds for the Infrastructure and Innovation components. At the time of writing this report, work was underway to secure approval of this reprofiling and to confirm the provinces and territories would be matching financial commitments.

The first component, prediction, is well on track to be completed by March 2023, and consolidation of data is ongoing in Fall 2022. The roll-out of the integrated water prediction for the five basins (Great Lakes-St. Lawrence; Saskatchewan-Nelson; MacKenzie; Columbia; and Churchill) is currently at the optimization stage. Further to that, the implementation of the prototype for storm surge and inundation warnings for the Atlantic coast areas is said to be nearing completion. This water prediction capability was built by leveraging and building on the MSC’s knowledge and expertise in weather and environmental modeling, as well as by capitalizing on the existing weather supercomputing infrastructure.

For the second component, infrastructure, there has been a lot of progress, with 377 projects for cableways (282), stilling wells (69), and contaminated sites (26) addressed between 2018 and 2021. As of October 2022, for the 2021-2022 fiscal year, another 56 projects were being completed, with no major expected challenges associated to extreme weather conditions, a first in many years.

As a result of this work, the NHS now currently assesses its infrastructure deficit at 148 cableways in Canada, from a high of 392 in 2016, which represents a notable decrease of 62%. The expectation is that by the end of the funding in 2024, only 39 cableways will be left requiring work. On a lesser scale, creosote wooden wells remediation and removal has been brought down from 542 to 309 as of September 30, 2022, a decrease of 42%. Where the full well removal was not possible, respective remedial measures were implemented and an ecological risk assessment was completed. However, as infrastructure work has advanced over the period, some at risk infrastructure were identified that were not in the scope of Budget 2018. Some of these stations have increased climate change risk that can result with damage or destruction to the NHS infrastructure, as recently experienced in the November 2021 floods in British Columbia.

For the third component, capacity, as of 2022-2023, 90% of the positions (forecasted at 25 FTEs) had been filled and were operational. The program recognizes that the new staff brought new and varied skill sets and experiences that enhanced the capacity to adapt to new and emergent priorities. Highlights noted by the program included, among others:

• Engineering Services support operations on numerous reports, reviews, innovation work, oversight, etc.
• Hydrological Analysis for Great Slave Lake 2020
• Renewed and additional support to transboundary and inter-jurisdictional boards
• Creation of the community of practices
• Evaluation of new technologies to improve efficiency and reduce costs associated with Water Survey of Canada Operation
• Less reliance on engineering firms for infrastructure work.

For the last component, innovation, numerous new technologies have been initiated and are currently tested at over 30 field sites. Examples were demonstrated to the evaluation team during a site visit in the Ottawa area, with technologies such as image velocimetry (using fixed-camera and drone based images) and surface velocity radar (see figure 5). These offer interesting promises and efficiencies, such as live data and imaging 24/7. However, known limitations needed to be accounted for and worked around, such as winter operation with ice, privacy of imagery, and important data storage required. Work is ongoing at these sites, and the program expects to provide a full review of the implementation, testing, and lessons learned by Spring 2024. Also part of the innovation was the implementation and migration of the Aquarius system to a cloud based solution.

Figure 5. Example of image velocimetry reading on Rideau River (Ottawa, ON)

While progress has been made and the delivery is on track, management noted that some challenges occurred that required mitigation. These included:

• High dependency on corporate enablers such as human resources, procurement, IM/IT, real property, meant that some learning at the onset was required to get everyone on board and ramp up efforts to make prompt progress on all deliverables.
• Supporting inter-jurisdictional water management is becoming more challenging as a result of increasing frequency and awareness of water-related issues such as floods and droughts, and the complexity of water management responsibilities. The increased capacity in this regard proved essential.
• As mentioned above, COVID-19 had numerous impacts on operations and renewal activities, which was mitigated by good planning, reprofiling, and good communications with all partners.
• Getting P-Ts on board with investing their parts in the infrastructure has proven challenging in some cases.

When asked, 50% of the program partners indicated that they have seen evidence of improved results in NHS as a result of the funding, with the other half saying they had seen no evidence of improved results. Among the comments, some indicated the results didn’t exceed the basic NHS mandate. A few partners noted the improved infrastructure as a positive result. A few partners noted improvements in prediction, with the caveat that a lot of work still needed to see the final results. Others noted a need to see how innovation will be implemented in the future.

2.6 Use of performance information

The evaluation found that the NHS program is supported by quality performance information. At no time during the evaluation was the lack of information a challenge, with mechanisms and data sources available to support analysis.

Senior management have indicated that with the information available to them, they have a good pulse of the program, where it is going well, and where it is going less well. To support the delivery of the Budget 2018 funding, they have created a governance structure, complete with information rich dashboards that track elements such as budget and costs, project risks, and project schedules, which are updated and presented every second month to the ADM and other partners.

Another evidence of quality performance information is in the annual reports submitted to the individual P-T partners as a result of agreements on hydrometric monitoring. As an example, the 2019-2020 annual report on British Columbia is a thorough 90-page report that speaks about the changes to the provincial network, clearly lays out the costing/types of expenses, modernization cableway and decommissioning schedule, and detailed information provided (with pictures) of infrastructure work done during the year.

Similarly, in the management of inter-jurisdictional boards, there is evidence of rich annual reports produced by the Boards, as witnessed in the case of the International Osoyoos Lake Board of Control. These provide a review of the board activities for the year (i.e., board and public meetings, public communications, special projects), as well as a complete description of the hydrologic conditions and compliance with the rules set by the water board.

One area identified as a challenge by both the program and the evaluation team are the indicators used to support the program activities. On both fronts, staff felt that the indicators do not represent the program very well. For the program indicators, the challenge resides in the Departmental Results Indicator (Percentage of Canadians that use ECCC information to address water-related impacts on health, safety, economy and environment) and a final outcome (Canadians use authoritative weather and related information to make decisions about their health and safety) that is far removed from the day-to-day operations of the program. It is important to recall that the activities (and direct reach) of the program is: 1) providing hydrometric information to a wide range of users and Canadians; 2) supporting inter-jurisdictional water management. Nonetheless, the program has worked around these limitations and has been providing data that supports the DRI and final outcome and has granular information to support the assessment of its performance.

Another challenge that was identified by some staff is the dissemination of water information to Canadians. As it currently stands, other than water data under the umbrella of the NHS, information is scattered based on the administrator of the information. As a result, a given water station operated by a conservation authority, a hydropower company, or a parks authority will not show up in the wateroffice.ec.gc.ca page, and data for the station could potentially be difficult to find. There could be benefits to Canadians to have this centralized hydrometric information, and new products, in one location, but this would require increased resources and cooperation.

Conclusions, recommendations and management response

3.1 Conclusions

Relevance

The National Hydrological Services continue to be relevant, and respond to a need for management of water in Canada, as demonstrated recently in the ECCC Minister’s mandate letter. The program is also essential in providing quality climate change data to allow for better understanding, predictability and mitigation.

Responsiveness

Overall, the National Hydrological Services has been responsive to emerging changes since 2018. It has responded well to COVID-19, and pivoted to allow for continuity in its key services. It is currently testing new technologies that could benefit the program. The National Hydrological Services relies on agreements with P-Ts to support the evolution of the hydrometric network and associated services to respond to the needs of Canadian populations. The National Hydrological Services is also trying to adapt its infrastructure to withstand the impacts of climate change in order to allow continued provision of essential hydrometric data and to support the study of climate change.

In addition, there is an opportunity for the Department to learn from the NHS’ approach where a forward-looking program component tested new technologies and data management processes to innovate for the future. This represents a good practice in program design and evolution.

Further there are opportunities to strengthen the NHS approach to engagement of Indigenous groups given the current lack of guidance and uniformity.

Efficiency

The domestic and international partnerships in place, through structured agreements, allow for efficiency in operations given required cooperation and transparency of both partners. However, these agreements present challenges such as misalignment with differing budgetary cycles and timeliness of reporting. Overall, the NHS internal organization and mechanisms are sound, with good governance in place, clear roles and responsibilities, and good coordination. Finally, the NHS has been found to be using its resources efficiently, with challenges identified related to coordination and administration of some resourcing activities.

Capacity to deliver on mandate

Over the evaluation period, the NHS went from a period with capacity constraints, identified in part through its Blue Ribbon Panel, to one where it could, with the increased funding under Budget 2018, address issues that had been hampering the program for years.

However, with this funding set to end shortly, there is a recognized risk by management for the future of the NHS, as gains in infrastructure, innovation, and increased support to water boards would be halted. There is also a risk that sunsetting of the funding would limit future sustainability of the program, and critically reverse the positive results achieved from this funding. This could further result in leadership challenges with provincial and territorial partners.

Results achieved

The NHS adjusted its overall results framework to better align the program’s business in serving mainly partners such as P-Ts, and continues to collect a suite of relevant and quality results data. With regard to the Budget 2018 investment, results are being achieved. Prediction efforts are nearing completion with both integrated water predictions in five water basins and prototype for storm surge and inundation warnings for the Atlantic. Infrastructure is also well advanced, with over 400 projects completed or nearing completion, thus significantly reducing its infrastructure deficit. Enhanced capacity has allowed the NHS to increase engineering support to infrastructure projects, water boards, and hydrological analysis. Finally, innovation technologies are currently being tested at over 30 field sites, with completion planned by Spring 2024.

Use of performance information

The NHS has a rich suite of quality performance information, including performance dashboards to support the delivery of its Budget 2018 funding, annual reports for its agreements with P-Ts as well as for its inter-jurisdictional boards. Senior management noted performance information was a strength of the program. Minor challenges reside with the Departmental Results Indicator that is not as aligned to the program as it could be. There could be benefits to Canadians to have centralized hydrometric information, and new products.

3.2  Recommendation

The following recommendation is directed to ECCC’s Assistant Deputy Minister of the Meteorological Service of Canada (MSC), as the senior departmental official responsible for the National Hydrological Services (NHS) Program.

Annex A – Evaluation matrix

Themes	Questions	Indicators	Sources/Methods
Relevance and Responsiveness	1. To what extent there is a continued need for federal involvement in water management in Canada? 2. To what extent the Program has been responsive to emerging changes since 2018 (e.g., technological advancements, changing government priorities, emergence of new partners/services, COVID-19, various needs of specific sub-groups of the population, adaptation to climate change)?	Evidence of need addressed by the Program. Evidence of gaps fulfilled by the federal government. Evidence of actions towards emerging issues, challenges and developments including: evolving technology, changing government priorities, new partners/services, COVID-19, climate change, differential needs and circumstances of specific population groups/regions Internal and external perspectives, including Indigenous groups, on responsiveness of the program and suggestions for improvement.	Document review Key information interviews Case study Stakeholder Survey
Efficiency	3. To what extent the domestic and international partnerships have been supporting an efficient delivery of the program? What works well and what are the areas for improvement? 4. To what extent the internal organisation and mechanisms (such as governance, internal coordination, roles and responsibilities) support the efficient delivery of the Program? 5. Has the Program used its resources efficiently? How could efficiency be improved?	Evidence of contribution of domestic and international partnerships (i.e., governance and financial partnerships) to Program efficiency Challenges and benefits related to current domestic and international partnerships (i.e., governance and financial) Evidence of existence of a defined governance structure, roles, internal coordination and roles and responsibilities Evidence of clear accountability and processes to make decisions Internal perspectives on clarity of roles and responsibilities, efficiency of internal collaboration in delivery. Comparison of planned and actual spending Evidence of measures taken to deliver activities efficiently or improve efficiency Internal perspectives on additional measures or mechanisms for improving efficiency. Document review Key informant interviews Case study Stakeholder Survey
Capacity to deliver on the mandate	6. To what extent the Program has the capacity (e.g., expertise, training, support and tools) and resources (qualified staff, financial) to deliver on its mandate?	Internal and external perspectives on resources and capacity alignment with Initiative mandate Evidence of appropriate financial resources and capacity to deliver the program. Document review Data analysis (financial information) Key informant interviews Case study
Result achieved	7. What results have been achieved by the Program?	Evidence of achievement of expected outcomes (3), per the logic model. Evidence of other results and benefits achieved by the Program, including: Develop capacity to forecast water quantity Address critical failing infrastructure Strengthen engineering and technical capacity Evaluate and test innovations in measurement technology and data integrity.	Document review Administrative and performance data analysis Key informant interviews Case study
Use of performance information	8. To what extent the Program has quality performance information (accessible, sufficient and reliable) that was accessible to decision-makers and was it adequate to support this evaluation?	Evidence that performance information is accessible, sufficient and reliable for decision-makers and evaluators. Performance data analysis Key informant interviews

Annex B – Reference list

Canada Water Act, RSC 1985, (1985).

Council of Canadian Academies, (2022). Building a Resilient Canada, The Expert Panel on Disaster Resilience in a Changing Climate, Council of Canadian Academies, Ottawa (ON).

Department of the Environment Act, RSC 1985, (1985).

ECCC (2011). Federal Adaptation Policy Framework for climate change

ECCC (2022). Canada Water Act annual report for 2020 to 2021

IJC (1909). Boundary Water Treaty

International River Improvements Act, RSC 1985, (1985).

The Constitution Act, RSC 1982, (1867).

Warren, F. and Lulham, N., editors (2021). Canada in a Changing Climate: National Issues Report; Government of Canada, Ottawa, ON

World Meteorological Organization (2019). Strategic Plan (2020-2023)