4. Findings
This section sets out the findings of this evaluation, broken down by evaluation issue (relevance and performance) and by related evaluation questions. For each evaluation question, a rating is provided based on an assessment of the evaluation findings. The rating statements and their significance are outlined below in Table 3. A summary of ratings for the evaluation questions is provided in Appendix B.
4.1. Relevance
4.1.1. Continued need for the Program
Evaluation issue: relevance | Rating |
---|---|
1. Is there a continuing need for the Program? | Acceptable |
Given the importance of freshwater as a resource, the known and emerging threats to water quality and their potential health impacts, as well as existing legislative obligations, there are clear societal, environmental and legislative needs to monitor, assess and report on Canada’s water quality.
- Canada contains roughly seven percent of the Earth’s renewable freshwater. Water quality degradation affects both aquatic life and human uses of water. For example, higher concentrations of nutrients may result in uncontrolled plant growth and reduce the amount of dissolved oxygen available for fish and other aquatic animals. They can also foster the growth of algae, some of which can cause health effects in humans and animals. Degraded water quality can also undermine economic activities such as fisheries, tourism and agriculture.Footnote 13
- According to the CESD 2010 Fall Report, Canadians have historically regarded fresh water as the country’s most important natural resource, and fresh water is considered a critical factor to Canada’s future prosperity.Footnote 14 However, the quality and quantity of Canada’s water resources are under pressure from a variety of sources, including urban runoff and sewage, invasive species, agriculture, and industrial activities. Other long-term threats include population growth, economic development and climate change.Footnote 15
- Adequate long-term trend information on water quality is important for the timely identification of current and emerging threats and preventative actions, and to help ensure that aquatic ecosystems are conserved and protected for the well-being of Canadians. As well, sufficient information on water quality and aquatic ecosystems health is needed to be able to restore or mitigate the impact of an industrial accident or spill on a watercourse.Footnote 16 There is expected to be a continuing need for water quality monitoring over time as the nature of environmental contamination changes (e.g., climate change, northern contaminants, invasive species and agricultural pesticides), as community and resource development takes place (e.g., Ring of Fire mine in Ontario)Footnote 17, and as the nature of partnerships change (e.g., Indigenous peoples’ involvement in CSSP activities and the public’s increasing participation in monitoring).
4.1.2. Alignment with federal government priorities
Evaluation issue: relevance | Rating |
---|---|
2. Is the Program aligned with federal government priorities? | Acceptable |
The work of the Water Quality and Aquatic Ecosystems Health Program supports several priority government programs, and addresses commitments outlined in federal/provincial/international agreements related to water quality monitoring.
- Through the provision of water quality and aquatic ecosystems health data and information from ongoing research and monitoring, the Program supports a range of priority government programs, including the Federal Action Plan on Clean Water, the Clean Air Agenda Adaptation Theme, and the Chemicals Management Plan. Program activities also support the Federal Sustainable Development Strategy (FSDS).Footnote 18 As well, the Program fulfils federal environmental monitoring commitments under the bilateral agreement between ECCC and the Province of Alberta to implement the JOSM (three-year Memorandum of Understanding [MOU], 2011–2015). Research as part of the NCP meets federal commitments related to the Arctic Council’s Arctic Monitoring and Assessment Program.
4.1.3 Alignment with federal duties and responsibilities
Evaluation issue: relevance | Rating |
---|---|
3. Is the program consistent with federal roles and responsibilities? | Acceptable |
The federal government has responsibilities related to water management, and has jurisdiction over transboundary waters, although it shares responsibilities for freshwater quality management with the provinces. To ensure clarity, the Program administrators have implemented a series of agreements that define the Program’s roles in inter-jurisdictional water quality monitoring.
- The Program’s current focus is on activities related to water quality and aquatic ecosystems health monitoring and research that support legislated mandates and commitments, priority government initiatives (including regulatory activities) and reporting responsibilities, as noted below.
- Under the Canadian Environmental Protection Act, 1999 and the Department of the Environment Act, ECCC is mandated to conduct scientific research and environmental monitoring in order to provide robust scientific data, knowledge and evidence to support policy, regulations, enforcement, and decision-making. However, the federal and provincial governments share responsibility for overall freshwater management. The provinces have authority to legislate with respect to various aspects of water supply and use, pollution control, hydroelectric and non-nuclear power development, irrigation, and recreation within their borders.
- Under the Canada Water Act, 2014, the federal government has jurisdiction over and responsibility for managing inter-jurisdictional waters,Footnote 19 water on federal lands (e.g., national parks), federal facilities (e.g., office buildings, labs, penitentiaries and military bases), First Nations reserves, and two of Canada's three territories (Nunavut and the Northwest Territories). The Act also empowers the Minister of the Environment to enter into intergovernmental arrangements to establish and maintain an inventory of any waters where there is significant national interest, and to collect, process and provide data on the quality, quantity, distribution and use of those waters.
- Under the International Boundary Waters Treaty Act, 2001, the Department has commitments relative to the use of boundary waters between Canada and the United States.
- Under the Fisheries Act, ECCC, through the CSSP, is required to undertake sanitary and marine water quality monitoring surveys and assess pollution sources in order to classify shellfish growing areas as safe for harvesting. An MOU signed in 1990 describes the duties and responsibilities of the three participating departments or agencies. In addition, ECCC is responsible for administering and enforcing the pollution prevention provisions under the Fisheries Act, which prohibit the deposit of deleterious substances into fish habitat.
- Preserving the health of Canadian waters is a shared responsibility, and partners in program delivery related to freshwater quality monitoring include other government departments and provinces and territories. Federal-provincial water quality monitoring agreements with British Columbia (1985), Manitoba (1988), New Brunswick (1988), Newfoundland and Labrador (1986), Prince Edward Island (1989), the Prairies Provinces Water Board (1992) and the Mackenzie River Basin Transboundary Waters (1997) constitute a long term commitment to acquire comparable water quality data across the country to support shared roles in water resource management.
- Interviewees noted that, in addition to supporting obligations, as outlined above, federal involvement was crucial for ensuring a nationwide approach to monitoring and providing an unbiased, credible, long-term and integrated understanding of aquatic ecosystem health. Data users noted that water quality monitoring data would be less comprehensive, more fragmented (e.g., focused on industrial activity or provincial compliance), less accessible and ultimately less useful without a nationwide approach. Provincial/territorial interviewees said that their governments did not have the financial resources or scientific expertise to sustain the water quality monitoring activities that are needed.
- Program partners also noted that ECCC has avoided the potential for overlap or duplication between federal and provincial/territorial activities related to water quality monitoring and aquatic ecosystems’ health initiatives by streamlining and rationalizing its activities and ensuring consistent and frequent consultation with provincial/territorial partners.
4.2. Performance – effectiveness
4. To what extent have intended outcomes been achieved as a result of the Program?
Evaluation issue: performance – effectiveness | Rating |
---|---|
i. Research capacity is maintained and aligned with priority areas | Acceptable |
ii. Science-based knowledge and data is available to address mandates and commitments | Acceptable |
iii. Risk-based mechanisms are in place to support water quality monitoring | Opportunity for improvement |
Evaluation issue: performance – effectiveness | Rating |
---|---|
i. Timely science-based information and data is used to support decisions | Opportunity for improvement |
ii. Key risks are identified, assessed or managed | Acceptable |
Evaluation issue: performance – effectiveness | Rating |
---|---|
Water resources policies and programs are informed by data and information. | Acceptable |
Immediate outcome 1: Research capacity is maintained in areas that align with ECCC and federal government priority areas – Acceptable
Research is being conducted in collaboration with others, and addresses priority areas including the oil sands, the Arctic, and the Chemicals Management Plan.
- In collaboration with others, ECCC scientists actively contribute to new knowledge related to water quality threats, and have developed new approaches for the assessment, conservation and remediation of aquatic ecosystems.Footnote 20
- A Bibliometric Analysis conducted for the evaluation showed that over the 2010–2014 period, a total of 397 WSTD scientists contributed to 1,208 scientific publications (average of 242 publications per year). Eighty-eight percent of the articles published were co-authored by at least one WSTD researcher and one or more external authors. These WSTD publications included 2,637 unique external authors from 12 different categories of institutions and were published in 299 unique journals.
- G&C projects also contribute to research capacity. The file review revealed that most of the G&C projects sampled (13 out of 17) focused on producing “new knowledge for the conservation and restoration of water resources and aquatic ecosystems” and addressed a range of water quality-related issues including water quality and watershed modelling, identification and quantification of contaminants, and the effects of oil sands development.
- The Program’s research contributes to key priority areas by
- Significantly improving the ability to track low-level contaminants and provide information about the impacts of oil sands activity through the provision of monitoring, research and analysis of water resources in the Lower Athabasca region;
- Providing water quality surveillance information to inform risk assessment and risk management for the Chemicals Management Plan; and
- Contributing new knowledge on long-range contaminants in the Canadian Arctic for the NCP.Footnote 21 Canadian Arctic Contaminants Assessment Reports state that progress has been made in understanding their trends and sources, while noting that the assessment of biological effects remains a challenge.Footnote 22
- ECCC interviewees and program partners rated the achievement of this outcome as moderate, stating that, while the Program’s existing research capacity is very strong, retiring scientists and technicians are not being replaced and those who remain reported feeling “spread very thin.” Program representatives reported that the Program’s focus is on identified high priority areas, with limited attention on other emerging issues, such as pharmaceuticals, nanoparticles and marine monitoring. Some interviewees expressed a desire to see more research on aquatic ecosystem health, noting that work in this area is primarily limited to activities under formal federal/provincial agreements for monitoring in the Great Lakes.
Immediate outcome 2: High quality science-based knowledge and data are made available to address Canada’s water quality and aquatic ecosystems health legislated mandates and commitments – Acceptable
Credible, science-based and relevant water quality and aquatic ecosystems health information is made available to stakeholders and the public through websites, data portals and peer-reviewed publications to support the Program’s mandate and commitments related to its component programs.
- Consistent with its legislative responsibilities, the Program ensures that key information related to freshwater quality monitoring is made available to the public through ECCC’s water websiteFootnote 23 and includes information on water quality guidelines and water quality monitoring. From this site, the public can view information on national, regional and local water quality conditions through the National Freshwater Quality Indicator (updated in June 2015 to cover the 2010–2012 period).Footnote 24 Freshwater quality is rated at a total of 336 long-term monitoring sitesFootnote 25 across Canada, by determining whether chemicals in water and the physical properties of water are at levels that could harm aquatic plants and animals.
- As part of the Freshwater Quality Monitoring Program, aquatic ecosystem health data are provided from the CABIN data portal. As of 2016, the CABIN portal includes data for approximately 15,500 samples contributed from various parties.
- In addition to the freshwater quality monitoring data, other components of the Program make their data available to the public, for example:
- Through the JOSM data portal, the public can access monitoring maps, site information and environmental monitoring data;
- As part of the CSSP, marine water quality information is available through the CESI, and is provided to DFO and the CFIA, which provide information externally on the classification of shellfish harvest areas. The Shellfish Growing Area Quality Indicator indicates the percentage of growing areas approved or conditionally approved for harvesting for human consumption (updated in 2011; most recent data are for 2006–2010).
- The quality of water monitoring data is assured through ECCC’s laboratories and analytical services, which provide credible (ISO certified), legally defensible science. These services support the various water quality component programs, such as freshwater quality monitoring and surveillance, oil sands monitoring, the CSSP, and research initiatives such as the NCP. In addition, a Freshwater Quality Monitoring and Surveillance Quality Assurance Framework has been completed and is being implemented to provide national guidance for consistent quality assurance.
- The evaluation found no evidence to indicate that ECCC is not meeting its commitments related to the provision of scientific knowledge and data as required under legislation, water quality monitoring agreements and priority programs, as described in Sections 4.1.2 and 4.1.3.
- Both internal and external interviewees rated the achievement of this outcome as good, and noted the high quality and demand for WSTD science in order to inform policy, regulations and new methodologies. Promoting knowledge transfer with other scientists was another identified area of strength; for example, the relationship between water quality monitoring scientists and CABIN scientists and the newly installed National Laboratory Information Management System (LIMS). External interviewees expressed an interest in increased timeliness of data, more data related to aquatic ecosystem health and greater flexibility in determining monitoring locations and detection levels.
Immediate outcome 3: Risk-based mechanisms are in place to support water quality monitoring using cooperative and integrated approaches with others – Opportunity for Improvement
A considerable amount of work has been done to prioritize freshwater quality monitoring activities through the development of a risk-based approach (RBA). While there has been substantial progress, the approach has yet to be fully implemented. For marine water quality monitoring, the Program makes use of pollution surveys and surveillance to minimize risks.
- The Freshwater Quality Monitoring Program has been significantly transformed in the past few years, partially in response to a Commissioner of the Environment and Sustainable Development (CESD) 2010 audit.Footnote 26 In particular, the Program has developed and begun implementation of a RBA for its monitoring activities.
- The RBA is intended to focus available resources on monitoring the activities and substances that pose the greatest risks to water quality (i.e., determine the optimum number and location of water monitoring sites across Canada).The RBA was overseen by a Network Planning Team set up in 2011 and the approach was developed in several stages, as follows:
- the scope of federal responsibilities for water quality monitoring was defined;
- a risk-based assessmentFootnote 27 of all long-term freshwater quality monitoring sites was conducted;
- a geospatial risk-based basin analysis (RBBA)Footnote 28 was conducted to develop cumulative risk maps of threats to water quality at the sub-basin level; and
- statistical power analytical tools were used to assess optimal sampling strategies.
- The approach combines risk-based components, specifically the likelihood, extent and severity of impact, in order to provide a summary of relative risk to water quality and the aquatic ecosystem. The analysis uses ECCC’s scientific data, information and knowledge of the site and watershed (e.g., incorporating the status and trends, CABIN assessments, and surveillance data with available background science information).
- While a considerable amount of work has been done in this area, the RBA has not been fully implemented and program representatives are still developing the approach and working with key clients to determine how best to apply it in light of user needs and existing agreements. It was determined in a case study of the pilot study of the power analysis statistical tool that the tool was not useful for or applicable to the circumstances of the PPWB, since water quality monitoring requirements in the area were already agreed to and outlined in an inter-jurisdictional water quality agreement.Footnote 29 The annual CESI/CCME Workshop is leveraged as a forum to inform, collaborate with and receive feedback from other agencies and partners.
- For marine water quality monitoring, pollution surveys and surveillance activities are used to target monitoring activities for the CSSP in a way that minimizes the risks associated with shellfish consumption. As part of this activity, ECCC continues to update and maintain an extensive inventory of land-based sources of pollution impacting coastal environments, including approximately 350 wastewater systems discharging into shellfish harvesting areas. ECCC’s outputs under the CSSP are measured annually in order to further develop and improve the Program, with the resources available.Footnote 30
- ECCC representatives and program partners rated the achievement of this outcome as good, given the work undertaken since 2010 to review the Program’s processes, roles and responsibilities and develop the tools for evaluating risks at monitoring sites. As described above, interviewees said that the risk based mechanism is still in the development phase, and noted that 2016–2017 would be critical in determining how best to apply the mechanism.
- According to interviewees, other countries and jurisdictions consider some of ECCC’s risk assessment and risk-based sampling methods to be best practices, and in some cases, have adopted a similar approach.Footnote 31
Intermediate outcome 1: Water resource managers use timely and relevant science-based information and data to support decisions related to water quality – Opportunity for improvement
A range of internal and external stakeholders use reliable scientific water quality and biological indicator data from the Program to inform policies, direct actions, assess impacts, and understand status and trends. While Program staff are taking steps to address concerns, there are still challenges related to providing timely access to freshwater quality monitoring data and other scientific knowledge in order to inform decision-making.
- There are numerous examples of how the Program’s science-based information and data are used to support policies, programs and regulations, including the following:
- Monitoring measurements are used by various levels of government to assess and report on status and trends (e.g., “State of the Lake” reports, “State of Parks” reports) and evaluate progress made in protection and remediation programs (e.g., Chemicals Management Plan);
- A JOSM-specific information portal provides monitoring results for an area covering some 140,000 km² and enables interested parties to conduct their own analyses of the impacts of oil sands development;
- Results from the NCP continue to play a role in the assessment of future chemicals of interest and policies needed to protect the North. Program results are used for reporting purposes under the Global Monitoring Plan established under Article 16 of the Stockholm Convention;Footnote 32
- CABIN-generated biological indicator information is used by a wide range of water resource managers across Canada to understand watershed health and inform management, policy and regulatory decisions. The number of CABIN users has grown from less than 100 in 2002 to 1,460 in 2016.
- The CESI National Freshwater Quality Indicator uses Program data to provide an overall measurement of the status of water quality to protect aquatic life across Canada. This can be used by the public to understand water quality and human impacts on water quality.
- The key issue identified by some Program partners and data users is the lack of timely access to freshwater quality monitoring data, which hinders its effective use for risk identification and mitigation. They said that currently, data is rarely received within less than a year; a time frame that does not meet their needs in order to develop appropriate policies and mitigation measures.
- Program representatives share these concerns, noting that, among other issues, timelines for the release of data are impacted by quality assurance/quality control (QA/QC), in particular, the time required for laboratory analysis and the need to wait until the end of the season to assess quality over the entire season. Data is usually available within 18 months, but can take up to two years or longer.
- Program staff are looking at a number of measures to increase the accessibility and timeliness of data, but progress has been slow and the work has yet to be completed. For example,
- The Data Management Task Group, set up in 2013, developed a standard approach for publishing scientifically valid and comparable long-term water quality monitoring data through the GC Open Data Portal. At the time of writing of this report, Program staff were in the process of releasing the water quality monitoring data from the 2000–2015 period from the core monitoring sites in an openly accessible format. Over the longer term, Program staff will be working on setting up a data warehouse into which the five key water quality monitoring databases will be migrated to create a single open database system available to the public. This project was previously scheduled for completion by the end of 2017, but is now considerably delayed. It should be noted that the project is dependent on corporate and financial functions outside the Department’s control.
- Starting in 2016, Program staff were implementing a National Data Request and Data Release Protocol and a National Registry for documenting data requests. Under this Protocol, it is expected that all data will be made available to members of the public who clearly specify the stage of data verification/validation.
- In addition to issues related to timely access to water quality data, concerns were also expressed by both internal and external stakeholders regarding the public accessibility of scientific research and publications. This information is generally shared within the academic and scientific community; however, a lack of public dissemination of research studies to provide the tools and inform water management decision-making for stakeholders outside of the scientific community was identified.
Intermediate outcome 2: Key risks associated with water quality and aquatic ecosystems health are identified, assessed or managed – Acceptable
Key risks have been identified and assessed for the freshwater and marine water quality monitoring programs and are being used to manage activities.
- Risks for both the Freshwater and Marine Water Quality Monitoring programs have been identified and assessed, as follows:
- The Marine Water Quality Monitoring Program risks include point and non-point sources of pollution that could impact shellfish harvesting areas. These risks are assessed through comprehensive sanitary and bacteriological water quality surveys and identification of pollution sources. This information is used to recommend the classification of growing areas and their boundaries (e.g., approved or conditionally approved for harvesting or closed for harvesting). This process is fully implemented as part of the CSSP. The 2013 international program audit report indicated that the program is supporting food safety outcomes, and that risks are being managed through the use of existing measures and the introduction of continued program refinements.
- The Freshwater Quality Monitoring Program risks, identified as part of the risk-based approach previously described, include: (1) human activities in the watershed; (2) impacts and risks to water quality and aquatic life; and, (3) health/vulnerability of the aquatic ecosystem. These risks are analyzed for all federal freshwater sites and used to help ECCC focus on priorities and performance deficiencies. Results may point to a need for increased monitoring activity and resources in areas where the risk of impairment is high, but they should also be used to balance the need for baseline data at reference locations, identify knowledge gaps and prioritize future monitoring activities. This analysis has been used to implement changes (e.g., the results of the power analysis were used to justify an increase in the frequency of sampling for pesticides at some sites),Footnote 33 although, as previously described, the RBA has not been fully implemented.
Final outcome: Canada’s water resources policies and programs are informed by water quality and aquatic ecosystems health data and information from ongoing research and monitoring of waters under federal jurisdiction – Acceptable
Monitoring data and scientific research from the Program are used to inform water resource policies and programs across Canada.
- As described above in the assessment of immediate and intermediate outcomes, data and information from the Program is being used to inform water resource policies and programs across Canada. Partners use and highly value the long-term monitoring data provided.
- ECCC interviewees rated the achievement of this outcome as reasonably good, although several areas for improvement and challenges were identified. Some interviewees noted that the Program provides information and knowledge that allows ECCC to identify, characterize and target threats to water quality and aquatic ecosystems health, indicating that the Program’s real contribution is to enable the Department to identify what it needs to do to address threats and ensure sustainability. As previously noted, the identified challenges were difficulty in providing timely access to data, lack of funding and support for research into emerging issues, and limited research into aquatic ecosystem health.
- Program partners were more positive and rated the achievement of this outcome as good, and nearly all of them provided examples of how policies and programs are or have been informed by program information (e.g., CCME national water quality policy on nutrients). In their view, it is of the utmost importance that ECCC maintain its long-term monitoring program.
4.3. Performance – efficiency and economy
4.3.1. Program delivery
Evaluation issue: performance – efficiency and economy | Rating |
---|---|
5. Is the program implemented efficiently and economically? | Acceptable |
Program staff are implementing measures to ensure efficient operations, including the use of integrated watershed and risk-based approaches, measures to ensure that activities remain focused on key priorities, and a collaborative delivery model.
- According to the majority of internal interviewees and Program partners, Program staff carry out their activities efficiently, and they provided examples of efficient practices including the following: co-location of hydrometric and water quality monitoring staff to minimize sampling expenses; application of the risk based approach to promote better decision-making relative to monitoring within financial constraints; the creation of Research Manager positions to ensure that all activities help to achieve key priorities; and the setting up of the National Laboratory Information Management System (NLIMS) at four out of eight departmental laboratories, with full implementation scheduled for 2016–2017.Footnote 34
- The Program has also moved away from its previous regional perspective towards use of an integrated watershed-based approach across federal, provincial and territorial jurisdictions (Pacific, Arctic, Hudson Bay and Atlantic). Water monitoring and assessment water from this perspective is viewed as a more efficient and effective approach because it takes into account activities occurring in multiple jurisdictions and considers the impacts of upstream watersheds.
- The Program’s collaborative approach also contributes to program efficiencies. ECCC’s internal science capacity is significantly increased through extensive collaboration across Canada and internationally. There is a wide variation in the types of collaboration between the WSTD and its partners, and these were viewed favourably by interviewees.
- The CABIN’s network of networks approach and blended (online and field) training is another example of efficient collaboration with partners. The data gathered by the CABIN every year has grown from about 50 samples in the early 2000s, prior to the development of a training program, to an average of about 900 per year from 2003 to 2015. Across Canada, more than half of all CABIN data is now contributed by external agencies.
- The majority of internal and partner interviewees said that the current program delivery methods, with duties and responsibilities shared with provinces and territories and/or other departments, operate effectively and optimize the expertise and mandates of all concerned. However, a few partners had concerns regarding the following:
- draft water agreements yet to be signed or requiring modification;
- the split in responsibilities related to air- and surface-water versus groundwater monitoring between ECCC and Natural Resource Canada (i.e., if ECCC were responsible for all water monitoring, it might promote a more cohesive management of the water file); and
- the potential impact of increased community level participation in water monitoring tasks in the future on data quality.
- The use of G&Cs was also viewed as an effective mechanism to address research issues and make efficient use of university and government scientific investments. A review of G&C files revealed that about one quarter of total project resources came from funded partners, and that most projects (65%) would be sustained by other means in the future.
- A few interviewees noted that efficiencies may have been achieved at the expense of flexibility to address emerging issues and new research areas, and about half of the program partners noted in their answers that more direct, regular and structured communications by Program staff with provincial/territorial partners would make the Program even more efficient. To increase collaboration with the provinces and territories, a few program partners suggested that it would be beneficial to reinstate the former national annual conference on water quality monitoring. Currently, information is shared as specified in formal agreements and in specific watershed forums.
4.3.2. Program governance and management
Evaluation issue: performance – efficiency and economy | Rating |
---|---|
6. To what extent is the governance structure clearly defined and appropriate for achieving expected results? | Acceptable |
Governance for the key components of the Program is clearly defined in formal agreements and/or through established committees.
- The evaluation found that governance appears to be clearly defined and appropriate at the individual component level within the Water Quality and Aquatic Ecosystems Health Program (e.g., for freshwater quality monitoring, the CSSP, the JOSM and the NCP).
- Specific governance for these multi-stakeholder program components is defined either under formal water quality agreements, MOUs (e.g., the CSSP) or under committee structures (e.g., the CSSP, the JOSM and the NCP).
- For the most part, interviewees confirmed that the governance procedures for the Program components were clearly stated and appropriate, and included clearly defined priorities and accountabilities.
4.3.3. Performance measurement
Evaluation issue: performance – efficiency and economy | Rating |
---|---|
7. Are performance data being collected and reported? If so, is this information being used to inform senior management and decision-makers? | Opportunity for improvement |
Program staff track progress made under the Program and submit various legally required reports, generally at the activity/output level. However, there is no performance measurement strategy or formally approved logic model that clearly states intended Program outcomes in the immediate and longer term, and identifies how progress toward these outcomes will be measured.
- Program administrators report on performance in the Departmental Performance Report, in the Canada Water Act Annual Report and in the CESI. As well, the Marine Water Quality Monitoring component has specific performance indicators and the latter’s administrators report in the Fisheries Act Annual Report to Parliament.
- The Departmental Performance Report provides information on indicators that track the percentage of sites within ECCC’s national core water quality monitoring network where water quality monitoring was carried out. In 2015–2016, additional performance indicators were introduced to assess client satisfaction and annual per capita peer-reviewed publications. This provides a better alignment of Program performance information with the broader Water Program (1.2) and ECCC’s Science Strategy.
- The drafters of the Canada Water Act Annual Report use Program information to report on progress (activities, outputs) relative to water quality monitoring, water research, inter-jurisdictional water boards and ecosystem initiatives.
- A broader initiative entitled “Measuring Environment Canada’s Research and Development Performance” is currently under way and its objective is to measure and report on research and development across ECCC, including work related to water quality.
- Program administrators have developed a “Plan / Do / Check / Improve” framework to enhance program management. It is based on the scope of activities, client needs, key risks and program priorities, and incorporates a total quality management system and performance excellence model. This framework is intended to be used to monitor, adjust and continuously improve water quality monitoring, and, while not yet formally implemented, it is currently influencing work planning.
- While these activities contribute to performance reporting, as previously noted, the Program does not have a formally approved logic model that identifies its immediate, intermediate term and longer term intended outcomes, other than those developed for this evaluation, and there is no performance measurement strategy that clearly identifies the indicators to be used to monitor progress on performance in relation to these intended outcomes.
Page details
- Date modified: