Joint Evaluation Report of the St. Lawrence Action Plan, 2016 to 2022

September 2024

List of tables

• Table 1. Constraints and impact on the evaluation
• Table 2. SLAP external communications, 2016–2021: Issues, objectives and target audiences
• Table 3. Table 3. Planned vs. actual communications spending, 2016-17 to 2021-22 (thousands of dollars)

List of figures

• Figure 1. Governance committee decisions are made in a timely manner to support operations
• Figure 2. Governance committees’ decisions are communicated in a timely manner to support operations
• Figure 3. Proportion of information gathered through reporting that is used to support decision-making or the achievement of SLAP objectives
• Figure 4. Users’ opinions on the operability and user-friendliness of the Oproma platform
• Figure 5. Alignment of SLAP external communication activities with participants’ needs
• Figure 6. Alignment of SLAP external communication activities with participants’ needs, by type
• Figure 7. Interest in being informed of SLAP’s activities and results
• Figure 8. Quality of the user experience on the SLAP website

List of acronyms and abbreviations

ASC

Agreement Steering Committee

AEC

Agreement Executive Committee

IC

Issue Committee

BIC

Biodiversity Issue Committee

UIC

Use Issue Committee

WQIC

Water Quality Issue Committee

ECCC

Environment and Climate Change Canada

Agreement

Entente Canada-Québec sur le Saint-Laurent 2011-2026 and Amendments 1 et 2

WG

Working Group

WGNEP

Working Group on Numerical Environmental Prediction

WGSSLM

Working Group on the State of the St. Lawrence Monitoring

MELCCFP

Ministère de l’Environnement, de la Lutte aux changements climatiques, de la Faune et des Parcs du Québec

SLAP

St. Lawrence Action Plan 2011-2026

RIMP

Regional Integrated Management Plan

CIP

Community Interaction Program

List of definitions

Joint actions

Projects developed under this Agreement that involve at least one Québec ministry and at least one federal department or agency, and that bring added value to the participants’ regular activities.

Collaborators

Any entities, other than the participants, with which the participants collaborate to achieve common objectives.

Decision makers

The Parties to this Agreement, as well as any other individuals or corporations wielding decision-making powers with respect to conservation.

Participants

Government ministries and agencies that may be involved in the joint actions carried out under this Agreement, as listed in section 8 of Appendix A – Management of the Agreement.

Departmental
representatives

Official representatives of participants in the Agreement.

Project leaders

Staff members of participants who take part in Joint Action Program projects.

Users

Individuals or corporations that use a resource or benefit from an ecological service of the St. Lawrence for commercial, subsistence, recreational, spiritual or other purposes.

Introduction

This report presents the results of a joint evaluation of the St. Lawrence Action Plan (SLAP), a partnership between Environment and Climate Change Canada (ECCC), the Ministère de l’Environnement, de la Lutte contre les changements climatiques, de la Faune et des Parcs du Québec (MELCCFP), and other participating departments and agencies of the Government of Canada and the Gouvernement du Québec.

The joint evaluation of SLAP addresses issues of governance, reporting and external communications during the period from 2016–2017 to 2021–2022. The evaluation was carried out in accordance with the requirements of the Treasury Board of Canada’s Policy on Results (2016) and the Gouvernement du Québec’s Directive concernant l’évaluation de programme dans les ministères et les organismes (PDF)(available in French only). It contains recommendations on opportunities for improvement. The joint evaluation complements the federal evaluation of SLAP carried out in parallel by ECCC’s Evaluation Division.

1. Scope

A preliminary analysis of documents and data as well as exploratory interviews (with representatives of organizations involved in SLAP governance and coordination) revealed the following areas of interest for the evaluation:

1. Decision-making and leadership, including the composition of SLAP committees and working groups;
2. Reporting;
3. External communications.

The exploratory work also confirmed that coordination activities and internal communications would not be evaluated. During the planning stage, the joint team also confirmed that the Community Interaction Program (CIP) would be excluded from the joint evaluation.

2. Evaluation questions

The evaluation questions are as follows.

• Q1. Does SLAP’s governance structure ensure effective decision-making?
• Q2. Is reporting effective?
• Q3. Are SLAP’s external communications effective?
  • (a) Are SLAP’s external communications objectives and activities well defined and known to participants?
  • (b) Do SLAP’s external communications reflect the achievements of participants?
  • (c) Are SLAP’s external communications carried out in such a way as to reach the target audiences?

3. Methods

The evaluation was based on mixed methods approach involving the use of several sources of primary and secondary information and data, both quantitative and qualitative.

Document and data review: This review covered the documentation provided by the SLAP Secretariat. It included SLAP management documents, work plans and reviews, web and social media analytics, and financial, administrative and performance data. An Internet search was also carried out to identify any other relevant public documents or websites.

Key stakeholder interviews: A total of 11 interviews were conducted with Agreement Steering Committee (ASC). Interviewees answered questions about governance, reporting and communications.

Survey: The survey was conducted between November 28 and December 20, 2022. A total of 123 responses were received and analyzed – 59 from staff members of SLAP participants and 64 from collaborators and users. Participant staff answered questions about governance, reporting and communications. Collaborators and users only answered questions about communications.

Case study: The study focused specifically on decision-making by the ASC. It included a document review and three group interviews with a total of 11 people, including members of the SLAP Secretariat, the Co-Chairs of the three Issue Committees and the Co-Chairs of the Working Group on the State of the St. Lawrence Monitoring.

4. Limitations and constraints

Table 1 shows the main constraints that affected the evaluation of SLAP.

Findings and main observations

5. Governance

• The governance structure supports the fulfilment of SLAP’s mandate by promoting information-sharing, consultation, and monitoring of program projects. The Issues Committees and Working Groups are carrying out their roles effectively.
• However, several members of the Agreement Steering Committee (ASC) are of the view that the Committee is not exercising its strategic role effectively. First, it is seldom used to discuss emerging issues or problems, which is expected under SLAP’s management framework. Second, the recurrent absence of representatives from partner ministries and agencies diminishes the ASC’s ability to fully exercise its strategic role.
• Follow-up on ASC meetings could be strengthened to improve the effectiveness of decision-making and operations aimed at delivering SLAP’s Joint Action Program.
• Opportunities for contribution to the SLAP are limited for First Nations and the municipal and community sectors.

5.1.  SLAP’s governance structure generally works well

SLAP’s governance structure is specified in the Agreement. It provides representation for all SLAP partners and stipulates their participation in the various committees. SLAP participants are the ministries and agencies of the Government of Canada and the Gouvernement du Québec that contribute human and financial resources to SLAP’s joint work and administration. SLAP’s governance supports the following in particular:

• Information-sharing;
• Examination of subjects and issues that sometimes require the intervention of several stakeholders from the governments of Canada and Québec;
• Coordination between partners from the Government of Canada and the Gouvernement du Québec.

The ASC manages SLAP and is accountable for that work. It is the largest management body. In addition to regular SLAP participants, collaborators may attend ASC meetings at the invitation of the Co-Chairs. Counting regular members and guests, each ASC meeting was attended by between 32 and 44 people during the 2016–2021 period. The ASC meets twice a year.

The Agreement Executive Committee (AEC) has to meet before each ASC meeting to prepare and approve the agenda. The AEC meets at least twice a year to follow up on the annual plan for communications products and activities, deal with administrative matters and operationalize ASC decisions. Between 2016 and 2021, each AEC meeting was attended by between 6 and 10 people.

The three Issues Committees (ICs) and the two Working Groups (WGs) each have two co-chairs and meet about twice a year. The ICs and WGs are responsible for ensuring that progress is made on the joint actions, reporting on the achievement of objectives, identifying obstacles to the execution of projects and opportunities for developing other projects, and reporting on their work to the ASC.

SLAP’s complex governance structure generally works well and ensures appropriate representation. Moreover, the decisions made by the ASC effectively address the participants’ concerns and interests: most participants (87%) who responded to the survey said that SLAP decisions address the concerns, issues and interests of the organization or group they represent. Participants who disagreed indicated that the definition of SLAP issues and priority orientations could be expanded to better reflect existing issues or additional issues. The issues that representatives of SLAP participants would like to see better integrated include sustainable uses (n=1), the relationship between water quality and human health (n=1), agri-environmental issues (n=3), and the overlap with climate change issues in general (n=3).

SLAP’s success is largely due to the establishment, maintenance and renewal of an interdepartmental and intergovernmental professional network. Because of the COVID-19 pandemic, ASC meetings are now held virtually. A number of participants felt that a return to in-person meetings would be desirable, as they felt that such meetings encourage discussion and the formation of professional networks.

5.2.  The Issues Committees and Working Groups are generally effective

The ICs and WGs oversee SLAP research projects effectively. Some see them as a good way of managing and collaborating with their colleagues from the other government. Under the Agreement, the ICs and WGs can make certain decisions, particularly those directly related to projects being carried out or to specific issues. However, some participants felt that the ICs and WGs could have more decision-making power. These suggestions seem to stem from a lack of understanding of the Agreement, which allows for the main initiatives suggested by stakeholders, namely: (a) seeking additional funding or scientific capability from other departments, and (b) piggybacking on other departments’ projects (e.g., infrastructure or measurement projects).

Lastly, in the past, project approval times and levels were sometimes problematic. From the information gathered, that was due to the participating governments’ internal processes and not to SLAP’s governance structure. According to survey respondents, the situation is not the same for all of the ICs and WGs. While a large majority of respondents (82%) felt that decisions by governance committees^{Footnote 1} were made in a timely manner to support operations, almost half (5 of 11) felt that decisions by the Working Group on Numerical Environmental Prediction (WGNEP) were never or only occasionally made on time (Figure 1).

Figure 1. Governance committee decisions are made in a timely manner to support operations

(Source: Survey of SLAP participants in 2022)

Some stakeholders also pointed out that the professional expertise of some governance committee members was not consistent with the requirements of their role in SLAP. This situation was due in part to reassignments and retirements.

5.3.  The Agreement Steering Committee is not exercising its strategic role effectively

The ASC’s role and responsibilities are spelled out in Appendix A of the Agreement. Its mandate is to ensure dialogue between participants, make strategic management decisions for SLAP, and direct reporting on projects and activities. ASC meetings include a roundtable discussion during which emerging issues, concerns and problems can be raised. In addition, during their presentations to the ASC, IC and WG co-chairs are responsible for reporting on obstacles facing current projects, collaboration opportunities, and any other concerns or topics of interest to SLAP.

However, a number of participants felt that ASC meetings did not provide an opportunity to discuss issues or introduce emerging concerns and problems of interest to participants. Other participants were in “listening mode” since the organizations they represent were not involved in projects or had not committed resources to SLAP. The ASC also experienced significant staff turnover during the evaluation period, resulting in a decline in participants’ commitment and recurring attendance by substitutes. A number of ministries and agencies were not represented at some meetings. Also, meetings were held virtually due to COVID-19, which negatively impacted engagement and participation.

Participants also noted that the suggestions they made at ASC meetings for improving SLAP sometimes went unheeded. Specifically, there was no discussion or decision on certain shortcomings observed in SLAP’s operation or certain opportunities to enhance its implementation even though the issues were raised explicitly and, in some cases, recurrently. In response to the survey, 16 participants mentioned having identified opportunities for improving SLAP. Among the 13 participants who communicated these opportunities to the ASC, seven (54%) consider that a follow-up was carried out and six (46%) consider that their intervention remained without an adequate response.

Among the reasons participants noted for the lack of adequate response were the lack of financial resources, the difficulty of obtaining the necessary authorizations in a timely manner, and the lack of leadership by the ASC. The length of time required for decisions and follow-up was also mentioned.

It should be noted that the ASC’s role and strategic responsibilities were not known to all participants. This lack of awareness probably had an impact on the ASC’s difficulty in fulfilling its strategic role. This suggests that bolstering the ASC’s strategic role is not so much about revising the governance structure as about making adjustments to the way in which meetings are held and improving participants’ knowledge and understanding of SLAP’s management framework, including their role within it.

In addition, it would be useful if the ministerial representatives on the ASC had a designated substitute at an equivalent hierarchical or decision-making level to replace them when they are unavailable. Substitutes must have not only an understanding of SLAP’s governance structure but also the power to make decisions for the organization they represent in the same way as the primary member. Although such a system of designated substitutes is provided for in the Agreement’s management framework, in practice, either there are no substitutes, or the available substitutes only have the power to act as observers.

5.4.  Organizational changes have had an impact on action items follow-up for two committees

ASC, IC and WG meetings provide an opportunity to monitor project progress, take advantage of collaboration opportunities, and find solutions to the various obstacles that may affect the schedule and work objectives.

A majority of SLAP participants (over 70%) said that governance committee decisions were made in a timely manner to support operations. In addition, over 75% of participants felt that decisions made by the ASC, AEC, UIC, WGSSLM and WGNEP were always or usually communicated in a timely fashion (Figure 2).

The BIC and the WQIC are two notable exceptions to this rule. Almost all survey respondents (85%) said that the committees’ decisions were never or only occasionally communicated in a timely fashion. This was mainly due to a shortage of project team personnel, caused by organizational changes in some departments that had a key role in the activities overseen by these ICs.

Although decisions made by the ASC often give rise to follow-up actions, for a number of years, participants have been receiving meeting minutes several months after the meetings were held. According to some participants, this delay affected their ability to follow up on action items in a timely manner. However, the situation was rectified recently, following the ASC meeting in June 2022. Participants felt that this change had a positive effect on the ICs and WGs and more broadly on the management of SLAP.

Figure 2. Governance committees’ decisions are communicated in a timely manner to support operations

(Source: Survey of SLAP participants in 2022)

5.5.  Opportunities to contribution to the SLAP are limited for First Nations and the municipal and community sectors

According to the Canada-Québec Agreement, only participating ministries and agencies that contribute funds to SLAP activities are officially represented in the management structure. Thus, the First Nations whose traditional territory extends to the St. Lawrence or who have an interest in the sustainable management of its ecosystem do not have representation at this level in the SLAP.

Consulted representatives are of the view that participation opportunities for First Nations are limited. However, representatives who expressed this opinion did not specify what mode of participation would be advisable for First Nations. In addition, a First Nation has indicated its interest in collaborating with the SLAP.

Furthermore, SLAP participants also indicated that the concerns and interests of the municipal and community sectors could be better reflected in the programming of joint actions.

6. Reporting

• Generally speaking, the reporting process is effective and helps in tracking project progress and achievements.
• Project leaders consider the workload associated with SLAP reporting to be significant.
• Not all the information collected is used to support decision-making or the achievement of SLAP objectives.
• A number of users find the electronic reporting platform difficult to work with.

6.1.  The reporting process enables effective project monitoring

SLAP reporting requirements are set out in the Agreement. The purpose of reporting is to provide the ASC with information on project progress and achievements. The role of the IC and WG co-chairs is to gather information from project leaders on the progress and achievements of their projects and submit that information to the ASC. Coordinated by the SLAP Secretariat, reporting has three main phases:

• Mid-year status report;
• Year-end status report and annual review; and
• Planning for the next year.

At the end of the financial year, project leaders have to enter information about the progress of their project(s) in the Oproma platform. The reporting covers activities, deliverables, communication products, and the budget. Project leaders also have to update the status code (green, yellow or red). In so doing, they also make it easier to compile information about the progress of all projects on an annual basis.

Generally speaking, interviewees said that the reporting process was effective and made it possible to see the achievements of the different projects through the various reviews. The Oproma platform simplifies the reporting process and eliminates the need for project leaders to compile information manually. Reporting to the ASC twice a year provides a qualitative picture of project achievements, which can be used to report concrete results to SLAP’s target audiences.

6.2.  Not all the information collected is used

Some participants interviewed were unable to comment on the use of the information collected for reporting purposes or on its usefulness in supporting SLAP objectives. Others felt that the value of the large amount of information collected was unclear. Survey respondents, on the other hand, regarded it as useful. The majority of participants (54%, n=41) felt that a high or very high proportion of the information gathered through reporting supported decision-making and/or the achievement of SLAP objectives. Conversely, almost one in five respondents (17%) said that only a small proportion of the information gathered was actually used (Figure 3).

Figure 3. Proportion of information gathered through reporting that is used to support decision-making or the achievement of SLAP objectives

(Source: Survey of SLAP participants in 2022)

More than half of the participants (n= 32 to 47) said that most or all of the information collected was used to (1) help SLAP participants identify risks to the achievement of SLAP objectives (68%), (2) support decision-making by the co-chairs of the ICs, WGs and Coordination Committees (63%), and (3) support decision-making by project leaders (60%).

More than half of the participants (n= 32 to 47) said that none or only some of the information collected was used to (1) help SLAP participants identify collaboration opportunities (58%), (2) support decision-making by the Agreement Co-Chairs (56%), and (3) support decision-making by members of the Agreement Secretariat (53%).

6.3.  Reporting workload is significant

The hours devoted to reporting vary depending on the reporter’s role in SLAP. The key members of the SLAP Secretariat^{Footnote 2} spend an average of 328 hours per person per year on reporting, or almost 18% of total hours worked. Most project leaders and IC and WG co-chairs (85%) spend 20 hours or less per year on reporting. Some participants interviewed said that they considered the reporting workload to be significant, and that it reduced the time they were able to spend on projects.

In addition, almost all partners involved in SLAP have to carry out additional reporting exercises for the same activities: 72% said they had to complete two reporting exercises, and 25% said they had to complete three or more. The additional exercises are requirements specific to the internal processes of each ministry or agency. Different types of information are gathered during those exercises, and reducing their number does not appear to be possible.

Since reporting takes up valuable time and not all the information gathered appears to support decision-making, SLAP has the opportunity to streamline this information by focusing exclusively on what is clearly consistent with decision-makers’ information needs and SLAP’s objectives. Such streamlining could lighten the workload for the participants and free up time for Joint Action Program project execution and SLAP administration as a whole.

6.4.  Oproma platform is difficult to use for several participants

Before the Oproma platform was put into service, the Secretariat had to manually compile project information to carry out reporting exercises. Although the adoption of the platform has improved the efficiency of the process, several participants have difficulty using the Oproma platform. Many users (n=31) said they had difficulty finding the information they were looking for (23%) and the functions they wanted to use (31%). An even larger proportion of users found it difficult to carry out the required operations easily (39%) and quickly (42%). In addition, the majority (61%) of users never or only occasionally enjoyed their experience with the Oproma system (see Figure 4).

Figure 4. Users’ opinions on the operability and user-friendliness of the Oproma platform

(Source: Survey of SLAP participants in 2022)

Underlying causes of usability difficulties include high staff turnover among SLAP participants and the lack of effective knowledge transfer, either between co-workers in the same ministry or agency or between the Secretariat and new participants – notably through a personalized welcome on joining SLAP.

7. External communications

• SLAP external communications are guided by an overall strategy, but that strategy does not have a performance measurement framework. It is impossible to determine whether objectives have been achieved.
• Partners said that external communications activities did not meet their needs. Allotted resources were not fully utilized.
• Collaborators and users said that they received little or no information on the SLAP, despite their high level of interest in its activities and results.
• Despite efforts and a website redesign, navigation remains difficult; some content is out-of-date, and the News section is seldom used. SLAP does not have its own social media accounts and only occasionally uses government social media accounts. In addition, partners do not further disseminate SLAP publications.

7.1.  SLAP’s external communications are guided by a strategy

The Communications Protocol appended to the Agreement sets out the rules governing communications, the roles and responsibilities of communications managers and advisors, and SLAP’s branding standards. In 2017, SLAP adopted a communications strategy for Phase VI (2016–2021). The 2016–2021 strategy identifies five priority communication issues and associated objectives, and seven target audiences (see Table 2).

(Source: SLAP’s five-year communication strategy, 2016–2021)

Under the 2016–2021 strategy, social media are used for messages on theme days and more specific announcements to promote SLAP’s activities, achievements and programs. The most commonly used platforms are Facebook, X (formerly known as Twitter) and LinkedIn.

The content posted between 2016 and 2022 was varied and included notices about particular theme days or weeks related to SLAP, announcements of new funding related to SLAP activities, calls for projects for CIP funding, general information about the St. Lawrence and its biodiversity, and quizzes and anecdotes about the St. Lawrence ecosystem.

In 2017, SLAP’s communications advisors created a social media publishing guide. In the fall of 2022, they also developed forms to facilitate the collection of information from project leaders whose projects could be described on social media or on SLAP’s website.

7.2.  The performance of SLAP’s external communications is unknown

Every year, SLAP’s communications advisors prepare the work plan and complete a year-end review. The review of communications activities between 2016-17 and 2021-22 documents activities in four areas: Strategic Communications (communications plan for new programming, website redesign, etc.), Web, General Publications (SLAP’s Five-Year Report, etc.), and Sector (SLAP project-related products). Most of the activities in the annual work plans were completed.

The 2016–2021 strategy does not provide specific action plans for each communication channel and target audience. The approach taken by the communications advisors is to develop ad hoc communications strategies in consultation with members of the Secretariat, the IC and WG co-chairs, and project leaders.

External communications activities have no results indicators or targets, and their performance is not measured, with the exception of some measurements of website traffic and engagement with social media publications. Consequently, it is impossible to determine whether they are improving SLAP’s visibility and reaching its target audiences effectively, for example by making content accessible and easy to understand.

7.3.  External communications are not aligned with participants’ needs

SLAP’s external communications activities are planned each year in consultation with SLAP’s partners. The annual reviews show that most of the activities in the annual work plan are carried out during the year. Despite the delivery of expected products and services, participants said that SLAP’s communications activities were not consistent with their needs as described in the 2016–2021 communications strategy.

In interviews and in response to the survey, a large proportion of participants said that there were too few communications activities, and that SLAP continued to lack visibility. In particular, they indicated in the survey (n=23) that external communications were not frequent enough (31%), that communications products lacked visibility (19%), that timelines were too long (12%), and that they did not get enough support from the communications team (12%).

Nearly two-thirds (60%) of participants felt that the activities did not meet or fell short of their needs (see Figure 5), for all objectives (see Figure 6). Participants did not know whether their target audiences were actually being reached, and some of them would have liked more support from the communications advisors. External communications were also discussed in the ASC on several occasions; nine ASC members out of eleven (82%) are of the view that the SLAP lacks of visibility and emphasized the importance of popularizing scientific knowledge.

Figure 5. Alignment of SLAP external communication activities with participants’ needs

(Source: Survey of SLAP participants in 2022)

Figure 6. Alignment of SLAP external communication activities with participants’ needs, by type

(Source: Survey of SLAP participants in 2022)

Nearly two-thirds (64%) of participants felt that SLAP’s external communications were rarely or only sometimes timely. The length of time required to approve and disseminate communication products was the main reason identified for the difficulties encountered. In particular, delays in approving certain products, such as reports, were attributed to the fact that their dissemination required the approval of two governments.

Participants offered some suggestions for improving SLAP’s external communications. More regular presentations by the Secretariat’s communications team and a communications plan for reaching the various partners (e.g., experts in the various departments, local players) more effectively were suggested. This would address the problem of inadequate knowledge of the strategy, which is at least partly due to staff turnover among SLAP partners. Participants also said that workshops specifically for the general public and forums similar to the St. Lawrence Forum (last held before the COVID-19 pandemic) could be useful. Some participants identified issues or challenges specific to external communications:

• Scientific information that is difficult to communicate to the general public;
• A lack of tangible field results to communicate;
• The challenge of communicating project actions and results to several target audiences, including municipal and community organizations and citizens’ committees.

7.4.  External communications funding was not fully utilized

For the 2016-17 to 2021-22 period, MELCCFP’s budget for SLAP communications was $580,800, and ECCC’s was $681,200. These budgets represent the two departments’ total financial commitment to communications, including communications products and human resources expenses. A tenth (10%) of the total funding was not spent. Participants felt that not enough resources were devoted to communications, when in fact a portion of the resources was not used, for both operations and human resources. Table 3 below shows planned and actual spending on communications products and human resources, for the two ministries.

(Source: Data provided by the SLAP Secretariat)

MELCCFP’s actual spending on communications operations was $23,700 less than planned spending. For ECCC, the difference was $95,900, and it is explained by the impact of COVID-19 on scientific research activities, some of which could not be carried out as planned. For this reason, the actual volume of communications work was less than anticipated and the use of external service providers was less than expected. Regarding salaries, MELCCFP spent all planned expenditures and the gap between planned and actual spending was only $7,600 (1 %) for ECCC.

7.5.  The level of interest in SLAP is high among collaborators and users, but they feel poorly informed

Both collaborators and users expressed a high level of interest in SLAP’s external communications. Almost all collaborators and users (98% of the 59 respondents) were moderately or very interested in being kept informed of SLAP’s activities and results, particularly with regard to scientific information, field activities, monitoring the state of the St. Lawrence, and decision-making tools. A smaller proportion (66%) expressed interest in information about SLAP’s governance (Figure 7).

Figure 7. Interest in being informed of SLAP’s activities and results

(Source: Survey of SLAP participants in 2022)

However, a majority of collaborators and users (54%) said they were poorly or not at all informed of SLAP’s activities and results. This suggests that current communications activities are not effective at reaching some target audiences.

7.6.  Use of the SLAP website and social media could be improved

The launch of SLAP’s new website in February 2021 was the culmination of several years of work on updating SLAP’s digital presence. The five-year communications strategy for 2016–2021 introduced the online publication of factsheets and plain-language summaries to improve the dissemination of project achievements. Under the strategy, those publications were to be posted on the SLAP website, with references to them on social media. Templates for disseminating results were developed to ensure product uniformity.

The SLAP website hosts a variety of factsheets and plain-language summaries. At the time of writing, there were about 100 factsheets and summaries on the website. The templates are being used, and products are generally uniform. A review of the documentation and the website confirmed that the “Catalog of publications” tab on the website provides easy access to various SLAP-related publications. Searches can be filtered by various criteria, such as year of publication, alphabetical order or geographical area.

Most users and collaborators (97%, n=62) and participants (86%, n=50) did not visit the SLAP website regularly (total: n=112, 92%). Among those who visited occasionally, a majority (n=73 to 78, depending on the statement) said they were satisfied overall. However, one in four users (25%) only occasionally or never enjoyed the user experience, and more than one in three (37%) only occasionally or never found up-to-date information (see Figure 8).

Figure 8. Quality of the user experience on the SLAP website

(Source: Survey of SLAP participants in 2022)

Despite efforts to redesign the website and make it more efficient and accessible, improvements are still desirable to enhance its quality and user-friendliness. Half of the participants (50%, n=32) felt that changes should be made to the website to improve the user experience. Navigating the website remains difficult for the uninitiated, because of the many sub-branches and the number of clicks required to access information. Some content is repetitive or out of date; for example, information about the WGNEP was out of date at the time of writing, the WG having completed its activities in 2019. Postings in the “News” section of the website are also infrequent. A total of 10 articles were published between 2020 and 2022: four in 2020, four in 2021, and two in 2022. The pages do not include publication or modification dates, nor do they offer the option of contacting a webmaster to report a problem or erroneous information. To do this, visitors to the SLAP website have to go to the “Contact us” page and choose one of the communication options. None of the options is clearly associated with website management.

The website redesign did not result in an increase in page views. With the exception of 2017 and 2022, for which partial data are available, website traffic measured by the number of page views remained relatively stable between 2018 and 2021 inclusive, at between 54,000 and 64,000 page views per year.

The average bounce rate^{Footnote 3} for the website’s main pages was between 51% and 66% per year, which is considered medium to high. This means that, on average, between 51% and 66% of visits did not result in any exploration of the website. The available data do not indicate how long the bounce visits lasted, which would have made it possible to estimate how many people visited a web page with no real interest or by mistake (short duration) or deliberately to find information and learn more (long duration).

SLAP relies primarily on its website for its online presence. However, there is reason to question whether this is a wise decision. Most participants, collaborators, and users (92%, n=112) who responded to the survey did not visit the SLAP website regularly, and the website redesign did not result in an increase in page views.

Furthermore, SLAP is not authorized to have its own accounts on social media platforms. To publicize its activities, achievements and results, it occasionally uses the ECCC and MELCCFP departmental accounts. Contrary to the 2016–2021 strategy, ECCC and MELCCFP publications about SLAP were not picked up by other participants.

Recommendations and joint response by the co-chairs of the agreement

The following recommendations are addressed to the co-chairs of the Canada-Quebec Agreement on the St. Lawrence, respectively the Director General of Freshwater Management Division at ECCC and the Assistant Deputy Minister of Water and Air Policy and Expertise at MELCCFP, who are the senior officials responsible for the St. Lawrence Action Plan.

Recommendations

Joint response from the co-chairs of the Agreement

We have taken note of the joint evaluation report of the St. Lawrence Action Plan for the period between 2016-2017 and 2021-2022 as well as the recommendations made.

The governments of Canada and Québec have been collaborating for 35 years now on the conservation and development of the St. Lawrence through successive Agreements which bring together around ten ministries from each government.

The mandate of the joint evaluation of PASL focused on governance issues and external communications.

The two governments attach great importance to good governance of PASL bodies as well as to the communication and dissemination of the results resulting from the Agreement to all partners, collaborators, and users.

Thus, the recommendations made will be studied by the teams of the two governments in order to identify the possibilities of taking them into account by the end of the current Agreement (2011-2026) and consider those which can feed into the development of a future Agreement.

In closing, we would like to highlight the work carried out by the joint evaluation team and thank all the staff members, PASL partners and users who provided their observations.