Evaluation of the enforcement modernization initiative: report

1. Introduction

This report presents the findings and recommendations of the Evaluation of the Enforcement Modernization Initiative (EMI). The last evaluation of the EMI was conducted in 2009-2010 under the “Enforcement Program” heading. The rationale of the present evaluation stems from it being a departmental priority. A commitment to complete this evaluation was included in a submission to the Treasury Board. It covers the four-year period from 2020-2021 to 2023-2024 of the implementation of the EMI under the following themes: alignment with departmental priorities and efficiency, design, delivery, change management, performance measurement framework, and effectiveness – achievement of expected results. Information for this evaluation was collected from March to July 2024.

1.1. Background

Environment and Climate Change Canada (ECCC) is dedicated to the protection and conservation of the environment and wildlife for current and future generations. The Enforcement Branch (EB) has an important role in monitoring and enforcing compliance with numerous federal environmental and wildlife protection laws and regulations. 

The EB is led by the Chief Enforcement Officer and comprises the following directorates:

• The Environmental Enforcement Directorate (EED) is responsible for enforcing federal legislation that addresses a variety of risks to the environment and human health, mainly caused by pollution. 
• The Wildlife Enforcement Directorate (WED) is tasked with enforcing federal wildlife legislation to protect plant and animal species and their habitats in Canada, including federally protected habitats. 
• Officer Development and Branch Standards (ODBS) oversees officer training, safety, information management, and operational policies. It is essential for EB’s effective functioning.
• The National Policy and Priorities Directorate (NPPD) is responsible for leading and coordinating the risk-based review of all laws and regulations under the EB’s enforcement authority. This directorate is also responsible for data science, policy analysis and research, coordinating the development of the Integrated Enforcement Plan (IEP), and performance reporting. 

To facilitate operational activities across Canada, both EED and WED set a national headquarters, located in the National Capital Region, and are present in 5 administrative regions (Pacific and Northern Region, Prairie and Northern Region, Ontario Region, Quebec Region, and Atlantic Region). There are approximately 260 environmental enforcement and wildlife enforcement officers (EOs) located across Canada. 

EOs are granted the powers of Peace Officers with special authorities under the Criminal Code. Under applicable policies, they carry out inspections and intelligence activities to ensure that businesses, industries, and individuals adhere to laws and regulations under the mandate of ECCC. When violations are suspected, officers conduct investigations to gather evidence. They employ non-court measures to enforce compliance, such as issuing warnings, tickets, administrative monetary penalties, and environmental protection compliance orders. In cases where a court action is necessary, they may recommend that the Public Prosecution Service of Canada prosecute alleged offenders. 

EB also works in partnership with many law enforcement and external partners, such as other government departments, provincial and territorial governments and their enforcement agencies, Indigenous communities, and international partners, to enforce ECCC’s laws fairly, predictably, and consistently. 

Furthermore, EB collaborates with ECCC branches, such as the Environmental Protection Branch and the Canadian Wildlife Service, in their roles to assess and manage risks through regulatory development and compliance promotion. EB also works closely with the Science and Technology Branch (STB). STB scientists provide important services such as analytical and laboratory services on samples provided by EOs to determine non-compliance. STB scientists are also called upon as expert witnesses in instances where their work is used as evidence in court actions. 

1.2. Overview of the Enforcement Modernization Initiative

Due to increased federal commitments to environmental protection and conservation, EB’s mandate has grown by 40 percent since 2009, including a 23 percent increase in square kilometers conserved. EB’s funding has remained generally stable despite the growth in mandate. We were informed that EB was able to maintain enforcement service levels by deferring investments in areas such as training and equipment purchases. 

In 2020-2021, in response to EB’s growing needs, the department secured $51M (with EB receiving $33M) over five years and $10M (with EB receiving $7M) ongoing to modernize the enforcement of applicable laws and regulations. To address its broad and complex mandate and contribute to the implementation of past audit recommendations, EB developed a risk-based approach. This approach is used to assess and identify where non-compliance can do the most environmental harm, to focus on high-risk activities and/or potential violators, and to prioritize EB’s efforts to reduce environmental and conservation risks identified. This new approach involved a change in all aspects of EB’s work, including roles, responsibilities, and processes. 

Under the risk-based approach, each year, EB sets priorities and creates an IEP that lays out inspection priorities by region and intelligence and training priorities for EED and WED, such as national and regional risk-based projects, maintenance/compliance monitoring, reactive work, reinspection and other priority projects. The IEP is based on five principles. It is:

• evidence-based
• risk-based
• operationally advisable
• transparent and accountable
• adaptive

Figure 1: Enforcement Branch Priority Planning Process

Source: The Enforcement Modernization Initiative Midterm Update, 2024.  

Several supporting projects were funded with the goal of establishing the foundation for modern, data-driven, and risk-based enforcement. The following projects were included in the present evaluation:

• Developing an Enterprise DataHub that would facilitate the transfer of enforcement data to a Protected B cloud-based environment.
• Using Microsoft Power BI to support integration of performance data for analysis and real-time monitoring of results.
• Developing and implementing a computer forensics laboratory. This project was intended to augment capacity to meet evidentiary standards in support of investigations.
• Creating a forum to align priorities and cooperation with key federal, provincial, and territorial enforcement partners and Indigenous counterparts. The aim was to have better information exchange and cooperative approaches and to potentially reduce the duplication with provinces and territories.

1.3. About this evaluation

The evaluation was conducted in accordance with the 2016 Treasury Board Policy on Results. The following methods were used in completing the evaluation:

• Document and Data Review: Examined documents including process documentation, project plans and assessment reports, and financial data.
• Case Studies: Reviewed EED and WED documents for 13 national priority projects.
• Benchmarking Analysis: A limited analysis of similar enforcement functions at Parks Canada and the Department of Fisheries and Oceans was completed.  
• Survey: A survey was conducted to capture the regional and operational perspective. The survey had a 53 percent response rate (128 of 242) for questions that all respondents were asked to answer. There was an 81 percent response rate associated with questions directed at manager-level respondents (30 of 37).
• Key Informant Interviews: Interviews were conducted with EB staff from all divisions and with ECCC internal partners.
• Site Visits: Two site visits were conducted at EB regional offices to gain an understanding of the operational realities for both WED and EED officers.

The following items were excluded from the evaluation scope:

• The work related to the Chief Review Officer
• Prosecution costs
• Program integrity components (for example, the Officer Safety Management System and Mobile Office and equipment)

Finally, the financial data could not be split according to the specific scoping item of the evaluation, which limited the financial analysis of the various elements of the initiative.

2. Findings

2.1. Alignment with departmental priorities & efficiency

Key findings: The need for an Enforcement Branch modernized enforcement approach and for a focus on risks was evidenced. The modernization initiative is aligned with departmental priorities. However, the survey results indicate that there is an opportunity to enhance priority alignment.

2.1.1. Alignment to ECCC and Government of Canada priorities 

The EMI is aligned to ECCC and federal government priorities. EB provides an essential role of enforcing the laws, regulations and enforceable instruments under ECCC’s mandate. By identifying and addressing high-risk potential non-compliance areas, EB supports the achievement of various ECCC and federal government goals, such as climate change emissions targets, addressing toxic substances, and protecting species at risk. 

Implementing a risk-based approach to enforcement was intended to optimize the use of limited resources and identify where regulations were functioning or not functioning. However, several interviewees suggested that improvements should be made to the prioritization of specific commitments to enhance alignment with federal government goals and priorities.

2.1.2. Gender-Based Analysis Plus  

A Gender-Based Analysis Plus (GBA Plus) analysis of EMI was completed at the outset. The analysis concluded that protecting the environment benefits all Canadians and there would not be any differential impacts from modernization on various groups within Canada. At design, EB outlined considerations for appropriate GBA Plus data collection and reporting plans for the initiative to support conducting enforcement activities in a fair, consistent, and predictable manner. 

EB also produced an analysis that took enforcement data and overlayed it with data from vulnerable communities to assess risks. Future opportunities to integrate this analysis into inspection planning are being explored, alongside work in the priority area of Environmental Justice and Anti-Racism. EB also has ongoing work to build collaborative approaches with Indigenous communities. This is outlined in section 2.6 of this report.

A GBA Plus project also took place within EB from an operational perspective. A survey was administered in May 2023 on GBA Plus considerations for operational employees to understand barriers and potential operational biases. The survey results demonstrated needs in training and equipment, among other actions, to enhance inclusiveness. The branch continues to act on survey recommendations and report on progress. 

2.1.3. Financial resource use 

While the initiative is geared towards generating a more proactive approach to enforcement based on data, availability of financial information on the modernization is limited. It was not possible to track all expenses by category or precise scoping elements of the evaluation. Therefore, a detailed financial analysis could not be performed at that level. 

2.1.4. Efficient use of resources and measures to improve efficiency

Interviewees had mixed views on whether the risk-based approach had improved efficiency. Also, 58 percent of survey respondents reported that the IEP process has not enabled an efficient allocation of EB resource, has not improved branch efficiency, and has not led to an efficient achievement of expected results. These results are displayed in Figure 2.

Figure 2: Responses to questions on efficiency

Some views on the sub-optimal use of resources included: 

• The perception that national projects did not yield expected results in terms of identified non-compliance. 
• The perception that the addition of field EOs would have contributed more to the identification of non-compliance. 
• The lack of operational experience of analysts in the IEP process.  
• The lack of funding and staff in the regions to participate in the IEP process.
• The misalignment of budgeting and costing for projects. 

It was highlighted that one efficiency gain from the initiative was the use of the Commercial Trade Database (CTD) and Commercial Trade Explorer (CTE) to decrease the time required to collect and analyze information manually using data science, predictive analytics, and machine learning. Additionally, developing artificial intelligence algorithms in the CTD and CTE allowed EB officers the ability to track changes in supply chain analysis that would not otherwise have been possible.

2.2. Design

Key findings: The Enforcement Modernization Initiative is meeting its principles of being risk/evidence based. While the integrated planning process has delivered annual inspection, training, and intelligence priorities in a timely way, there are opportunities for refinement in the processes.

The design of the risk-based model considered lessons from similar initiatives, both internal and external to the Government of Canada. The design team consulted independent experts from across Canada on the risk assessment methodology. In 2020, a series of consultations with internal stakeholders, external stakeholders from other government departments and agencies, and international counterparts were completed, and the foundational principles of the risk-based approach were developed.

The risk-based model was designed to identify potential high-risk non-compliance areas and gaps in information and intelligence. As described in section 1.2, modernization resulted in a centralized, risk-based approach to planning and prioritizing enforcement activities. A proportion of risk-based inspections are now being planned and undertaken in EED and WED each year. We also found evidence of risk application in other categories of inspections as well. For example, effective collaboration resulted in a reusable system to prioritize one regulator’s referrals to EB based on risk.

Threat Risk Assessments (TRA) and the Operational Intelligence Assessments (OIA) are evidence-based methods that draw on data from within and outside of ECCC. For example, in the Vehicles and Engines TRA, the use of a new methodology improved the accuracy of statistics on Canada Border Services Agency data derived across the vehicle and engine group’s imports from 15 to 20 percent to 80 to 100 percent.

The technological support (Enterprise DataHub, CTD, and CTE) built within the initiative is expected to significantly improve data analysis capability. For example, the CTD and CTE now allow for real-time non-compliance targeting of commercial trade.

While recognizing that the IEP process is based on the principles of evidence and risks, 53 percent of survey respondents (n=16) did not completely agree that the design was appropriate to achieve expected modernization results. While the design is aligned with the IEP evidence-based principle, the risk rankings and risk-based decisions are limited by the data they are based on. Interviewees noted that the limitation of the design sometimes resulted in scenarios where EB operations did not agree with the high-risk areas/commodities/species and inspection targets that were identified. In some cases, EB operations were not able to successfully identify large violations. Greater explanation and discussion of the findings of both TRAs and OIAs would be beneficial for EB operations. This would allow for a better understanding of how data and indicators impact rankings.

It is also worth noting that the TRAs are not easily adaptable to emerging risks outside of the three-to-five-year cycle of assessments (for example, new regulatory packages prepared to address newly identified risks). However, there are other mechanisms available to address emerging risks, including information gathering projects, regional projects, and OIAs.

Furthermore, social and economic changes may also impact compliance levels, and the impact of moving from regularly scheduled inspections to ad-hoc risk-based inspections is not yet clear. While EB recognized the need for a monitoring policy that allows them to detect these impacts, no policy was developed at the time of the evaluation.

2.2.1. Planning cycle

The integrated planning process outlines key phases as well as a schedule with outputs and milestones. The planning teams collaborate to deliver the annual IEP on schedule. The IEP outlines the list of high-risk national and regional program inspection projects and targets as well as EED and WED intelligence and training priorities. Potential refinements were identified for EB’s consideration in enhancing the annual plan and project cycle.

The transition from inspection and resource planning to a risk-based model requires comprehensive planning of all activities. Resources, priorities, and delivery of the plans may be significantly impacted by the unpredictable level of reactive inspections to be conducted (that is, inspections in response to referrals, complaints, etc.). EED’s recent reactive inspection targets were low compared to the actual number conducted. Therefore, it is important to ensure realistic planning targets for reactive inspections. Moreover, both directorates engage in non-enforcement field activities including training and investigations. Investigations are time sensitive and time consuming, which could lead to diverting staff for major cases and other priorities. While WED plans officer efforts in the IEP for non-enforcement field activities, the EED does not. There is an opportunity for EED to make the planning process more inclusive of all activities.

The annual enforcement planning is dependent, amongst other things, on the department’s annual budgeting process. The different timing of the two processes generates project and resource management challenges. A recent review of the IEP process from 2024 (referred to as the 2024 report hereafter) suggests that the IEP and regional plans should be adjusted in the spring, once budgets are allocated, to ensure the plans accurately reflect available resources.

An EB survey of IEP team members indicated a balanced opinion on the manageability of the timelines. However, a majority (57%) felt they had sufficient time and resources to inform and align their respective groups with the planning process. Factors contributing to time pressure include the part-time role of IEP planning team members, project presentation and feedback periods during summer/field season and holiday breaks (summer and December), and the additional effort required to track work reported by WED.

Finally, we found that a longer IEP planning cycle or a longer project planning cycle may be beneficial. Additional time could help improve engagement with operations and other stakeholders/partners, such as by integrating regional and regulation-specific intelligence; enhance understanding and meaningfully integrate input in key products; allow time for development and delivery of training for national projects; and allow for adequate entity or target validation prior to inspections.

2.3. Delivery

Key findings: While the Enforcement Modernization Initiative design may prove to be effective in the long term, implementation during the evaluation period has prevented Law Enforcement Branch to fully achieve modernization objectives. Progress and improvements have been made towards areas such as effective communications. It is also noted that collaboration and engagement with the Integrated Enforcement Plan planning teams and partners remain important going forward. There are opportunities to clarify roles and responsibilities for all steps in the process, but particularly the intelligence function, to ensure roles are clear and collaboration, engagement, and transparency are improved.

This section details the findings with regards to operationalization and collaboration over the various steps of the risk-based prioritization, planning process, and overall delivery of the modernized approach. 

2.3.1. Governance

The Enforcement Branch Directors Board, which consists of all EB executives and Regional Directors, approves the IEP. Planning teams are responsible for developing the enforcement priorities and projects for EED and WED. The planning teams have representation from across EB and meetings and decisions are well documented. Overall, current governance supports efficient delivery of the initiative. However, collaboration and engagement remain areas that could be improved. Recently, two Executive Director positions were created (one in each of EED and WED). These positions co-chair the IEP planning teams, and their roles and responsibilities were clearly established. The co-chairs are responsible for enhancing understanding, incorporating operational input into project plans, and working with Regional Directors to implement projects. The 2024 report notes that the nomination of the two co-chairs was viewed as positive. This was confirmed by interviewees who agreed that the role has helped raise consideration of operational perspectives. However, there was a perception that the Executive Directors may have added a layer of approvals, and that Regional Directors may now report to these positions.

2.3.2. Roles and responsibilities

Clear definition of the roles and responsibilities of each participant in the implementation of the IEP planning process is important to promote EMI understanding and to get buy-in for change. 

Roles and responsibilities may not have been properly defined (or accepted) at the beginning of the modernization initiative. Approximately half of the survey respondents did not agree that the IEP process had clearly defined roles and responsibilities at the onset of the initiative. The evaluation noted attempts to foster better understanding of responsibilities for each position in the process. For example, EB updated the Terms of Reference for IEP planning teams to address unclear expectations and accountabilities identified in implementation. While some improvements have been noted over time, some roles and responsibilities remain to be further clarified as there is still some incomprehension and/or disagreement at all levels for various steps of the process. For example, there are currently three separate Intelligence bodies (Strategic Intelligence and Risk Assessment Division (SIRAD) and national and regional intelligence) within EB. The lack of clear roles and responsibilities has impacted communication and the timeliness of some enforcement actions and, in some instances, has resulted in targeting non-operational sites.

Additionally, the 2024 report suggests:

• Clarifying roles and responsibilities for data collection on motives for non-compliance
• Defining roles for intelligence groups, SIRAD, and national intelligence, including the referral process for risk-ranked referrals.
• Greater delineation of roles for project plan development, project management, and feedback.

We also find that the review identifies a recommendation to develop a Responsible, Accountable, Supportive, Consulted, Informed matrix, but this had not been actioned by EB at the time of writing this evaluation report. 

2.3.3. Flexible delivery, collaboration, and inclusion of relevant stakeholders 

Collaboration within EB and amongst partners is essential at every step of the development and implementation of the IEP. Interviewees and survey respondents suggest that internal collaboration at the planning table and within the Intelligence function has not always been optimal. 

The survey and interview results demonstrated that there is an overall disagreement that the IEP is meeting the principles of being operationally advisable, adaptive, and transparent (see Figure 3 for details). This was attributed to a need to enhance collaboration through better operational engagement and consideration of inputs in the process. In the same way, improved communication of IEP results through closure reports and engagements would allow for a better understanding and transparency. Interviewees noted that the initiative lacked flexibility in adapting to implementation challenges throughout the process. The limits to adaptability were attributed to not sufficiently incorporating operational input, not sufficiently acknowledging regional differences in planning, and the limited ability of the risk rankings to adapt to emerging risks.

Figure 3: Perceptions on whether the Integrated Enforcement Plan meets its principles (n=30) 

 Note: Categories may not add to 100 percent due to rounding. 

There were mixed views on early, meaningful operational engagement in the process (TRAs, OIAs, and national project plans). The incorporation of input/feedback and validation of the key products’ findings (TRAs, OIAs) has impacted implementation. Many planning team members disagreed that there was sufficient operational input in the development of national projects in the 2023-2024 EB survey. Some mentioned that EB’s regional offices had varying contexts, capacity, and resources and that these operational realities and regional intelligence were not always considered.

Over time, EB has implemented measures to facilitate timely and meaningful engagement and collaboration with NPPD, IEP planning team members, enforcement operations, and other stakeholders and partners. These efforts focus on strategic intelligence products (TRA and OIA) and national project plans. Operational perspectives ensure that national project plans are operationally feasible and enforceable, and that project scopes are adequate and have all the required elements in place. For example, the national project implementation working groups (which include operations representatives) are now meeting weekly as project plans are developed and monthly during implementation to address issues during delivery.

Internal reporting also suggests the establishment of IEP co-chairs from WED and EED is seen as a positive engagement structure. In 2024-2025, NPPD implemented an early Regional Director meeting in advance of the fall annual planning meetings, there was greater presentation of regional projects, and all Regional Directors were present in the WED planning meeting. These efforts are expected to ensure operational input is considered in the projects’ planning and delivery, which will support the efficiency of the program.

While some improvements are noted, findings suggest that collaboration within Intelligence teams can be enhanced and better defined. Effective coordination between strategic and regional intelligence is crucial for the inspection of project entities and targets. This includes timely hand-offs, thorough validation and vetting of targets, and collaboration with officers to support inspections.

2.3.4. Collaboration with other ECCC branches

There are opportunities for EB to improve communications with other ECCC branches, mostly in terms of timely engagement and incorporating their expertise. EB partners felt engagement on products such as TRAs, OIAs and national projects could be improved. It is also noted that EB could have better communicated the results of enforcement projects to partner branches, where appropriate. Agreements among branches set out the ways of working and collaborating. At the time of the evaluation, the IEP planning process has not been integrated in all agreements. Moreover, there was a lack of clarity and communication on how new regulations and other prioritization processes are considered by EB in IEP planning.

Collaboration between Environmental Protection Branch expert support and EB operations was important. Representatives on working groups were essential for EED projects, and there was also positive collaboration with WED and the Canadian Wildlife Service at the regional level.

2.3.5. External partnerships

Collaboration with other enforcement bodies is important to support optimal enforcement service delivery. Relationships with provincial and territorial bodies are critical in the enforcement of federal laws and regulations. Collaboration takes the form of joint operations; information, intelligence, and data sharing; and referrals of suspected non-compliance for future actions when appropriate. EB attendance at in-person meetings or on task forces has been impacted by recent changes in resourcing due to fiscal constraints.

Documents and interviews indicated an opportunity for EB and regional offices to improve partnerships. For example, reciprocity in data and information exchange is essential for maintaining good relationships with other law enforcement bodies. It is also important for identifying high risks of non-compliance in the TRAs. Recent efforts to engage with provinces and territories at the executive level through an annual forum have yet to materialize updated or new agreements. Maintaining agreements with partners and ensuring policy compliance for dealing with personal data are important to avoid risks of breach; this could also have a potential impact on the success of enforcement projects. This includes the completion of privacy impact assessments, currently underway, for EB to be able to assess the impact of the program on personal information. It also requires consideration of a related application for Investigative Body status under the Privacy Act to handle personal information (applied in 2022 and outcome still pending).

2.3.6. Inspections for national projects

Inspections are a core aspect of enforcement operations activities. While the conduct of inspections remained essentially the same, EMI introduced inspections to address risks identified in TRAs and to address compliance rate gaps and risks in certain areas. The national project plans approved in the development of the IEP identify the scope, approach, procedures, and relevant guidance for high-risk inspections to be conducted in the spring.  

Inspections were consistently undertaken as per national project plans, with some exceptions. The outbreak of the COVID-19 pandemic coincided with the launch of EMI and the conduct of nationally planned inspections. The pandemic impacted field operations and laboratory support. This led to delays in national projects and caused many projects to be implemented simultaneously for two years, as new projects were proposed while delayed ones were still ongoing.  

EB should continue to enhance the enabling factors for the conduct of inspections under national projects. Half of the survey respondents disagreed that the appropriate tools, such as guidance, policies and technology, were in place and operational to support delivery. We observed cases where clearer planning or guidance by way of scope, comprehensiveness, and budgeting could be made in the national project plans. It was also suggested that exploring piloting of inspections against national project plans could be beneficial. 

The evaluation identified opportunities to improve overall guidance, including supporting policies and technology for inspections. This could involve updating EB's Operational Manual and related policies, as well as suggesting an update of the existing Comprehensive Outline for Regulatory Enforcement document for EED to guide officers in consistent enforcement activities.

2.3.7. Inspection data collection and entry into GAVIA

Ensuring timely, high-quality, and complete inspection data entry into the GAVIA database posed challenges during modernization implementation and remains an ongoing concern. Late or incomplete data in GAVIA impacts the ability of a region and/or the branch to plan, monitor, course correct, and report on enforcement projects. While data entry guidelines and expected timelines were established, application by enforcement operations is not always compliant. Despite implementing a WED headquarters point of contact and a data entry working group in October 2024 to assist with systematic data entry, efforts to enhance the quality assurance and control over the timeliness and quality of the data entered in GAVIA remain important. Over 2024-2025, the rollout of GAVIA 2.0 and the mobile office is anticipated to enhance information management practices. However, ensuring staff support, providing training, and updating documentation will be important during this transition.  

In some national project inspections, EOs may be tasked with gathering data through questionnaires and site intelligence reports, which include identifying reasons for non-compliance. Our review of internal documents showed recommendations were made to use checklists and site intelligence reports to enhance information and intelligence gathering, as well as to improve data quality and consistency for both WED and EED. Since the data is critical to the refinement of risk rankings, inconsistent information management undermines the logic and the application of the risk-based methodology.  

EOs remain concerned about their collection authorities and potential ramifications of their involvement in this data collection in the event of legal procedures. As a result, the information was not always collected by some EOs conducting inspections, who also believed that there was a lack of clarity on if and how the data collected was being used. The 2024 report recommends a decision on this role. In response, NPPD envisions developing training on collection authorities. The training was expected for delivery in 2024-2025 but was not available at the time of writing this report.

2.3.8. Skills, training, and knowledge to prioritize and plan inspections

Through the annual planning cycle, ODBS, the team responsible for training within EB, considers and aligns training needs with upcoming plans. In some cases, to support the national projects, training on EED regulations may be developed and implemented within a matter of months to EOs across Canada. While collaboration to identify training priorities with ODBS has worked well, they should continue to ensure availability and timeliness of training for national projects, when needed, and other advanced training. For example, WED interviewees and survey respondents mentioned that capacity, support, training, and guidance could increase for special investigation techniques and policies to identifying high-risk violators and potential criminal networks.

As generalists in inspections and investigations, EED has a high number of regulations with different regional contexts that can also be complex to monitor, causing challenges from a knowledge and application perspective. Regional Directors are currently leading an EED national study with respect to inspections related to regulations. During our site visits, officers mentioned that specific lead EOs are a resource to consult prior to an inspection. Identifying ongoing opportunities to engage lead EOs will benefit project planning and implementation.

2.3.9. Lab and expert support

EOs rely on administrative support, expert support, and laboratory support in the conduct of inspections and enforcement operations. Approximately half of the survey respondents disagreed that the appropriate mechanisms were in place to support project delivery.

The main issue identified related to laboratory support. EOs often collect and send samples for laboratory analysis to confirm suspected non-compliance and take the appropriate action based on the results. Interviewees and survey respondents agreed that STB laboratory capacity was, at times, insufficient, which impacted the ability for timely enforcement responses and meeting enforcement objectives effectively. Coordinating with STB to confirm laboratory capacity has reportedly improved through the IEP cycle.

2.3.10. Closure reporting

National project closure reports are designed to analyze and share inspections results and lessons learned, including potential refinements to risk rankings. Closure reports have often been delayed and are recommended by the 2024 report to be completed within 60 days of project completion. To date, 7 of 27 EED national projects have completed closure reports, and 1 of 7 was produced on WED national projects. Closure report completion may be impacted by internal capacity and other factors. Lack of completion in a timely manner impacts knowledge generation and sharing across the network. 

Closure reports are presented to IEP planning teams and to the Enforcement Branch Director’s Board. Due to the need to improve understanding and transparency of results, project closure reports should also be shared across EB to provide lessons learned to operational and intelligence staff. There is also value in sharing findings or lessons learned with relevant internal and external partners as appropriate. Finally, the feedback phase of the IEP cycle has not been completed to the full extent, and as a result, the refinement of TRAs has been delayed.

2.4. Change management

Key findings: Given the significant change it represents, the Enforcement Modernization Initiative required senior management commitment and investment to manage change. While progress has been observed, the Enforcement Branch still has opportunities to enhance implementation by ensuring optimization of the understanding of and adherence to the goals, governance structures, roles and responsibilities, and processes. 

Several interviewees agreed on the need for, and use of, a risk-based approach to prioritize enforcement activities. EMI implementation marked a shift towards a risk-based and centralized annual planning approach. As such, it was expected that comprehensive consultations with all key stakeholders (internal and external), awareness activities, and issues management processes would be in place to support EMI implementation. To this end, EB developed a consultation strategy from the outset that included meeting with various groups. Expert and working groups comprised operational staff, subject matter experts and external partners to inform the development of the TRAs. EB also socialized the new approach within the IEP planning team and the Enforcement Branch Director’s Board, and shared information in several digital formats and channels to create awareness across EB since the inception. NPPD has since also created and modified process documents, templates, guidance and IEP Terms of Reference to enhance understanding and mitigate uncertainties. 

The change management challenge may have been underestimated as there have been issues securing employee buy-in across the branch. Some of the issues raised during the evaluation include ineffective communications, perceived lack of meaningful engagement, and lack of pertinent feedback mechanisms; these contributed to the challenge of gaining buy-in. 

This situation can be partially explained by the fact that the onset of the COVID-19 pandemic coincided with the EMI roll-out. The shift to the almost fully remote working model impacted the amount of change management tools that could be employed, which had an impact on change management efforts and paused certain project implementation elements. That said, the evaluation found that the program continued by taking action to facilitate change over the evaluation period. For example, NPPD engaged three external assessments between 2021 and 2024 to seek advice on potential improvements to the process and has conducted surveys with EED and WED planning teams for the past two years to understand their views and preferences. Although interviewees noted some increased support, the evidence indicates that there are still areas where understanding and agreement could be improved. 

According to the evaluation survey, most respondents (74%) indicated they understand the overall purpose of the initiative, while in other areas, the responses indicated mixed views around their understanding of the decision-making process and structures, and where their role fits in the new risk-based planning process. 

Figure 4: Survey responses to Understanding and Awareness of the risk-based Integrated Enforcement Plan process (n=116) 

2.5. Performance measurement framework

Key findings: Quality and timely data entry and reliability of performance information have been ongoing areas identified for improvement, especially within the Wildlife Enforcement Directorate. Although several steps have been put in place to address these issues, opportunities remain for the Wildlife Enforcement Directorate to clearly define and update measures of success against initiative results in its performance measurement strategy.

The performance measurement strategy for the initiative is outlined in several documents. The EED has adapted their performance measurement strategy towards the risk-based approach, while WED still has room to improve in this area. 

Performance information against expected results for EED measures exists, except for one measure that does not yet have a target in place. Another target had been modified to be significantly lower than the original value. This was done to reflect operational challenges at the time, which were impacting the proportion of inspections conducted towards high risks. Finally, there are also internal measures being documented to track performance and to support decision-making, such as project-level detection rates. 

WED needs to align, update, and define measures against expected results and targets. Performance information was limited because some results had not been updated to reflect the risk-based approach or required further adjustments to better report on risk-based activities. 

Additionally, no long-term results have been defined for WED. Like EED, WED also tracks additional performance information to inform internal decision-making, such as national project result detection rates. Bulk inspections were observed to impact project-level detection rates, and while a methodology to better calculate rates for this inspection type has been implemented, internal documents indicate that consistent methodology for data input between regions remains an opportunity for improvement. Furthermore, due to the change in the way time and resources are allocated to inspections in the IEP, more efforts are needed to fully calculate the extent to which time is devoted to risk-based activities. 

Interviews suggest a need to improve the understanding of the objectives and results of the initiative, including the performance measures in place. For example, a more comprehensive analysis of the severity of the violations and enforcement measures stemming from the national projects could provide more nuance to the non-compliance detected. This analysis occurs through the closure reports. The lack of knowledge of results suggests there could be more communication and sharing of closure reports with stakeholders such as EB internal and external partners, as appropriate. In May 2024, NPPD committed to ensuring data integrity by amending performance metrics to create a robust reporting process, which was not available at the time of the evaluation. 

While the initiative has improved reporting and tools to collect performance information, there were challenges with data reliability which could impact the ability of different teams to generate reliable performance information. However, interviewees and survey respondents indicated that the performance information currently collected is still sufficient for fulfilling responsibilities and guiding decision-making.

EB collects, analyzes, and reports performance information regularly. This includes:

• EB Annual Statistics Report
• Real-time monitoring of enforcement results through four Power BI Dashboards for WED and EED
• GAVIA Dashboards tailored to different users
• IEP quarterly reports for monitoring progress, among others

IEP quarterly reports track progress against planned inspections and highlight regional challenges. Midterm reporting on the initiative was positive, sharing results and tracking progress. However, progress against 62 identified high-risk areas, sectors, and species could be tracked and reported to provide a more global picture.

Finally, EB has created four Power BI Dashboards for EED and WED to monitor enforcement data in near real-time and in conformance with Protected B requirements. A recent EB survey suggests that additional Power BI training could improve skills and knowledge among IEP planning team members.

2.6. Effectiveness – achievement of expected results

Key findings: The evaluation found progress has been made towards the achievement of expected results and key outputs. A key achievement has been the development of real-time targeting of non-compliance in commercial trade. 

2.6.1. Expected result: high-risk forms of non-compliance are identified in support of ECCC's pan-Canadian mandate

Completion of risk assessments related to ECCC’s mandate

The modernization initiative included the conduct of strategic TRAs for all laws and regulations within ECCC’s mandate. The EB is on track to reach their target of having all ECCC's laws’, regulations’ and instruments’ risks classified by 2025-2026. To date, EB has identified 62 high-risk areas, sectors, and species through these data-driven intelligence products. There are still areas to be assessed and risks to be addressed. 

Figure 5: Threat risk assessments enforcement areas

In addition to completing TRAs for ECCC’s mandate areas, the initiative was designed for the risk assessments to be refined on a three-to-five-year cycle. The EB is currently behind in meeting this requirement due to various factors detailed further in this report. The lack of timeliness of TRA refinement could result in EB not making decisions based on up-to-date information. While EED has updated plans (despite significant delays) for revising the TRAs they are responsible for, WED’s strategy for updating TRAs remains unclear.

Taking action on high-risk areas 

Through implementing the IEP, EED reported conducting 27 national TRAs based on data-gathering inspection projects completed during the evaluation period, including 758 risk-based inspections. WED completed 7 national projects and 2,224 high-risk inspections. As discussed in section 2.3 on Delivery, while national project inspections have largely been implemented, several projects experienced delays in implementation due to the COVID-19 pandemic and other factors. 

Overall, results suggest some degree of improvement to identifying and addressing high-risk non-compliance. The percentage of EED risk-based inspections uncovering non-compliance on regulations has met its target of above 13 percent in each of the four years in the evaluation scope. 

From 2022 to 2024, there has been an increase from 26 to 38 percent of the WED’s planned events that uncovered non-compliance.1 EB modified the non-compliance target as the former target was unrealistic (60%). A new target is to be established based on a three-year average. Given that there are only two years of data available, the target was not defined at the time of the evaluation. Therefore, only limited conclusions could be drawn.

We also found evidence of new technology and techniques in the intelligence cycle, as well as application of risk tools in other EED inspections areas to improve targeting of high-risk non-compliance. A key achievement has been the creation of the CTD and CTE. These tools have been transformative by turning high volumes of previously under-utilized data received on imports and exports into actionable intelligence. Reporting indicates the tools helped identify non-compliant imports and exports, as well as some predictors of non-compliance, such as certain information provided in a declaration upon import and/or export of a regulated good. Also, the tools have potential for identifying ongoing and emerging risks in one high-risk area due to growing knowledge of the system from multiple project iterations. In the future, EB intends to use these findings to improve predictive models for better targeting non-compliance and to establish potential offender scenarios. 

It is worth noting that EB has not undertaken assessments to analyze and mitigate potential risks of the CTD and CTE, such as a security assessment, Algorithmic Impact Assessment, or Privacy Impact Assessment. However, at the time of writing this report, EB reported that the work is underway.

Establishing compliance rates

EMI aimed to establish compliance rates in EED due to gaps in detections rates for all ECCC laws and regulations. Detection rates and information gathering projects were therefore designed to gain more information and determine the level of risk and/or compliance in lesser-known areas. As of 2023-2024, five of six high-risk areas had a compliance rate established, on track to meet the target identified by 2025-2026. 

The initiative aimed for more robust detection levels through inspections designed to substantiate compliance of a sample of regulatees. However, the randomization or the use of large inspection samples worked as a hindrance to implementation process. As a result, the level of non-compliance detected might be over-estimated. 

Furthermore, while projects intended to document the underlying reasons for non-compliance, closure reports suggest that this information was not always consistently recorded. The non-completion of inspections according to the quality and scope of plans, including the non-inclusion of site intelligence reports, prevents the confirmation of the risk ranking and impairs future planning. Concretely, this contributes to delays in refining the TRAs and in obtaining robust intelligence. 

Views on the achievement of this expected result

While progress was made in identifying high-risk potential non-compliance with laws and regulations, survey respondents had mixed views on the results of the methodology used. Survey results indicated that 37 percent of respondents indicated their efforts helped identify and address high-risk non-compliance. On the one hand, interviews highlighted a lack of results from some national projects in identifying high-risk non-compliance or significant violations leading to enforcement measures; on the other hand, the CTD and CTE were seen as valuable. Better communication of progress against indicators and project results may be needed to enhance common understanding of results.

2.6.2. Expected result: transition in inspection planning and resource allocation to target the highest risks 

Interviews confirmed that the transition in inspection planning and resource allocation to target the highest risks has begun. EED has seen an increase in the percentage of planned inspections conducted towards regulations deemed to be high risk over the evaluation period. The initial target for this performance indicator was 40 percent of planned inspections to be conducted by 2025. The target was modified in the performance information profile to be above 10 percent annually and to establish a baseline by 2025-2026. EED has met the modified target of above 10 percent in three of the four years. 

WED does not have a performance indicator for this expected result. Recent analysis illustrates that WED has increased the number of planned inspections conducted compared to responses to incidents since 2019-2020, suggesting a move away from an historic trend of 50-70 percent reactive work within the Directorate. 

EB reporting suggests WED’s change in planned inspections is reflective of the implementation of the IEP and the risk-based approach. Since 2020-2021, the percentage of WED planned inspections was over 64 percent of inspections each year, with a peak of 9,198 planned inspections (91%) conducted in 2023-2024. The number of WED planned inspections conducted has consistently increased by 180 percent per year on average. While there is an observed increase in planned inspections, reporting also indicates that some low-risk areas and inspection types (bulk inspections) continue to receive focus. Bulk inspections have continued to increase since 2018-2019. As previously noted, WED should sustain efforts to improve the tracking of activities according to risk. 

In this transition, staff from the Environmental Protection Branch expressed concerns that the new type of inspection is affecting EB's ability to handle referrals related to potential non-compliance. Although EB is supposed to respond to these referrals as part of their annual reactive inspections, Environmental Protection Branch partners are concerned that the new risk-based inspections are taking resources away from this type of activity. This could potentially lead to missed cases of non-compliance.

2.6.3. Long-term expected result: non-compliance risk is reduced

While performance measures were intended to be outlined in a second phase of EMI, internal documents indicate there are currently two measures to determine whether non-compliance risk is reduced:

• Pre/post analysis of existing enforcement records, and
• A new cost-benefit analysis. 

While EED established the measurement strategy, the indicators to measure this result still need to be defined for WED. 

EB staff review case management records to determine where follow-up inspections have identified reduced levels of non-compliance following enforcement action for EED. Over the evaluation period, EB has met the target of above 60 percent of verified compliance of unresolved violations during follow-up inspections. 

The risk of non-compliance being one of the main drivers of EB operations, EB is also concerned about the efficiency of its operations. EB is innovating by developing a cost-benefit analysis model to analyze the outcomes of enforcement action compared to the costs to aid in decision-making for priorities. At the time of the evaluation, a preliminary conceptual framework has been proposed, and documents suggest there were still several steps to be completed before the model is considered viable. 

The Enterprise DataHub

The Enterprise DataHub and Power BI initiative align with ECCC’s Digital and Data Analytics Strategies and with the Treasury Board Policy on Service and Digital that aim to improve data management with the goal of treating data as a strategic asset for improved performance, accountability, and/or innovation. 

Before modernization, enforcement data was not fully used to support operations and decision-making. To improve this, EB undertook an Enterprise DataHub project to improve data governance and management and to feed future risk analysis for targeting non-compliance. This project was supposed to deliver a cloud-based data platform by March 2024. At the time of evaluation, most outputs were delivered, and only one of its four core objectives was fully achieved.

Table 1 below outlines the core DataHub objectives and progress made at the time of the evaluation. 

The DataHub project experienced challenges. Notably, challenges included the lack of a clearly defined scope and deliverables, financial information tracking, and delays incurred in data cleansing. Some actions to course correct were taken, and since the project was not completed within original timelines, the end date was revised to March 2025. The delay in project implementation did not lead to critical risks or issues with respect to information management.

Once completed, the DataHub will enable EB to leverage applications such as geospatial tools and processing capabilities. It is anticipated that EB will have to update its data strategy to optimize the use of modern applications.

2.6.4. The computer forensics laboratory

EMI funded the construction of the computer forensics laboratory in Montreal, Quebec that opened in early 2022. The services to be provided by the laboratory involve acquiring, retrieving, analyzing and conserving electronic evidence to support investigations and potential prosecutions. Laboratory staff assist by searching electronic devices and securing data by disabling remote access, for example. They provide a review package to the investigator, a final report for prosecution, and answer questions as needed. Interviewees expressed appreciation for the excellent technological capacity and equipment and highly specialized training (over 640 hours) available to designated officers working at the lab to become certified as computer forensics examiners. EB also initiated a review of their policy/procedure framework to ensure the privacy protection and compliance of the laboratory. A Privacy Impact Assessment is anticipated to be scheduled after completion of the two other EB Privacy Impact Assessments underway. 

Currently, the laboratory is not delivering its full suite of services due to challenges with notably staffing and retention. There was one active staff member in March 2024. Interviewees identified factors that contributed to these challenges. For example, the inability to work from home during the pandemic resulted in a higher staff departure rate, and the competitive salaries offered in the enforcement sector also partially explain the situation. While partnerships with other federal enforcement bodies and adjustments to the classification of these positions were explored, EB is still facing major staffing shortages at the laboratory. 

As a result of the current laboratory situation, EB faces higher risks and costs. The branch has had to use third party digital forensics services, which are expensive, slow, and dependent on availability. These services are crucial for handling digital evidence in prosecutions. The demand for these services is expected to grow, potentially increasing the risk further. 

It was reported to us that EB has initiated discussions with the Human Resources Branch to identify solutions to optimize the laboratory’s activities. However, concrete actions remain to be implemented and all options to meet EB's needs for this type of service should be considered.

2.6.5. Improved federal, provincial, territorial and Indigenous cooperation and alignment

Protection of Canada’s environment is a shared responsibility between federal, provincial, and territorial governments and Indigenous counterparts. The importance of re-affirming and re-establishing existing partnerships after the impact of the pandemic on operations was a key concern raised during EB’s five-year strategic plan consultations and other reviews. Therefore, EB aimed to implement an executive annual forum to better cooperate, share information, and potentially reduce duplication with other levels of government. This forum complements existing operational cooperation in regions and at the officer-level. While some cooperation and information sharing with provinces and territories was done through several annual meetings between 2019 and 2022, the forum is currently paused. Key informants noted that although it was a positive forum for exchanging information and priorities, it has yet to yield firm commitments or updated agreements on clarification of mandates, cost sharing, and information sharing to date. 

At present, it is important to note that joint approaches to enforcement with Indigenous communities are also being explored and developed. EB’s 2019 Indigenous Strategic Policy Framework and 2021 five-year Indigenous Strategic Plan highlighted the need to increase efforts in engaging, cooperating, and fulfilling obligations, and to advance reconciliation. EB also recognizes the importance of improving guidance and tools for enforcement operations that were developed over the evaluation period. 

An EB Indigenous Engagement Portal designed to centralize guidance, training, and communications tools for EOs has been created. It aims to improve engagement with Indigenous communities as well as to make available a new suite of Indigenous engagement products and outreach materials, including an engagement repository. Branch-level policies for modern treaty negotiations, including a strategy and assessment criteria for negotiations, were developed, and an analysis of options for enforcement arrangements and restorative justice was completed.

Key developments in joint approaches have been a pilot project consisting of: a joint enforcement agreement, options for enhancing Indigenous Guardians programs, collaboration on compliance promotion, and enforcement protocols and training.

3. Conclusions

The Enforcement Branch (EB) is implementing the modernization initiative, moving towards a risk-based approach. We found that the Enforcement Modernization Initiative (EMI) aligns with departmental and federal government priorities. The lack of availability of the financial information limits the ability to conclude on efficiency, though survey results suggest there may be an opportunity to improve in this area. 

Regarding design, there was agreement on the need for a risk-based approach and on meeting evidence-based and risk-based principles in the process. However, disagreement on operationally advisable, adaptable and transparent principles was noted. Survey respondents disagreed that the design aligned to the expected results. The disagreement was because of the limitations of the data-driven approach in identifying high risks and targets as well as other factors. The integrated planning process identifies the annual priorities in a timely manner, although the process still has opportunities for refinements. 

Change management and implementation represented challenges in the initiative. Despite this, progress towards collaboration and engagement with the Integrated Enforcement Plan (IEP) planning teams and partners over the course of the initiative has been observed, and EB should remain committed to this area going forward. We also found it would be beneficial to streamline the Intelligence function, as noted in the recommendations below. 

We noted that the EB went through improvements on multiple fronts (including training, guidelines, data entry and controls), but reliability of performance information remains an area for continuous improvement. As several steps have been put in place, continued collaboration will enhance performance reporting issues. The Wildlife Enforcement Directorate (WED) needs to clarify and define indicators for all expected results and align the performance measurement strategy to the risk-based approach. 

Progress was observed towards the achievement of expected results and key outputs. In particular, the EMI has added value to non-compliance targeting associated with commercial trade. Still, the cycle has not been refined on schedule resulting in stagnant intelligence. 

Lastly, despite EB’s efforts to optimize the use of the computer forensics lab, a formal assessment is needed to determine its medium and long-term viability considering the current challenges that have led to its underutilization.

4. Recommendations and Management Responses and Action Plan

Recommendation 1: The Chief Enforcement Officer should streamline the Intelligence function in both the Environmental Enforcement Directorate (EED) and the Wildlife Enforcement Directorate (WED) and clarify roles and responsibilities to ensure its efficiency and effectiveness. 

Management Response:

The Chief Enforcement Officer agrees with this recommendation and that there would be benefits in better articulating how Intelligence functions and could better support the Branch in delivering on its risk-based enforcement priorities. 

To that end, the Enforcement Branch has already initiated a review of its Intelligence functions to clarify roles and ensure a better flow of intelligence functions throughout the Branch to enhance enforcement operations and improve alignment with priorities. 

Action 1: Review and implement a revised structure and governance of Intelligence functions

Recommendation 2: The Chief Enforcement Officer should assess the viability and sustainability of the computer forensics laboratory to provide cost-effective digital forensics services to Enforcement Branch in a timely manner.

Management Response:

The Chief Enforcement Officer agrees with the recommendation and will initiate a review of the computer forensics laboratory to determine cost-effective options to obtain digital forensics’ services, including conducting a review to address personnel recruitment and retention challenges.

Action 1: Review current operations of the computer forensics laboratory 

Recommendation 3: The Chief Enforcement Officer should review and update the Enforcement Modernization Initiative’s performance measurement and data collection strategy to better demonstrate value, costs and results. 

Management Response:

The Chief Enforcement Officer agrees with the recommendation.

The Enforcement Branch will review its existing performance measurement indicators and develop or update (when required) the performance indicators to reflect the goals and outcome of the Initiative.

This work will be done with the help of the Corporate Services and Finance Branch, Corporate Management Directorate.

Action 1: Review, seek approval and update performance management indicators