Evaluation of the Engagement and Intervention Model
Signatures
Anne Kelly
Commissioner
Date
Contributions
Authors:Footnote1
Prince Baffoe, Junior Evaluation Analyst
Rebecca Cherner, Evaluation Officer
Sidikat Fashola, Evaluation Officer
Sara Johnson, Senior Evaluation Manager
Karen Koundakjian, A/Senior Evaluation Manager
Natalie Mercer, Evaluation Analyst
Kayla Wanamaker, A/Evaluation Officer
Contributors
Danielle Guarino, Junior Evaluation Analyst
Mylène Mailhot, Student Analyst
Acknowledgement
The evaluation team would like to thank the Correctional Operations and Programs Sector for their assistance throughout this evaluation. The evaluation team is also grateful for the continued collaboration and support from the Security Branch. We would like to thank all members of the Consultative Working Group, including representatives from Correctional Service Canada’s Learning and Development Branch, Health Services Sector, Research Branch, Preventative Security and Intelligence Branch, the Women Offenders Sector, Internal Audit, Indigenous Initiatives Directorate, Incident Investigation Branch, Offender Redress, Strategic Policy, and Performance Measurement and Management Reports for their contributions throughout the evaluation.
The evaluation team would like to express appreciation to those who participated in the collection of data and contributed valuable information, including institutional staff members, staff from the Correctional Operations and Programs Sector, staff from the Health Services Sector, staff from the Human Resource Management Sector, staff from the Women Offender Sector, staff from the Senior Deputy Commissioner Sector, and staff from the Policy Sector.
We would also like to thank the several branches and sectors of CSC who assisted with providing data for this evaluation, including staff from Learning and Development, Performance Measurement and Management Reports, and Human Resources Corporate Reporting.
Finally, the evaluation team would like to thank everyone else who contributed to this evaluation, whose names do not appear here.
Evaluation Report
File #
Evaluation of Correctional Service Canada's Engagement and Intervention Model
June 2021
Table of Contents
- Acknowledgement
- Table of contents
- List of tables
- Acronyms/abbreviations
- Executive summary
- Summary of recommendations
- Introduction
- Findings: Relevance
- Continued need for the model
- Alignment with governmental priorities
- Consistency with departmental roles and responsibilities
- Findings: Design and delivery
- Training and identified best practices
- Introduction to the Engagement and Intervention Model
- Safety for All – All Staff Briefings
- Training for managing incidents with sub-populations
- Use of physical interventions
- Overall use of force
- Use of force for assault and behaviour-related incidents
- Type of force used
- Inmate injury
- Planned and spontaneous uses of force
- Institutional incidents involving physical or mental health distress
- Findings: Effectiveness
- Implementation of the EIM when responding to incidents
- Engagement in the EIM philosophy
- Implementation
- Institutional culture
- Managing incidents in the Structured Intervention Units
- Implementation of key activities
- Use of interdisciplinary teams
- Use of quality improvement activities
- Conclusions and recommendations
- Appendix A: Program description
- Appendix B: Engagement and Intervention logic model
- Appendix C: Design and delivery detailed tables
- Appendix D: Evaluation questions and matrix
- Appendix E: Evaluation strategy/methodology
- Appendix F: Staff survey and response frequencies
- Appendix G: Key informant interview protocol
- Endnotes
List of Tables
- Table 1. Limitations, impact, and mitigation strategies
- Table 2. Total inmate population demographics
- Table 3. Summary of changes in percentage of overall use of force (UoF) for the EIM, compared to the SMM
- Table 4. Summary of changes in use of force by incident type for the EIM, compared to the SMM
- Table 5. Summary of changes in use of force by force type for the EIM, compared to the SMM
- Table 6. Summary of changes in use of force resulting in inmate injury
- Table 7. Summary of changes in planned and spontaneous uses of force for the EIM, compared to the SMM
- Table 8. Summary of use of force data (UoF) for mental health related outcomes
- Table 9. Summary of incident data for physical health related outcomes
- Table 10. Summary of appropriate response protocols for use of force packages
- Table 11. Engagement in the EIM philosophy
- Table 12. Experience with the implementation of the EIM
- Table 13. Staff perceptions on institutional culture
- Table 14. Staff/contractors often or always involved in the application of intervention strategies
- Table 15. Usefulness of AIM for assessing risk
- Table 16. Engagement and intervention response options to incidents that are often or always carried out
- Table 17. Summary of changes in follow-up procedures after the deployment of a chemical or fnflammatory agent for the EIM, compared to the SMM
- Table 18. Summary of changes in offering of health/physical assessments post use of force for the EIM, compared to the SMM
- Table 19. Summary of changes in documentation post use of force for the EIM, compared to the SMM
- Table 20. Summary of changes in the percentage of use of force package reviews not met within the timeframe for the EIM, compared to the SMM
- Table 21. Summary of changes in violation of law or policy flags for the EIM, compared to the SMM
- Table C 1. Training compliance rates for the Introduction to the EIM – Online, by region, as of September 30, 2019
- Table C 2. Training compliance rates for the EIM Scenario Based Training, by region, as of September 30, 2019
- Table C 3. Percent of incidents involving a use of force for the SMM and EIM periods
- Table C 4. Percent of use of force packages in which use of force was deemed necessary or proportionate (EIM period April 1st, 2018 to September 30, 2019)
- Table C 5. Percent of use of force packages in which use of force was deemed necessary and proportionate (SMM period April 1st, 2016 to December 31, 2017)
- Table C 6. Percent and rate of use of force incidents involving an inmate with a GBA+ consideration
- Table C 7. Percent of all incidents involving use of force by offender security level for the SMM and EIM periods
- Table C 8. Percent of use of force for assault and behaviour-related Incidents for the SMM and EIM periods
- Table C 9. Percent of all use of force incidents with the following uses of force for the SMM and EIM periods
- Table C 10. Offender injury in use of force incidents for the SMM and EIM periods
- Table C 11. Percent and rate of use of force incidents resulting in an injury among diverse sub-populations of inmates
- Table C 12. Percent of planned and spontaneous use of force packages for the SMM and EIM periods
- Table C 13. (Type of) Force occurring during self-injurious behaviour related incidents for the SMM and EIM periods
- Table C 14. Summary of use of force review package data for physical and mental health related indicators for the SMM and EIM periods
- Table C 15. Percent of total incidents during which first aid was required for the SMM and EIM periods
- Table C 16. Summary of appropriate response protocols during planned or spontaneous uses of force
- Table C 17. Percent of use of force packages where a chemical or inflammatory agent is deployed with follow-up procedures for the SMM and EIM periods
- Table C 18. Percent of use of force packages in which at least one staff or inmate is offered health/physical assessments post use of force for the SMM and EIM periods
- Table C 19. Percent of use of force packages in which there was incomplete documentation for the SMM and EIM periods
- Table C 20. Percent of use of force package reviews not met within the timeframe
- Table C 21. Percent of use of force packages in which there was violation of law flag at the institutional review level for the SMM and EIM periods
- Table C 22. Percent of disciplinary measures actioned against staff for excessive use of force
List of Acronyms
- AID
- Assessment/Intervention/Debrief
- AIM
- Ability, Intent, Means
- CCRA
- Corrections and Conditional Release Act
- CCRR
- Corrections and Conditional Release Regulations
- CD
- Commissioner’s Directive
- CISM
- Critical Incident Stress Management
- CSC
- Correctional Service Canada
- CXCD
- Correctional Officer Continuous Development
- DRF
- Departmental Results Framework
- EIM
- Engagement and Intervention Model
- GBA+
- Gender Based Analysis Plus
- HRMS
- Human Resources Management System
- ICCPR
- International Covenant on Civil and Political Rights
- IIB
- Incident Investigations Branch
- LGTBQ2+
- Lesbian, Gay, Bisexual, Transgender, Queer, and Two-Spirited
- MHNS
- Mental Health Needs Scale
- NHQ
- National Headquarters
- NTS
- National Training Standard
- OC
- Oleoresin Capsicum
- OCI
- Office of the Correctional Investigator
- OMS
- Offender Management System
- PMMR
- Performance Measurement and Management Reports
- RTC
- Regional Treatment Centre
- SIU
- Structured Intervention Unit
- SMEAC
- Situation, Mission, Execution, Administration, and Communications
- SMM
- Situational Management Model
- STG
- Security Threat Group
- UoF
- Use of Force
- WOC
- Warrant of Committal
Executive Summary
This report presents the evaluation of the Engagement and Intervention Model (EIM), which is a risk-based model designed to assist Correctional Service Canada (CSC) staff working in both security and health activities in responding to, and resolving institutional incidents, using the most reasonable interventions. To have and maintain safe operations of institutional environments, the EIM was introduced in January 2018, and replaces the previous institutional incident management model, the Situational Management Model (SMM), in its entirety.
The EIM differs from the SMM in four primary respects. First, it emphasizes a balanced approach to risk assessment using a person-centered perspective and intervention strategies to prioritize an offender’s well-being. Second, engagement and intervention strategies include those focused on mental and physical health, in addition to those that are security focused. Third, the use of non-security partners, such as Health Professionals, is urged. Fourth, there is a focus on reducing the risk of physical harm through non-use of force responses.
Key sources of information used to examine the relevancy, design and delivery, and effectiveness of the EIM included internal and external documents and literature, data from the Human Resources Management System (HRMS), interviews with 15 key informants, and online survey data from 237 staff members. Data from the Offender Management System (OMS) were also used to compare institutional incident and use of force indicators between incidents managed under the SMM, the previous incident management model, and the EIM.
Relevance
There is evidence of a continued need for the EIM to prevent, respond to, and resolve situations within federal institutions that could potentially disrupt the safety and security of inmates or staff. This need is further reinforced through the requirement to consider unique characteristics and situational factors of inmates when responding to institutional incidents.
Alignment with governmental priorities
EIM aligns with, and supports, the federal government’s priority of providing a safe and secure environment for Canadians, in general, and inmates, in particular.
Consistency with departmental roles and responsibilities
The EIM priority of guiding both security and health services staff to use the most reasonable intervention strategies is aligned with the roles and responsibilities of the CSC.
Design and delivery
Training and identified best practices
The data (where available) suggest that most CSC staff had received the EIM training prior to its implementation, with the exception of National Headquarters (NHQ) employees. However, key informants articulated a need for refresher training and scenario-based training for non-security staff (e.g., health services). The effectiveness of training could also be enhanced by involving multiple disciplines in completing training together, particularly in scenario-based training. There is also a need to incorporate more content related to women inmates and inmates with cognitive impairments when educating staff about de-escalation strategies. This need may also extend to other sub-populations as many survey respondents did not know if they possessed enough knowledge to de-escalate incidents among various sub-populations of inmates.
Use of physical interventions
It should be noted that the EIM emphasizes that the appropriate intervention strategies will be chosen following the initial and ongoing assessment of the individual(s), the situational factors, and the associated level of risk, and the interventions may or may not include use of force options.
Considering all institutions together, there has not been a decrease in use of force during institutional incidents since the implementation of the EIM. This is also true when comparing use of force incidents within inmate security levels. The percentage of use of force review packages where the amount of force used was deemed necessary and proportionate is high. Overall, while there is evidence of some positive changes, particular attention needs to be paid to the more frequent use of force with younger, ethnocultural, and Indigenous inmates.
Considering all institutions together, there was a decrease in force used during behaviour-related incidents. Examining regional treatment centres (RTCs) specifically, there was also a decrease in force used during behaviour-related incidents. Overall, there is evidence of some positive changes.
Considering all institutions together, there was a decrease in the discharge of inflammatory or chemical agents during use of force incidents. Examining RTCs specifically, there was also a decrease in the discharge of chemical or inflammatory agents. Overall, there is evidence of some positive changes.
Considering all institutions together, findings show a decrease in inmate injury during use of force incidents. Overall, there is evidence of some positive changes. Identified areas for concern in use of force practices among older inmates are mental health, physical disability, and physical health. There is also a need for more guidance and training on how to manage older inmates when force is required.
Considering all institutions together, there has been an increase in planned uses of force and a decrease in spontaneous uses of force under the EIM when compared to the SMM. This is evidence of some positive changes.
Institutional incidents involving physical or mental health distress
Considering all institutions together, during the EIM period, there have been two positive changes regarding incidents involving an inmate with mental health concerns. There has been a decrease in use of force during incidents involving an inmate with a suicide alert and among those occupying a mental health bed. Overall, this is evidence of some positive changes. Examining RTCs specifically, there has been a decrease in use of force during incidents involving an inmate with an active suicide alert as well. Effect sizes indicate this change is small, which is promising.
Overall, there has been a decrease in the percentage of incidents in which first aid was required, including in RTCs. While there is evidence of a positive change, the effect sizes indicate the observed changes are negligible.
Overall, there has been a decrease in the percentage of interventions conducted in accordance with the Guidelines for Health Service Responsibilities, including in RTCs.
Effectiveness
Implementation of the EIM when responding to incidents
Staff are engaging in key components of the EIM philosophy, as most reported that they are taking a person-centered approach and placing mental and physical well-being at the center of engagement and intervention strategies. Most are also able to identify cues of distress or altered levels of consciousness when dealing with inmates, to continuously reassess situational factors as the incident unfolds and to categorize the level of risk, and to employ de-escalation strategies when responding to incidents. Where staff may not fare as well is in their ability to select appropriate force options, and in their ability to get help to safely manage incidents of mental and physical distress when dealing with inmates.
While for the most part key elements of the EIM have been implemented as intended, the model does not appear as intuitive and easy to apply, especially when it is compared to the SMM. It is also perceived that the EIM has not resulted in a decrease in the use of force. There is a lack of clarity of roles and expectations of staff during an incident, a lack of teamwork among staff in responding to incidents, and a lack of readily accessible staff to effectively manage incidents during off-peak hours. Although, for the most part, roles and responsibilities of the Sector Coordinator are being implemented appropriately, survey responses suggested that the implementation of roles and responsibilities could be improved. Moreover, there may be a need to have more clarity in who is in charge during the course of an incident. Additionally, Sector Coordinators are finding it difficult to transition from the first Officer on the Scene to their role as Sector Coordinator.
While the culture of some institutions allows the EIM to be successfully implemented, CSC’s culture at the organizational level may present challenges to the implementation of the EIM. This may be due to a perception that there is a strong focus on security rather than the use of interventions, and a culture that is resistant to change. The EIM has not had a positive influence on the culture of some institutions.
With respect to managing incidents in the SIUs, findings show that the EIM philosophy has not had a positive influence on the outcomes of incidents managed within these units.
Implementation of key activities
While Correctional Officers/Primary Workers and Correctional Managers were most often identified as being involved in the planning and application of intervention strategies, Sector Coordinators were not as involved as would be expected. Sector Coordinators and Health Professionals appear to be more involved in the application of intervention strategies at RTCs than overall in institutions. Staff perceptions suggest there has been an increase in interdisciplinary teamwork since the implementation of the EIM as Correctional Managers, Sector Coordinators, Health Professionals, and individuals who have a good rapport with the inmate were slightly more involved in the application of intervention strategies than they were under the SMM. Despite this, there still remain some obstacles to collaboration, for example, the extent of integration and interaction between correctional and clinical staff, and a lack of access to non-security staff during off-peak hours.
While two-thirds of staff understood the intent of the AIM tool and viewed it as useful for assessing risk, only about half of staff survey respondents found it feasible to use the AIM tool during an active incident.
There appears to be frequent use of non-physical interventions as response options during an active incident, with tactical maneuvering being least commonly used. Moreover, the frequency of use of these response options does not appear to have changed between the SMM and EIM periods.
Use of quality improvement activities
While many of the management roles and responsibilities are being fulfilled under the EIM, some issues were raised, including the lack of dissemination of trends and deficiencies in how the EIM is operating, and a need for more follow-up from management in the instance of violations of law or policy. When model periods are compared, overall, there has been a decrease of required post-incident care (e.g., post-incident decontamination shower), particularly with respect to care where an inflammatory or chemical agent has been deployed.
Overall, with respect to documentation of incidents, including RTCs, there has been a decrease in both the percentage of inmates being given the option to report their version of events and in video recording issues. There has been no significant change in the percentage of forms that have not been completed. It was also noted that there were issues with performance monitoring and reporting, particularly when it came to data quality, data accessibility, and to a balanced approach to reporting on how well the EIM is performing.
Although there is value to conducting debrief sessions, a lack of adequate staff, operational constraints, and time constraints are identified as barriers to conducting them. These barriers may have an impact on the quality of debriefs.
While the majority of respondents believed that current policy provided adequate guidance for conducting use of force reviews, there were some inefficiencies with regard to the policy and process identified, namely that at times the policy is over-prescriptive and that the process itself can be cumbersome. During the EIM period, there has also been an observed increase in the proportion of institutional and regional reviews not being completed on time. For RTCs, there has been an observed increase in reviews not being completed on time at the regional level.
When the EIM period is compared with the SMM period, there has been an increase in use of force packages for which there has been a finding of violations of law or policy. This finding also applies to RTCs, where the effect size of the observed change is medium. When it comes to disciplinary measures for excessive uses of force, there may be reason to believe that there is under-reporting or inaction to correct this behaviour as many key informants spoke to a need for more follow-up subsequent to violations of law or policy.
As a whole, the EIM is contributing to CSC’s mission as, in principle, it emphasizes the use of the most reasonable, safe, secure, and humane approaches to managing inmates during incidents. Moreover, the model addresses five of the six corporate priorities as indicated in the Relevancy Section of this report. However, the areas for improvement described above demonstrate that there is still work to be done in order for the EIM to realize its true potential.
Summary of recommendations
Five recommendations were formulated to action improvements on the performance of EIM, as well as on program expansion.
Recommendation 1-Training
The Evaluation Division recommends that Correctional Service Canada (CSC) reassess the EIM training, including refresher training, to ensure more clearly, well-defined, and effective: (a) scenario-based modules that incorporate the diverse sub-population of inmates (e.g., offenders with mental health needs); and (b) roles and responsibilities of all parties (e.g., the Sector Coordinator, as well as staff including non-security staff) during an incident.
Recommendation 2-Incidents involving mental health and physical distress
The Evaluation Division recommends that CSC devise options to increase capacity to respond to incidents involving mental health and physical distress, particularly those occurring during evenings and weekends.
Recommendation 3-Policy review
The Evaluation Division recommends that CSC review and revise, as necessary, Commissioner’s Directive 567 - Management of Incidents and 567-1 - Use of Force, in consultation with operational staff, to ensure the proposed guidance, including prescribed timelines, are relevant in an operational environment.
Recommendation 4-Corrective actions
The Evaluation Division recommends that CSC review the guidance on corrective actions to ensure it provides more appropriate direction on breaches of law and/or policy.
Recommendation 5- Information collection
The Evaluation Division recommends that CSC develop a national protocol for reporting information found in Statement/Observation Reports in an accessible manner.
Introduction
The Engagement and Intervention Model (EIM) is a risk-based model designed to assist Correctional Service Canada (CSC) staff working in both security and health activities in responding to, and resolving institutional incidents, using the most reasonable interventions. To have and maintain safe operations of institutional environments, the EIM was introduced in January 2018, and replaces the previous institutional incident management model, the Situational Management Model (SMM), in its entirety. The EIM was given authority through the promulgation of Commissioner’s Directive (CD) 567 - Management of Incidents, by which all interventions used to manage incidents are to be consistent with law and policy in the application of the EIM.Endnote i This directive helps to maintain the safe operations of institutional environments, as well as respectful environments that promote dynamic security and interactions between staff and inmates.
The EIM integrates a strong focus on the guiding principles of life preservation, interdisciplinary teamwork, CSC Mission & Values, necessary and proportionate response, and leadership. To implement this model nationally, there was a requirement for new training to be developed and for revisions to be made to the existing training.Endnote ii The EIM differs from the SMM on four primary respects:
- There is an emphasis on a balanced approach to risk assessment, in which assessments are person-centered rather than solely behaviour-centered. Thus, all intervention strategies prioritise an offender’s well-being;
- Engagement and intervention strategies are broadened to not only include those which are security focused, but also those which are focused on mental and physical health;
- The use of non-security partners such as Health Professionals (e.g., Psychiatrists, Psychologists, Nurses, etc.), as well as Chaplains, and Elders in responding to security incidents is highlighted; and
- There is a greater focus on reducing the risk of physical harm through non-use of force responses in responding to security incidents, such as de-escalation and controlled non-intervention approaches.
To ensure that appropriate leadership and health considerations are integrated, a Sector Coordinator role was added to the EIM.Endnote iii The main responsibility of the Sector Coordinator is to ensure intervention options are appropriate, and to continuously reassess their appropriateness, including the monitoring of both the physical and mental health of the inmate.Endnote iv
The EIM therefore builds on the SMM by adding more response options and post-intervention accountability. It also provides greater clarity for these elements, and expands on the use of partners during a response, particularly CSC health services. With the goal of eliminating some of the limitations identified from the SMM, the EIM’s main objectives include, but are not limited to the:
- Increase in the use of non-security partners to assist with verbal intervention and de-escalation options;
- Introduction and implementation of the Sector Coordinator role to ensure on-scene leadership, with specific responsibilities to ensure health considerations are integrated into the intervention;
- Increased focus on the use of non-physical interventions where there is a low-risk of imminent harm to the offender or others; and
- Increase in the use of health-partners to respond to incidents involving mental or physical distress.Endnote v
Context
International standards, federal legislation, and CSC’s internal policies and directives prohibit torture and other cruel, inhumane or degrading treatment of prison inmates. The first principle of the United Nations’ Body of Principles for the Protection of Detained or Imprisoned Persons states that “All persons under any form of detention or imprisonment shall be treated in a humane manner and with respect for the inherent dignity of the human person.”Endnote vi Section 26 of the Criminal Code of Canada also declares that, "Everyone who is authorized by law to use force is criminally responsible for any excess thereof according to the nature and quality of the act that constitutes the excess.”Endnote vii Staff behaviour and decision-making in managing incidents in institutions are guided by CSC’s core values of respect, fairness, professionalism, inclusion, and accountability.Endnote viii Any engagement or intervention strategy selected by frontline officers should therefore be necessary and proportionate to the level of risk;Endnote ix surpassing this threshold could result in criminal and/or civil liability.
The SMM was intended to ensure that incidents occurring in federal correctional institutions were managed in a way that supported a safe and secure environment for both inmates and staff. However, over the years, deficiencies in its operational application emerged. The death of an inmate, Matthew Ryan Hines, during an institutional incident in 2015 led to an investigation by the Office of the Correctional Investigator (OCI) into the circumstances surrounding his death. The OCI identified failures in the timeliness and appropriateness of the response of CSC staff to medical emergencies and/or acute mental health distress of inmates.Endnote x The OCI also identified inadequacies in the appropriate and safe use of chemical and inflammatory agents, including the application of accountability and oversight for their use.Endnote xi Further, an investigation by the OCI and the Canadian Human Rights Commission into the experiences of older individuals in federal custody also identified a need to incorporate best practices and lessons learned regarding the use of force on older inmates.Endnote xii
In August of 2017, CSC’s Security Branch released a report summarizing their analysis of use of force incidents at CSC institutions. The report identified various areas for improvement, including a need for CSC to:
- Introduce a modified SMM that emphasized situational analysis, de-escalation, and non-physical responses to institutional incidents;
- Add data fields to the Offender Management System (OMS) and implement an automated Statement/Observation Report that would enhance the efficacy, quality, and breadth of analysis as well as monitoring that could be conducted in relation to use of force incidents;
- Provide front-line staff more guidance and improved training on incident response options that emphasize better de-escalation tactics and/or planned interventions;
- Review policy to strengthen the alignment of the Sector Coordinator position with Correctional Manager roles and responsibilities, and enhance the Sector Coordinator’s ability to coordinate layered responses to the specific circumstances and situational factors involved in an institutional incident;
- Conduct a review of Oleoresin Capsicum spray (OC spray) to ensure the continued safety of its use and as well to examine alternative non-lethal use of force tools;
- Review use of force trends and analysis on an ongoing basis and improve communication with management groups at the local, regional, and national levels; and
- Examine the potential use of body cameras in the video documentation of use of force incidents.Endnote xiii
In addition, the Internal Audit Sector of CSC completed an Audit of the Framework and Implementation of Situation Management at CSC in 2018.Endnote xiv Results concluded that management needed to address a number of issues to improve the manner in which CSC staff manage institutional incidents. In particular, the audit results indicated that:
- There was no clear prescription and assignment of who was in charge of controlling and leading a response to an institutional incident when multiple staff members were responding to an incident;
- Training on the use of force module was not consistently provided to staff;
- There was a lack of guidance material regarding how to perform a use of force review, including the nature of corrective action required for different types of non-compliance;
- There was insufficient performance monitoring and reporting at the local, regional, and national levels;
- Intervention plans were not always documented as required; and
- First aid and physical assessments were not always completed following a use of force incident.Endnote xv
Findings from the OCI investigations into the use of force in federal institutions, Security Branch’s review and report on use of force within CSC, and CSC’s internal audit of situation management all confirmed that a shift in responding to and managing institutional incidents within CSC institutions was necessary. The EIM was developed and introduced to improve how CSC staff assess and intervene during an institutional incident.
Program description
The EIM can be described by summarizing the model according to the following three core components: the philosophy behind the model, how the model works, and management post-institutional incident (a detailed description and visual depiction of the model is included in Appendix A).
The EIM is aligned with the core responsibility of Care and Custody within the Departmental Results Framework (DRF), and within the program area of Institutional Management and Support (P1). For a detailed description of the EIM logic model refer to Appendix B.
About the evaluation
CSC’s Departmental Evaluation Plan (2019-2024) includes a commitment to conduct an evaluation on the impact of the EIM on managing security incidents. The 2017-2018 Annual Report of the OCI Endnote xvi also recommended that CSC conduct an evaluation of the EIM to provide stakeholders with an enhanced understanding of its performance in key areas since its implementation. The results may be used to provide CSC with necessary information to make strategic policy, operations, and resource allocation decisions as they relate to the EIM.
In accordance with the Treasury Board of Canada Policy on Results Endnote xvii (2016), the evaluation focused on the relevance and effectiveness Footnote 2 of the EIM in the management of institutional incidents. As such, the current evaluation focuses on two core evaluation objectives:
- Relevancy of the EIM, such as its alignment with government priorities and consistency with federal roles and responsibilities (e.g., does the EIM address a demonstrable need within federal corrections?); and
- Effectiveness of the EIM (e.g., are there any barriers to quality improvements in the way that the EIM is currently functioning?).
In addition to these two core evaluation objectives, the current evaluation also focuses on the design and delivery of the EIM to examine the alignment of the EIM implementation with identified outcomes.
Methodology
The following describes the methodology employed for this evaluation in a brief format to help with understanding the results presented in this report.
Evaluation findings are presented across three main areas: relevancy, design and delivery, and effectiveness. To evaluate relevancy, a review of government literature, grey literature, and internal documents, such as CSC policies, legislation, evaluation reports, research reports, and operational documents, was conducted.
To evaluate the design and delivery of the EIM, a number of key sources of information were used.
- Offender Management System (OMS) data were extracted and used to identify institutional incidents and use of force review package indicators for both the SMM and the EIM time periods (for comparison purposes). Specifically, incident reports and use of force review packages for incidents that occurred on or after April 1, 2016, as well as those that had been either submitted or reviewed on or before December 31, 2017 (21 months), were retained and analyzed as incidents that were managed under the SMM. Incident reports and use of force review packages for incidents that occurred on or after January 1, 2018, as well as those that had either been submitted or reviewed on or before September 30, 2019 (21 months), were retained and analyzed as incidents that were managed under the EIM.
- All results indicating that there were changes between the EIM and the SMM periods are statistically significant. Further, reference is made to the effect size when reporting statistics, which is a statistical concept measuring the strength of a relationship. A detailed description of the results for design and delivery is included in Appendix C, including the results of significance testing and effect sizes.Footnote 3
- Compliance reports from CSC’s Human Resources Management System (HRMS) for EIM-specific training were also analyzed, and the number and percentage of staff who were deemed EIM training compliant were reported.
- Key informant interviews were conducted with stakeholders who were directly or indirectly involved in, or familiar with the EIM (e.g., Project Officers, Chiefs of Mental Health, and Assistant Wardens of Operations). In all, 15 key informants were individually interviewed, using an interview guide of approximately 20 open-ended questions.
- Online staff survey data from 237 respondents were analyzed, representing a 12.2% response rate. Notably, any responses that were missing were excluded from analyses (frequency counts and percentages). Additionally, I don’t know responses are presented as part of the findings when a large proportion of respondents selected this response option, however, frequency counts and percentages are not discussed as they are excluded from the survey analyses presented herein. This resulted in the total number of responses changing across items. Finally, given that the majority of survey items were rated on a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree), responses were reported in an aggregated fashion (i.e., strongly agree and somewhat agree responses were aggregated for each item and are reported as agree; strongly disagree and somewhat disagree responses were aggregated for each item and are reported as disagree).
Finally, to evaluate effectiveness, key informant interviews and staff survey data were used. Additionally, HRMS data on disciplinary actions taken for excessive use of force by staff were examined. These data were provided for two time periods - between April 1, 2016, and December 31, 2017 (SMM), and between January 1, 2018, and September 30, 2019 (EIM).
Where possible, findings specific to the Regional Treatment Centers (RTCs) were reported separately. Additionally, where possible, findings pertaining to diverse sub-populations of inmates were reported (as defined by gender based analysis plus (GBA+), including female, younger, older, Indigenous, and ethnocultural inmates). Refer to Appendix D for the evaluation questions and matrix, and Appendix E for a detailed explanation of the methodology.
Limitations, impact and mitigation strategies
The evaluation team encountered certain limitations while conducting the evaluation and put in place strategies to mitigate their impacts on the findings and recommendations (see Table 1 below).
Table 1. Limitations, impact, and mitigation strategies
Limitation | Mitigation strategy |
---|---|
Sample size/response rate too small to conduct meaningful analyses and/or draw definitive conclusions. | Any OMS and staff survey findings with a sample size of less than five (n < 5) were not reported. |
Unable to evaluate the reliability of the data flag indicating whether a physical, mental, or cognitive impairment was present. | Only sufficient and reliable data that were available were examined for diverse sub-populations of inmates. Footnote 4 |
No OMS data on LGBT2Q+ Footnote 5 inmates | Only sufficient and reliable data that were available were examined for diverse sub-populations of inmates. |
Inability to access Statement/Observation Reports (SORs) due to Coronavirus Disease (COVID-19) causing the prohibition to do site visits to collect data from SORs. | OMS data were used as a standalone data source without additional context from SORs. Questions were included in the staff survey and key informant interviews to gain perspectives around non-use of force intervention practices. |
Due to the onset of COVID-19 the, timelines for the evaluation were extended by six months. | The final report will receive Commissioner approval by June 2021. |
Authors:Footnote 1
Prince Baffoe, Junior Evaluation Analyst
Rebecca Cherner, Evaluation Officer
Sidikat Fashola, Evaluation Officer
Sara Johnson, Senior Evaluation Manager
Karen Koundakjian, A/Senior Evaluation Manager
Natalie Mercer, Evaluation Analyst
Kayla Wanamaker, A/Evaluation Officer
Contributors
Danielle Guarino, Junior Evaluation Analyst
Mylène Mailhot, Student Analyst
Page details
- Date modified: