Evaluation of the Community Mental Health Initiative

The Evaluation team would like to thank all the staff, offenders, and community service providers who took time to share their thoughts on the CMHI by completing a survey or agreeing to participate in an interview.

We would like to thank staff who provided information and documentation utilized in this evaluation. In particular, we would like to express our appreciation to the Evaluation Consultative Group Members, including Health Services (Natalie Gabora-Roth ), Community Reintegration Operations (Robert Safire), Aboriginal Relations (Jennifer Hayward), Women Offender Sector (Chris Hill, Mark Christie), Performance Management (Lisa Hill), Performance Assurance (Jennifer Guillon, Jessie Rylett), and Regional Psychiatric Center (Prairies; Treena Witte). Thanks also to Andrea Moser, Jane Laishes, Michael Martin, and Thanujah Yogarajah for their assistance responding to specific questions and providing information.

We are also grateful for the assistance provided by staff members who coordinated the site visits (Pacific Region: Gurjit Toor, Heather Pierce, Treena Prox, Karen Sloat; Prairies Region: Amy Howie, Lisa Gluck, Clarence Turgeon, Jason Mackenzie, Darrell Lindsey; Ontario: Johanna Kudoba; Quebec, Martine Lacroix, Stéphanie Gérin and Suzanne Pelletier; Atlantic: Paul Harris).

Thank you to the Executive Steering Committee for your feedback and guidance. Terry Nicholaichuk, Deqiang Gu, and Treena Witte at the RPC were instrumental in providing many of the analyses in this report. When additional analyses were required they worked diligently to provide these under very pressing timeframes.

We would also like to thank Vanessa Anastasopoulos for her assistance with conducting interviews. Kelly Taylor provided methodological advice and peer review assistance, and Cara Scarfone and Amanda Nolan also made various editorial contributions. The Team would also like to thank Mark Nafekh for data analysis assistance and Annie Yessine for her analytical expertise. Thank you to Lindsey Pecaric for her assistance providing a cost-effectiveness literature review.

Jean-Pierre Rivard, Ghalib Dhalla, Robert Riel, and the regional personnel provided invaluable assistance gathering the financial data, and Cameron Bouchard from the Public Safety Geomatics Division provided expertise and assistance in generating the geo-mapping analysis.

Addressing the mental health needs of federal offenders is one of Correctional Service Canada’s (CSC) five strategic priorities (CSC, 2008a). In 2007/08, 10% of men offenders and 22% of women offenders in federal custody were identified as presenting with mental health problems at intake. These percentages represent 67% and 69% increases, respectively, since 1996/97 (CSC, 2008b). In the 2006 report of the Standing Senate Committee on Social Affairs, Science and Technology on mental health care in Canada, Out of the Shadows at Last, also known as the Kirby Report (Kirby, 2006), the Committee challenged CSC to meet the mental health standards of care for offenders under federal jurisdiction that are typically afforded to non-offender populations. Moreover, the Committee specifically recommended that CSC provide services to ensure continuity of mental health care from the institutions to the community.

CSC is mandated by legislation (86(1) of the Corrections and Conditional Release Act [CCRA], 1992) to provide mental health services to federally sentenced offenders although the provision of these services have been deemed inadequate (see Canadian Public Health Association, 2004; Correctional Investigator Canada, 2004; and Kirby, 2006). Findings and recommendations from the Kirby Report, as well as reports from the Office of the Correctional Investigator (2004), CSC Review Panel (2007), and the Government of Canada’s Performance Report (2007b) have highlighted the need and provided additional support for the development of a comprehensive mental health strategy. CSC is implementing such a strategy for federal offenders, the fundamental goal of which is to ensure a continuum of mental health services to offenders from institutional intake to release into the community. This approach focuses on:

1. intake screening and assessment;
2. primary care;
3. intermediate care;
4. intensive care (at Regional Treatment Centres); and
5. transitional care.

In 2005, following submission by the Health Service Branch, Treasury Board allocated funds to implement the Community Mental Health Initiative, the fifth component of the overall Mental Health Strategy.

The Community Mental Health Initiative (CMHI), implemented in 2005, falls within the transitional care component of the national mental health strategy. The key components of the CMHI are:

• Increased discharge planning, provided by clinical social workers, for offenders with mental disorders at men’s and women’s institutions;
• Allocating mental health specialists (clinical social workers and mental health nurses) to support offenders with mental disorders (OMDs) residing in the community [including Community Corrections Centres (CCCs) and Community Residential Facilities (CRFs)];
• Providing resources and services to respond to the special needs of OMDs in the community (e.g., contracts and funds for psychiatry and other mental health interventions, specialized assessments, tutors, etc.); and
• Providing mental health training to correctional services staff, halfway house staff, and community partners (Champagne, Turgeon, Felizardo, & Lutz, 2008).

CMHI Budget

A total of $29.1 million over a five year period for the Community Mental Health Initiative was approved in 2005. The majority of the funding ($15.3 million) was designated for salaries, including $7.8 million for operating costs and approximately $6 million for common services, employee benefit plans, and accommodations.

An evaluation strategy was developed by the Evaluation Branch in consultation with the evaluation consultative group, comprised of stakeholders from CSC Health Services, Aboriginal Initiatives Directorate, Women Offender Sector, Performance Management Branch, and Regional Psychiatric Centre Prairies Research Branch. The purpose of the evaluation was to provide information required to make investment decisions in the area of community mental health beyond the expiration of the funding in March 2010. Due to the recent implementation of the CMHI, the evaluation was primarily implementation focused, but immediate and intermediate outcomes were assessed where possible given the availability of data.

Qualitative and quantitative methodologies were utilized to conduct the evaluation. Information was collected through:

• Surveys of CSC staff members who had experience working with offenders with mental disorder in August 2008;
• Interviews with offenders in the community who received services from community mental health specialist teams;
• Consultations with regional coordinators to develop implementation timelines;
• Automated data collection, including queries of CSC’s Offender Management System (OMS) and other databases created and maintained by Health Services and the Regional Psychiatric Centre (RPC);
• Review of relevant documentation, including implementation and post-implementation reports from the RPC, the mental health training summary report from CSC’s Health Services Sector, operational documents, relevant CSC policies and procedures, and financial documentation from the Integrated Management Reporting System (IMRS); and
• Review of relevant literature, including government and non-government publications, reports from international jurisdictions, and academic and professional publications.

To examine the effectiveness of the CMHI in the successful reintegration of offenders into the community, outcomes for offenders who received discharge planning services and community mental health specialist services were compared to a historical comparison group that did not receive CMHI services.

SUMMARY FINDING: The community mental health initiative remains consistent with CSCpriorities, government-wide objectives, and practices in other jurisdictions and addresses a realistic need for mental health services among CSC offenders. The CMHI resulted in increased access to mental health services, including discharge planning services and mental health services in the community provided by community mental health specialists or contractors. Offenders who received community mental health specialist services were less likely to be suspended or revoked than the comparison group who did not receive CMHI services. Some implementation challenges were observed, including delays in staffing CMHI positions (which resulted in re-profiling and lapses of CMHI funding), challenges related to information sharing between the institution and the community, and some stakeholders reported that some offenders in need of mental health services were not being referred for services.

• FINDING 1: The CMHI remains consistent with departmental and government-wide priorities
• FINDING 2: Given the increasing number of offenders entering CSCwith mental health disorders, there is a need to provide services for these offenders to address their mental health needs and assist them to successfully reintegrate into the community.
• FINDING 3: The CMHI is consistent with other jurisdictions’ practices, particularly those that employ community-based models of mental health intervention for offenders.

• FINDING 4: Delays in implementing CMHI services were attributed primarily to staffing challenges. Successful implementation of the CMHI was more likely when there were: (1) dedicated human resource and administrative support to expedite the staffing processes; and (2) a wide recruitment campaign to draw many potential candidates to staff the initiative.
• FINDING 5: Discharge planning referrals are not occurring in accordance with CMHI guidelines regarding timeframes (i.e., nine months prior to anticipated release date).
• FINDING 6: The most common reason for CMHI referral rejections occurred because offenders did not meet inclusion criteria. Staff also suggested that some offenders in need of services were not being referred. Examination of findings suggests that this may be due to a lack of knowledge among CSC staff members regarding CMHI referral criteria, lack of reliable tools to facilitate early identification of those in need of services, and/or a lack of available services in the communities to which the offenders are being released.
• FINDING 7: Implementation challenges were reported related to coordination and information sharing among institutional and community mental health and case management teams.
• FINDING 8: Existing CMHI sites appear to be well-placed to serve offenders with mental health needs as demonstrated by the number of offenders with mental health needs at existing CMHI sites. However, there are several CSC sites with significant proportions of of offenders with mental health needs that have not been identified for CMHI services.
• FINDING 9: Implementation delays have led to several instances of re-profiling and lapses of CMHI funding. Financial data for the CMHI have not always been coded consistently utilizing the appropriate cost-centres in IMRS

• FINDING 10: Mental health training was provided to 830 individuals in the community and 352 CSC institutional staff members who worked with individuals with mental disorders. Among institutional staff, trainees were primarily CSCnurses. The training was effective in improving community personnel’s mental health knowledge and self-perceived competency to work with offenders with mental disorders.
• FINDING 11: Offenders referred to, and accepted for, CMHI services, including discharge planning and community mental health specialist services, are receiving these services. However, data regarding their termination from these services may not always be consistently recorded in CMHI database.
• FINDING 12: The number of CMHI service contracts and number of offenders receiving services have increased over time. Contract services are being delivered in a timely manner and few offenders referred for services were waitlisted or not provided with the services.
• FINDING 13: Community capacity building efforts have increased over time and service-building contacts have generally focused on the areas of highest need or importance according to CMHI referrals.
• FINDING 14: Stakeholders generally reported enhanced continuity of care and services, although some concern was reported regarding continuity of care after warrant expiry.
• FINDING 15: Stakeholders generally reported that the CMHI contributed to an improved quality of life for offenders. However, the CMHI standardized assessment of quality of life was not administered to offenders per CMHI guidelines. As a result, findings pertaining to this assessment were inconclusive due to small sample sizes.
• FINDING 16: The majority of offenders received either clinical discharge planning (CDP) or community mental health specialist (CMHS) services, but not both. Offenders receiving CMHS services were less likely to be suspended or revoked than the comparison group, after statistically controlling for pre-existing group differences. There was no evidence to suggest that the CDP group differed from the comparison group with respect to these outcomes. These preliminary findings should be interpreted with caution due to small sample sizes and short follow-up times.

• FINDING 17: Although the CMHI has demonstrated several positive short-term outcomes, limitations related to financial coding and the short implementation period precluded the ability to conduct a reliable cost-effectiveness analysis at this stage of the CMHI.

• RECOMMENDATION 1: To sustain and enhance mental health services provided in the community, CSC should support the implementation of mental health services through the development of a strategic staffing process and recruitment campaign.
• RECOMMENDATION 2: Procedures or processes to improve early identification of offenders’ mental disorder and treatment needs should be explored in order to enable accurate identification of offenders with mental health needs, to better facilitate treatment referrals, and to establish continuity of care from an earlier stage.
• RECOMMENDATION 3: CSC should explore and develop mechanisms to increase information-sharing across institutional and community mental health and case management teams.
• RECOMMENDATION 4: Several sites that are not presently included in the CMHI that have large proportions of offenders with mental health needs should be considered for CMHI services, through reallocation or expansion of CMHI services.
• RECOMMENDATION 5: CSC should ensure accurate, standardized coding of CMHI expenditures in financial databases to ensure that expenditures are adequately recorded and monitored and so the cost-effectiveness of the CMHI can be adequately assessed at some future time.
• RECOMMENDATION 6: Additional mental health training should be provided to institutional staff members, including parole officers and other case management team members in order to assist in identifying OMDs and providing early referrals for CMHI services.
• RECOMMENDATION 7: Offenders accepted for CMHI services should be tracked to ensure that treatment has been provided and to monitor the length of time that offenders receive services.
• RECOMMENDATION 8: CSC should continue to support and enhance the level of services available to offenders with mental disorders in the community. Further, CSC should explore the development of additional partnerships/links with organizations (such as provincial governments and non-governmental organizations) to facilitate continuity of care following warrant expiry.
• RECOMMENDATION 9: CSC should review the Quality of Life Scale administration guidelines to ensure that guidelines for administration are practical and develop procedures to ensure that CMHI staff engage offenders in completing the assessment of quality of life as per the guidelines.
• RECOMMENDATION 10: CSC should review the possible reasons for lack of continuity from CDP services to CMHS services. Based on this review, CSC should develop strategies and procedures to better impact community reintegration for CDP offenders.

The goal of the evaluation was to provide information required to make investment decisions in the area of community mental health beyond the expiry date of the funding at the end of March 2009/10. The continued relevance, success, cost-effectiveness, unintended outcomes, and implementation issues associated with the initiative were assessed. Note that implementation of the initiative was delayed due to difficulties staffing CMHI positions. Therefore, the evaluation was implementation focused. However, the success of the initiative as it related to achievement of immediate and intermediate outcomes were assessed where possible given the state of implementation and the availability of reliable data. The comprehensive evaluation matrix is shown in Appendix A, identifying the CMHI evaluation questions, performance indicators, and sources.

The Logic Model for the CMHI is shown in Appendix B. As described earlier, the CMHI includes four main activities, namely: staff training, the provision of community mental health specialist services, clinical discharge planning, and establishment of community service partnerships through contracts.

Immediate outcomes of the CMHI include:

• Increased staff awareness of mental health issues;
• Standardized provision of services;
• Offender access to available services; and
• Increased availability of services and support for offenders with mental disorders being released and in the community.

Intermediate outcomes of the CMHI include:

• Improved services for offenders with mental disorders;
• Improved correctional outcomes for offenders with mental disorders; and
• Improved quality of life for offenders with mental disorders.

Ultimately, the goal of the CMHI is to contribute to the safe accommodation and reintegration of eligible offenders into Canadian communities by providing them with reasonable access to mental health care.

The extent to which these outcomes have been achieved will be explored further in the evaluation results.

A multi-method approach incorporating qualitative and quantitative methodology was utilized to address the evaluation objectives. This included a review of program documentation and reports (e.g., CMHI Guidelines), financial data, surveys and interviews with key informants, and offender data extracted from the Offender Management System (OMS) and CMHI -specific databases maintained by Regional Psychiatric Centre Prairies (RPC) and the Health Services (HS) at NHQ.

Financial information was collected from the Integrated Management Reporting System (IMRS). Representatives from the Comptrollers Branch provided a complete summary of CMHI budgets, re-profiles in funding, and expenditures.

Feedback regarding issues related to the relevance, implementation, and success of the CMHI was obtained from three different key informant groups:

1. CSC staff ;
2. offenders; and
3. community service providers.

CSC Staff

An electronic survey was distributed through CSC internal email announcements (i.e., General Communication) to CSC staff members who had experience working with offenders with mental disorders, including staff members who were directly involved with the CMHI as well as others who were familiar with the CMHI. The survey was active for a period of 16 days from August 25, 2008 to September 9, 2008. Informal contacts were also held with national and regional CMHI personnel to establish implementation timelines and discuss implementation challenges.

Offender interviews were conducted at parole offices, CCCs , and institutions in each region. The Evaluation Branch selected the site in each region that had the most CMHI offenders currently under supervision. CMHS staff coordinated the offender interviews.

Surveys were conducted with community service providers, including agencies (e.g., John Howard Society, Stella Burry Community Services, etc.) and individuals (e.g., psychiatrists) under contract to provide services directly to offenders through CMHI. These agencies were identified through bi-annual reports submitted to HS that also contained service providers’ contact information. Where contact information was missing or out of date, agencies were contacted directly for an update. The original list of community service providers consisted of 42 independent agencies/organizations. Of these, 35 were contacted, and 7 were unreachable for a variety of reasons (e.g., invalid email addresses). The surveys were sent through email to the identified contact person at each agency, who was asked to complete the survey and to send it to any colleagues within their agency who had direct knowledge of the CMHI. A follow-up email was sent as a reminder to complete the survey, and also informed respondents that should they have difficulty accessing the survey in the email, alternate arrangements were available.

Finally, offender information (such as offender risk, need, demographic characteristics, correctional outcomes, time spent in the community, and other pertinent information) was extracted from the Offender Management System (OMS; automated database maintained by CSC) and other databases created and maintained by Health Services at National Headquarters and RPC. OMS is an electronic filing system designed to monitor and track offenders under the supervision of the Correctional Service Canada. Data captured in OMS include the Offender Intake Assessment (OIA), a comprehensive and integrated examination of offenders at the time of their admission. The process begins with an assessment of immediate mental and physical health concerns, security risk and suicide potential and offender risk factors and dynamic need indicators. The OIA indicator “diagnosed as disordered currently” was used to identify those offenders with mental health needs who were released to parole offices and CCCs across Canada.

There were three assessment instruments used in the evaluation of the CMHI. The first two, Alberta Continuity of Service Scale for Mental Health (ACSS-MH) and the Quality of Life Scale (QoL) were intended to be administered directly to the offender in the course of their treatment. Because very few ACSS-MH scales had been completed at the time of the evaluation, the ACSS-MH questions were incorporated into the offender interviews. The third measure, Level of Functional Impairment (LFI), was used for research purposes to rate the impairment of both the treatment and comparison groups. A training questionnaire was also administered to training participants.

Alberta Continuity of Services Scale for Mental Health (ACSS-MH)

The ACSS-MH (Adair et al., 2004) is a tool used to assess the service consumer’s (or in this case, federal offenders participating in the CMHI initiative) perception of continuity of services. The tool was adapted for use with CMHI participants (and certain questions were omitted) so results may not be directly comparable to results reported in the literature. The modified survey consisted of 40 items rated using a 5 point Likert scale from 1 (strongly disagree) to 5 (strongly agree). The scale was administered by an interviewer during a face-to-face interview by one of the evaluation staff members. Eighteen questions were worded negatively, so reverse coding was necessary before summing the item scores to derive a total score. The ACSS-MH is comprised of three subscales, namely system fragmentation (21 items), relationship base (9 items), and responsive treatment (10 items). Questions focus on areas such as service accessibility, service integration, individual or team mental health care providers, and the overall satisfaction with the services received.

Quality of Life Scale (QoL)

The QoL was utilized to measure the overall quality of life of a client (Lehman, 1988). Offenders were asked to complete 26 self-report questions from the scale. The questions were answered on a 7-point scale ranging from “terrible” to “delighted” and focused on several key areas including: general life satisfaction, living situation, daily activities, social relations, family, finances, work, legal and safety issues, and health. According to CMHI guidelines,^Footnote4 the questionnaire was to be administered to offenders at one-month , three-months, and six-months after their receipt of community care. It can be administered by someone actively involved in their care such as their clinical social worker or nurse (i.e., CMHS ). Due to initial implementation challenges, the QoL data were not collected on the majority of offenders. Data were available at time 1 and time 2 (one and three months) for 36 offenders in the CMHS group.

Level of Functional Impairment (LFI)

The LFI scale was used as a research tool to assess treatment and comparison group offenders’ degree of impairment in four areas including daily living/personal hygiene, intellectual, occupational, and social/interpersonal functioning. OMS data were used to rate each item on a four-point scale (0 – 3), with higher aggregate scores indicating greater impairment (range of scale from 0 to 12). Total score ratings of 5 or greater constituted moderate to severe functional impairment (CSC, 2008d). Inter-rater reliability was obtained on the LFI Rating Scale. Five practice cases were rated, followed by 10 cases from the random sample. For the initial 10 cases, 8 out of 10 were consistently identified as being referred for services and an intra class correlation of r = 0.34 for the overall functional impairment score. An additional 10 cases were then rated. Of these 10 cases, 7 received the same referral decision from the raters and an intra class correlation of r = 0.58 for the overall functional impairment score was obtained. Another 10 cases were rated and resulted in 8 out of 10 receiving the same referral decision and an acceptable intra-class correlation (r = 0.86) overall (CSC, 2008d). The LFI was originally designed to enable identification of a comparison group for outcome analysis in combination with other indicators such as an Axis I diagnosis. Furthermore, LFI scores were used to provide descriptive information about the treatment and comparison groups.

Participants were asked to complete a 15 item mental health training evaluation form, in which 9 of the items asked participants to rate the usefulness of each of the nine modules of the training program (i.e., introduction, myths and realities, what is a mental disorder, types of mental disorders, cognitive disorders and FASD, risk and mental disorder, effective strategies, resources, and legislative requirements) and another 3 asked the participants to rate the usefulness of the participant resource manual, the trainer(s), and the overall training. These 12 items were rated on a 7 point scale from 1 (not useful) to 7 (very useful). Lastly, three open-ended questions were included to ask the participants to

1. provide suggestions for improvement of the training;
2. identify topics that should be addressed in follow-up training sessions; and
3. provide suggestions as to how the trainer could improve his/her delivery.

Mental health training participants were asked to complete a 10 item mental health quiz prior to and immediately following training. The quiz was comprised of multiple-choice and true/false questions as well as fill-in-the-blank and open-ended questions. Topic areas addressed in the quiz included (but was not limited to) mental health disorders, mental health symptoms, treatments, side effects of psychotropic medications, and myths about OMDs.

The self-perceived competency scale is an 8-item scale designed to assess competencies that were targeted by the training. Sample items were

and

Each item was rated on a 7 point scale from 1 (strongly disagree) to 4 (uncertain) to 7 (strongly agree). Participants of the mental health training were asked to complete this scale prior to and immediately following training.

Themes were generated from open-ended survey and interview questions, and were compared across multiple team members to ensure agreement. Themes are presented in the appropriate Key Findings sections below (and detailed in Appendix C). Key informant interviews/survey questions were often asked on a 5-point Likert scale ranging from strongly disagree to strongly agree (1 = strongly disagree; 2 = disagree; 3 = neither agree nor disagree; 4 = agree; and 5 = strongly agree). In general, interview/survey results were collapsed across the agree and strongly agree categories to create an ‘agree’ category and the disagree and strongly disagree categories were combined to create a ‘disagree’ category.

With the assistance of Public Safety Geomatics Division, evaluation team members created two series of maps using ArcGIS software to provide a visual representation of the locations of CMHI offices and the percentages of offenders who (a) were identified as having a mental disorder at each supervising office in each region, and (b) who received CMHI services, including CDP and CMHS services. The first set of maps identifies the percentage of offenders at each site who are identified by the OIA indicator “diagnosed as disordered currently” while the second set of map presents the proportion of offenders who received CDP and CMHS services. For both series of maps, the symbology identifies the type of office (Parole Office or CCC ) and the presence or absence of CMHI services at each office.

Health Services (HS) Mental Health Training Summary Report

HS maintained databases on all issues related to mental health training as part of the CMHI (e.g., attendance, training evaluation questionnaires, self-perceived competency scale, mental health knowledge quiz) and produced a summary report on the outcomes of all of the 2 day training sessions delivered from January 2007 through June 2008. At the time when the mental health training summary report was written, attendance records up to the end of 2007/08 were verified for accuracy against HRMS by HS staff for all CSC staff members. HS also provided the evaluation team with access to their mental health training databases for additional analyses reported in the present report.

Regional Psychiatric Centre (RPC) Data Analysis and Reports

RPC is under a Memorandum of Understanding (MOU) with NHQ Health Services to manage CMHI data (e.g., referral and outcome data), and to provide a series of reports documenting the pre-implementation, implementation and post-implementation milestones and correctional outcomes at each stage of the initiative. According to the terms of the agreement, RPC was expected to provide:

• one pre-implementation report (documenting the establishment of the retrospective comparison group);
• five implementation reports (documenting the implementation of CMHI services for both treatment groups); and,
• two post-implementation reports (documenting the efficacy of CMHI in improving correctional outcomes and enhancing the quality of life for OMDs ). RPC has provided some of the analyses included in the report and is cited as a source of these analyses where applicable.^{Footnote 5}

There were several limitations that impacted upon the ability to examine the evaluation objectives of CMHI, including the use of a historical comparison group, lengthy implementation delays, an inability to identify offenders who may have dropped out or had their CMHI services terminated shortly after accepting their referrals, the small number of offenders who received both CDP and CMHS services to date, and the use of the OIA indicator as an index of CMHI referral criteria.

The comparison group used for the effectiveness analysis was historical in nature. The CMHI treatment recipients documented in the effectiveness analysis received treatment from May to December 2007, but the comparison group was comprised of offenders eligible for release between April 1st, 2003 and March 31, 2005. This was necessary to ensure no confounds of the comparison group with respect to possible exposure to treatment through CMHI. However, this adds a possible confound in that changes occurring over time might affect groups differentially (i.e., cohort effect). As documented in other sections of the report, mental health intervention is an increasing priority within the federal government and within CSC. Enhancements to service availability and delivery in recent years in the institution may have had an impact on the treatment groups, but not the comparison groups. This possible confound may be limited as funding to begin to address elements of the institutional mental health strategy was only provided in April 2007. However, there is no way to isolate the CMHI in relation to other CSC initiatives occurring at the same time.

Lengthy delays in staffing positions resulted in delays in full implementation of the initiative. As the national implementation timelines illustrate (see implementation section of the report), there were many time consuming administrative tasks that took place prior to staffing positions through competitive processes (e.g., creating job descriptions, classifying positions, posting employment opportunities, running competitive process). Although the initiative was announced in May 2005, the first offender referral for CMHI service was not made until two years later in May 2007.

With a five-year initiative, implementation delays, and a requirement for an evaluation by June 2009, there was a limited follow-up period to examine offenders’ progress in the community. The effectiveness component of the evaluation included a potential follow-up time for offenders of 6 to 13 months in the community (depending on when they began receiving services). Descriptive data for CMHI offenders was provided through to June 2008. However, the offender sample utilized for the outcome analysis (i.e., recidivism) included only offenders referred for service from the beginning of the initiative (May 2007) until December 2007, in order to allow for an adequate follow-up time in the community. Also, given the short period of time in which the initiative was operational, the treatment groups are small (N = 53 for offenders receiving discharge planning services; and N = 79 for offenders who have received community mental health specialist services). Furthermore, treatment dosage should be considered as short follow-ups (especially for CMHS participants) may limit the amount of CMHS service that can be delivered. Given these reduced samples and very short follow-up times, it was difficult to draw strong conclusions with the resulting data. Longer follow-up time and increased numbers within each treatment group will be required to provide more reliable results. Also, it was hoped that Aboriginal and women-specific correctional outcomes could be assessed. However, given the small sample sizes, demographic information for these groups was reported but no further analyses could be performed.

The treatment group dataset was generated with the receipt of a completed Referral for Service form indicating that the offender was accepted for service. Once the CDP or CMHS staff received a referral form from an offender’s parole officer, the hard copy was stored in the offender’s file and an electronic copy was saved on the national network drive to be entered by HS analysts. Receipt of this form indicating the offender was accepted for service marked the offender as a treatment recipient. Acceptance to treatment was based on two criteria:

1. presence of a major mental disorder, or
2. a moderate to severe impairment from a personality disorder, acquired brain injury or organic brain dysfunction, or developmental disability or intellectual impairment. The offender must have met the criteria and voluntarily agreed to participate.

If an offender was referred but refused to participate, the referral form would still be submitted but the refusal decision noted and he or she would not be included as part of the treatment group. However, if the offender received services that were later terminated (either by the service provider or himself/herself), that information may not always have been consistently recorded. Therefore, it is possible that offenders in the treatment group might not have received treatment for any significant period of time and it was not possible to differentiate these offenders from those who received more extensive treatment.

There were some offenders who received both CDP and CMHS services (N = 23; CSC, 2008c). It is unclear why this number was so low because ideally, continuity of service should be provided from the institution through offender discharge planning, followed by CMHI services provided by the CMHS team. Because the majority of CMHS offices are in large urban centres, it was expected that more CDP recipients would have received CMHS services. Reasons for this limited continuity of service may be related to implementation issues, in that offenders being released to CMHI sites did not have adequate time to complete a CDP because the CDP service was not yet operational in their releasing institution. Similarly, offenders who received CDP services may have been released to sites that were not yet CMHS operational locations.

Because the group of offenders receiving both services was too small to analyse independently, the offenders who received both CDP and CMHS services were included in both treatment groups. Once the treatment group numbers increase, this group of offenders receiving both services should be examined separately and in more detail to understand whether there is an enhanced effect of receiving both services.

For the geo-mapping exercise, two sets of data are presented, one based on those who received CMHI services and the other was a representation of offenders who have been identified as having a mental health need in the community. Mental health needs were identified using the OIA indicator “diagnosed as disordered currently”. The OIA indicator was used as a proxy measure for those who met the CMHI referral criteria. The OIA indicator is not an entirely accurate representation of those eligible for CMHI. First, the OIA indicator is based on offender self-reported information and it is static (i.e., assessed only at intake, and is not updated during the course of an offender’s sentence). In fact, a review of OMS data indicated that only 55% of offenders who were accepted for CMHI services^{Footnote 6} had the OIA indicator “diagnosed as disordered currently” on their OMS file, suggesting that not all offenders with serious mental health needs were being identified utilizing this indicator. Second, the OIA indicator does not take into account the referral criteria for the CMHI (described earlier, including the criteria of a major mental disorder or severe impairment in one of several areas of mental functioning). Based on a sample of CSC offenders with the OIA indicator selected for possible inclusion in the CMHI outcome analysis comparison group, only 61% of those with the OIA indicator actually met the CMHI referral criteria (CSC, 2008d).

The sample included in the outcome (recidivism) analysis was comprised of three groups: the clinical discharge planning recipients (CDP), the community mental health specialist service recipients (CMHS), and a comparison group (CMHI Comparison; refer to Appendix D for profiles of offenders referred for services). The two treatment groups (CDP and CMHS ) included all offenders who received services from the beginning of the initiative until December 2007, the cut-off date that allowed for a potential 6-month community follow-up for the treatment groups.

The comparison group was generated by using a historical cohort of offenders who were eligible for release between April 1st, 2003 and March 31, 2005, and who had the OIA indicator “diagnosed as disordered currently” (CSC, 2008d). Those offenders who met the referral criteria but did not receive services (because the CMHI had not been implemented at that time) became the CMHI comparison group. The referral criteria for CMHI services include the presence of a major mental disorder (e.g., schizophrenia, mood disorder) personality disorder (e.g., paranoid, borderline, schizoid) with moderate to severe functional impairment or acquired brain injury/organic brain dysfunction (e.g., FASD ) or developmental disability/intellectual impairment with moderate to severe impairment. Offenders with the OID indicator “diagnosed as disordered currently” were identified and then rated using the LFI Scale to determine whether they met the criteria. Those who did formed the CMHI comparison group and those who did not were eliminated from further analyses.

Demographic and risk-related information is provided in Table 4. Overall, the majority of offenders in the CDP, CMHS and CMHI comparison group were relatively young, male, and had a low to moderate reintegration potential. However, some differences were observed between the groups. Both the CDP and CMHS groups were significantly younger at the time of index offence than the comparison group (F(2, 224) = 7.10, p < 0.05; CSC, 2008d) and the CDP group was significantly younger at first release than both the CMHS and comparison groups (F(2, 218) = 5.25, p < 0.05). There were a higher proportion of Aboriginal offenders in the CDP group than in the other two groups. The CDP group had the smallest percentage of offenders who had a high reintegration potential and the highest percentage of offenders who had a low reintegration potential. In addition, the offenders who were accepted for CDP services were held at higher levels of security (a variable usually associated with higher risk). The CMHS group had a smaller proportion of offenders with Schedule I offences and significantly longer (F(2, 224) = 6.48, p <.05) index sentences than either the CDP or comparison group. The CDP group had significantly more total prior convictions (F(2, 224) = 3.10, p < 0.05) than the CMHS group and the comparison group (p <.056 on LSD post hoc analyses) and the differences on the number of violent and non-violent convictions approached significance (See Table 4; CSC, 2008d, p. 12.)

Notes:

1. post -hoc comparisons between pairs of groups were completed, where the overall test was significant, to determine whether there were any differences among the three groups. Statistically significant differences between the groups are identified with an a, b at p <.05.
   
2. 1 Security classification and reintegration potential had three level of the variable to be tested for each of the three treatment/comparison groups. In order to minimize the chance of erroneously finding a difference (i.e., family-wise error), comparisons were completed for CDP vs. comparison and CMHS vs. comparison groups only.

Additional information regarding the clinical profiles of the offenders in each of the CDP, CMHS, and comparison groups is summarized in Table 5 (CSC, 2008d). The most common type of mental disorder across all three groups was a major mental disorder, with mood disorder being the most common (34% for CDP ; 32% for CMHS ; and 38% for CMHI ). Offenders with schizophrenia constituted 25%, 15%, and 37% of the CDP, CMHS, and comparison groups, respectively. There was a significant difference between the groups on the LFI,^{Footnote 7} F(2, 224) = 4.75, p < 0.05. Post hoc analyses indicated that the CDP group had significantly higher scores on the LFI scale (M = 5.7, SD = 2.1) than the CMHS (M = 4.8, SD = 2.5) and comparison groups (M = 4.4, SD = 2.3). The CDP group’s mean score of 5.7 corresponds to a moderate to severe functional impairment.

Notes:

1. 1 Diagnosis is based on actual psychiatric diagnosis(es) when available or any documented reporting of diagnosis(es) information in OMS for the comparison group.
2. post -hoc comparisons between pairs of groups were completed, where the overall test was significant, to determine whether there were any differences among the three groups. Statistically significant differences between the groups are identified with an a, b at p <.05.

A total of 519 surveys were completed by staff members who had knowledge and experience working with OMDs.^{Footnote 8} Staff position titles are described in Table 6.

*Note: The Other category includes: Chaplain, Teacher, Warden and Aboriginal Community Liaison Officer. Percentages may not total 100 due to rounding.

The Evaluation Branch teams conducted interviews with offenders in the following locations:

1. St. Johns, Newfoundland (Parole Office and Her Majesty’s Prison);
2. Montreal, Quebec (CCC Martineau);
3. Hamilton, Ontario (Parole Office);
4. Winnipeg, Manitoba (Parole Office, Stony Mountain Institution); and
5. New Westminster and Vancouver, British Columbia (Parole Offices).

A total of 33 interviews were completed across the five regions in Canada: 4 from the Pacific Region, 12 from the Prairie Region, 5 from the Ontario Region, 5 from Quebec Region, and 7 from the Atlantic Region.

The original list of community service providers consisted of 42 independent agencies/organizations. Of these, 35 were contacted, and 7 were unreachable for a variety of reasons (e.g., invalid email addresses). A total of 14 community service providers completed surveys, of which 5 were from one organization. Therefore, a total of 10 independent service organizations returned surveys. Thus overall, feedback was obtained from 24% of the 42 organizations.

Adair, C. E., Wild, T. C., Joyce, A., McDougall, G., Gordon, A., Costigan, N., Mitton, C., & Mingshan., L. (2004). Continuity of Mental Health Services Study of Alberta: A Research Program on Continuity of Mental Health Care. Alberta: Authors.

Arrigona, N., Prescott, L. R., Brown, J. H., & Hook, J. (2003). Implementation evaluation of the Enhanced Mental Health Services Initiative. Austin, TX: Criminal Justice Policy Council

Boudreau, V., Finance Minister (2008). 2008-2009 Budget. Retrieved http://www.gnb.ca/0160/budget/buddoc2008/Speech0809_E.pdf

Brink, J. H., Doherty, D., & Boer, A. (2001). Mental disorder in federal offenders: A Canadian prevalence study. International Journal of Law and Psychiatry, 24, 339-356.

Canadian Public Health Association (2004). A Health care needs assessment of federal inmates in Canada. Canadian Journal of Public Health , 95(S1).

Champagne, D., Turgeon, C., Felizardo, V., & Lutz, J. (2008). Correctional Service of Canada Community Mental Health Initiative: Clinical Discharge Planning and Community Integration Service Guidelines. Ottawa, ON: Correctional Service Canada.

Chandler, D., Spicer, G., Wagner, M., & Hargreaves, W. (1999) Cost-effectiveness of a capitated assertive community treatment program. Psychiatric Rehabilitation Journal. 22(4), 327-336

Corrections and Conditional Release Act, R.S.C. 1992, c.20

Correctional Investigator Canada (2004). Annual Report of the Correctional Investigator: 2003-2004. Ottawa, ON: Author.

Correctional Service of Canada (2002). Commissioner’s Directive 850: Mental Health Services. Ottawa, ON: Author.

Correctional Service of Canada (2006). 2006-2007 Report on Plans and Priorities. Retrieved from http://www.tbs-sct.gc.ca/rpp/0607/CSC-SCC/csc-scc02-eng.asp#_2_6.

Correctional Service of Canada (2007). 2006-2007 Departmental Performance Report. Retrieved from http://www.tbs-sct.gc.ca/dpr-rmr/2006-2007/inst/pen/pen02-eng.asp#s_2-1-2.

Correctional Service of Canada (2008a). Report on Plans and Priorities 2008- 2009. http://www.tbs-sct.gc.ca/rpp/2008-2009/inst/pen/pen00-eng.asp.

Correctional Service Canada (2008b). The Changing Federal Offender population Highlights 2008. Ottawa, ON: Author.

Correctional Service of Canada (2008c). Community Mental Health Initiative (CMHI) Outcome Evaluation: Implementation Report #2. Regional Psychiatric Centre (Prairies): Author.

Correctional Service of Canada (2008d). Community Mental Health Initiative (CMHI) Outcome Evaluation: Preliminary post -Implementation Report. Regional Psychiatric Centre (Prairies): Author.

Correctional Service of Canada Review Panel (2007). A Roadmap to Strengthening Public Safety. Ottawa, ON: Minister of Public Works and Government Services Canada. Retrieved from: http://www.ps-sp.gc.ca/csc-scc/index-eng.aspx

de Kogel, K. (2006) Post Release Supervision of Disordered offenders in the Netherlands. Research and Documentation Center: Ministry of Justice Netherlands. Accessed: October 27, 2008, http://www.publicsafety.gc.ca/prg/cor/_fl/KatydeKogel.ppt#1

Department of Health , National Institute for Mental Health in England (2005). Offender Mental Health Care Pathway. England: Author.

Department of Finance Canada (2007). The Budget plan 2007: Aspire to a stronger, safer, better Canada. Retrieved from http://www.budget.gc.ca/2007/pdf/bp2007e.pdf

Department of Finance Canada (2008). The Budget Plan 2008: Responsible leadership. Retrieved from http://www.budget.gc.ca/2008/pdf/plan-eng.pdf

Government of Canada (2007a). The Budget Speech. Retrieved from http://www.budget.gc.ca/2007/speech/speeche.html

Government of Canada (2007b). Canada’s Performance Report 2006-07: The Government of Canada’s Contribution. Retrieved from: http://www.tbs-sct.gc.ca/reports-rapports/cp-rc/2006-2007/cp-rc00-eng.asp

Government of Newfoundland and Labrador (2008). Budget highlights: Securing a sustainable future. Retrieved from http://www.budget.gov.nl.ca/budget2008/highlights.htm.

Hamid, H., Abanilla, K., Bauta, B. & Huang, K.Y. (2008). Evaluating the WHO Assessment Instrument for Mental Health Systems by comparing mental health policies in four countries. Bulletin of the World Health Organization, 86(6), pp. 467-473.

Jacobs, P., Yim, R., Ohinmaa, A., Eng, K., Dewa, C. S., Bland, R., et al. (2008). Expenditures on Mental Health and Addictions for Canadian Provinces in 2003/2004. The Canadian Journal of Psychiatry, 53(5), 306-313

Kirby, M. J. (chair) (2006). Out of the shadows at last: Transforming mental health , mental illness and addiction services in Canada. Ottawa, Ontario: Standing Senate Committee on Social Affairs, Science and Technology.

Lehman, A. F. (1988). A quality of life interview for the chronically mentally ill. Evaluation and Program Planning, 11, 51-62.

Levin, H. M. & McEwan (2001). Cost-effectiveness analysis, second edition. Thousand Oaks, California: Sage Publications

Lurigio, A.J., Rollins, A., & Fallon (2004). The Effects of Serious Mental Illness of Offender Re-entry. Federal Probation, 68(2), 45-52.

Mental Health Commission of Canada (2008). Mental Health Commission of Canada. Retrieved from http://www.mentalhealtcscommission.ca/index-en.php

Motiuk, L., Boe, R., and Nafekh, M. (2003). The safe return of offenders to the community: A statistical overview. Ottawa: CSC.

Motiuk, L. L., & Porporino, F. J. (1991). The prevalence, nature and severity of mental health problems among federal male inmates in Canadian penitentiaries (No. R-24). Ottawa, ON: Correctional Service Canada.

Nafekh, M., Allegri, N., Fabisiak, A., Batten, D., Stys, Y., Li, H., et al. (under review). Evaluation Report: Correctional Service Canada’s Correctional Programs. Ottawa, ON: CSC.

National Alliance on Mental Health (NAMI) (2008), Assertive Community Treatment (ACT), Accessed: October 22, 2008, http://www.mentalhealtcscommission.ca/English/Pages/AntiStigmaCampaign.aspx

Office of the Correctional Investigator (2007). Annual report of the Office of the Correctional Investigator 2006-2007. Ottawa, ON: Author.

Porporino, F. and Motiuk, L. (1994). The prison careers of offenders with mental disorders (No. R-33). Ottawa, ON: Correctional Service Canada.

Province of Alberta (2008). News Release: Budget 2008: The right plan for today & tomorrow. Retrieved from http://alberta.ca/ACN/200804/233527746F053-9B11-FDEB-08ACEC186300DC0B.html

Province of Manitoba (2008). The 2008 Manitoba Budget Address. Retrieved from http://www.gov.mb.ca/finance/budget08/speech/speech.pdf.

Province of Nova Scotia (2008). Nova Scotia Government Business Plan. Retrieved from http://www.gov.ns.ca/finance/site-finance/media/finance/GovBusPlan.pdf.

Province of Ontario (2008). 2008 Ontario Budget: Growing a stronger Ontario. Retrieved from http://ontariobudget.ca/english/pdf/papers_all.pdf

Roberts, E., Cummings, J., and Nelson, K. (2005) A review of economic evaluations of community mental health care. Medical Care Research and Review. 62:5, 503 543.

Saxena, S., Thornicroft, G., Knapp, M. & Whitefoot, H. (2007). Resources for mental health : Scarcity, inequity, and inefficiency. The Lancet, 370, pp. 878-889.

United Nations (1966). International covenant on economic, social and cultural rights. Adopted and opened for signature, ratification and accession by General Assembly resolution 2200 A (XXI) of 16 December.

United Nations (1977). Standard Minimum Rules for the Treatment of Prisoners. Adopted by the First United Nations Congress on the Prevention of Crime and the Treatment of Offenders, held at Geneva in 1955, and approved by the Economic and Social Council by its resolution 663 C (XXIV) of 31 July 1957 and 2076 (LXII) of 13 May. (http://www.uncschr.ch/html/menu3/b/h_comp34.htm).

United Nations (1988). Body of Principles for the Protection of All Persons under Any Form of Detention or Imprisonment. Adopted by General Assembly resolution 43/173 of 9 December.

United Nations (1990a). Basic Principles for the Treatment of Prisoners. Adopted and proclaimed by General Assembly resolution 45/111 of 14 December.

United Nations (1990b). United Nations Standard Minimum Rules for Non-custodial Measures (The Tokyo Rules). Adopted by General Assembly resolution 45/110 of 14 December

United Nations (1991). Principles for the protection of persons with mental illness and the improvement of mental health care. Adopted by General Assembly resolution 46/119 of 17 December.

Wolff, N. (2005). Community reintegration of prisoners with mental illness: a social investment perspective. International Journal of Law and Psychiatry, 28(1), 43-58

World Health Organization (2001). The World Health report 2001: Mental health : New understanding, new hope. Geneva: Author.

World Health Organization (2003). Mental health legislation & human rights. Geneva: Author.

World Health Organization (2005a). Mental health atlas: 2005. Geneva: Author.

World Health Organization (2005b). WHO resource book on mental health , human rights and legislation: Stop exclusion, dare to care. Retrieved from http://www.who.int/mental_health /policy/who_rb_mnh_hr_leg_FINAL_11_07_05.pdf.

Appendix B: CSC's Community Mental Health Initiative Logic Model

• This Appendix provides information regarding themes from open-ended survey questions. Responses to dichotomous (yes-no) and rating scale questions are reported in the text of the document.
• Percentages were calculated using total number of respondents who had the opportunity to respond to the question.
• Note that total percentages may not sum to 100% since multiple themes were noted by individual respondents.
• Note that only responses to questions where clear themes emerged relevant to the evaluation questions are listed here. In some cases, few responses were generated by interviewees or survey respondents, or no clear themes emerged based on the responses that were generated. Thus, some questions may not be shown here due to lack of clear emerging themes.

Appendix D: Referral Profiles from RPC Implementation Report #2

*Note: The categories of MMD do not add up to 100% due to co-morbidity.

*Includes categories ‘commence as soon as possible ’, ‘commenced’, and ‘plan to commence within one month ’.

Note: The percentage of participants who received a lower score on the post -quiz relative to the pre-quiz was also calculated. The following values correspond to each question respectively: 5.4% (n= 29), 2.8% (n= 16), 3.0% (n= 16), 2.0% (n= 11), 6.5% (n= 35), 0.2% (n= 1), 3.9% (n= 21), 8.9% (n= 48), 5.0% (n= 27), 3.7% (n= 20). It should be noted that there may be several explanations for lower post -quiz scores including the stringent scoring criteria used to maintain consistency in data; the marking scheme was derived from the selection of responses within the participant manual and is not all inclusive. The tests were marked by non-clinical staff at NHQ (and not the trainers), who would not be aware of other “correct” responses discussed during training but not included within the scoring guide.

*p <.01, ** p <.001

Notes:

1. Table reproduced from CSC(2008, October), Community Mental Health Initiative (CMHI) Outcome Evaluations: Preliminary post -Implementation Report, p. 19.
2. a Outcome follow up is based on: (1) CDP group – 1st release following most recent referral to CDP, (2) CMHS group – closest release to referral date, if the release is prior to the referral date, the referral date is used,(3) CMHI -Comp – 1st release following 1st eligibility date between 2003 and 2005.
3. b Mean for time to first suspension and revocation is based on the number of individuals suspended or revoked.
4. c Percentage calculated based on the number of offenders revoked.

Notes:

1. Table reproduced from Table 3 of CSC(2008, October), Community Mental Health Initiative (CMHI) outcome evaluations: Preliminary post -implementation report
2. ‡Outcome follow-up is based on: 1) CDP group – 1st release following most recent referral to CDP, 2) CMHS group – closest release to referral date, if the release is prior to the referral date, the referral date is used, 3) CMHI -Comp – 1st release following 1st eligibility date between 2003 and 2005. †Mean for time to first suspension and revocation is based on the number of individuals suspended or revoked. ††Percentage calculated based on the number of offenders revoked.

Notes:

1. Table reproduced from Table 3 of CSC(2008, October), Community Mental Health Initiative (CMHI) outcome evaluations: Preliminary post -implementation report
2. ‡Outcome follow-up is based on: 1) CDP group – 1st release following most recent referral to CDP, 2) CMHS group – closest release to referral date, if the release is prior to the referral date, the referral date is used,
3. CMHI -Comp – 1st release following 1st eligibility date between 2003 and 2005. †Mean for time to first suspension and revocation is based on the number of individuals suspended or revoked. ††Percentage calculated based on the number of offenders revoked.

Notes:

1. post -hoc analyses using LSD.
2. For suspensions, comparison vs. CMHS group: mean difference = -0.70, standard error = 0.29, p =.017; CDP vs. CMHS group: mean difference = -1.08, standard error = 0.33, p =.001.
3. For revocation, CDP vs. CMHS group: mean difference = -0.77, standard error = 0.42, p =.07.

Note: The omnibus test of model coefficients found that group added significantly to the model after controlling for age at release, functional impairment, and reintegration potential , -2 log likelihood = 1217.795, total model Χ² (6) = 58.151, p <.001. Change in Χ² (2) = 6.756, p =.034.

Table f. Cox regression analysis to examine whether survival is a function of group (CDP and CMHS groups)

Note. a Group: 0 = CMHS and 1 = CDP

The odds ratio of 1.734 indicates that the CDP group is at a 1.734 odds of being suspended compared to the CMHS group. This means that the odds of the CMHS group being suspended compared to the CDP group is 0.577 (i.e., 1/1.734 = 0.577). Therefore, the CMHS group is associated with a 42% (i.e., 1-0.577) reduction in odds of suspension compared to the CDP group.

Note: The omnibus test of model coefficients found that group added significantly to the model after controlling for age at release, functional impairment, and reintegration potential , -2 log likelihood = 763.301, total model Χ² (6) = 35.233, p <.001. Change in Χ² (2) = 13.379, p =.001.

Note. a Group: 0 = CMHS and 1 = CDP.

The odds ratio of 2.476 indicates that the CDP group is at a 2.476 odds of being revoked compared to the CMHS group. This means that the odds of the CMHS group being revoked compared to the CDP group is 0.404(i.e., 1/2.476 = 0.404). Therefore, the CMHS group is associated with a 60% (i.e., 1-0.404) reduction in odds of revocation compared to the CDP group.