Forensic psychology: Part 2: Chapter 3: Background

Prediction and Classification in Corrections

by Laurence L. Motiuk, Ph.D.^{Footnote 1}

Preamble

The intent of this paper on prediction and classification is to provide a general overview of the following:

1. correlates and predictors of criminal behavior;
2. psychometric properties of reliability, validity and practical utility;
3. predictive accuracy statistics; and
4. ethical/clinical dilemmas of actuarial devices.

A great deal is known about the personal characteristics of offenders, interpersonal influences, situational determinants and environmental conditions which presumably shape their adjustment — both in the institution and the community (Motiuk, 1991).

The following examines the relative efficacy of selected variables in identifying adjustment patterns among prisoners and ex-prisoners in the community.

First, the predictive value of both intake (i.e., admission) and institutional variables are explored with respect to a variety of selected criterion measures of prison adjustment. Then, both intake/release and institutional variables are examined in relation to post-release outcome.

i) Criterion measures of prison adjustment
Most investigations exploring the issue of prison adjustment have evaluated inmates in terms of:

1. Disruptive or rule-breaking behaviour: riots; assaults; homicides; rule infractions; incident reports; misconducts; drug abuse; escapes; transfers; self-mutilations; and suicides.
2. Illness behaviour: illness complaints; sick call attendance; medical diagnoses; medication line attendance; and hospitalizations.
3. Psychophysiological responses: blood pressure; EEG alpha frequency; adrenocortical activity; and sleep disruption.
4. Prosocial behaviour: educational evaluations; work performance ratings; and attitudinal changes.

ii) Factors linked with prison adjustment
Variables that are at least minimally predictive of prison adjustment include:

1. Intake variables: gender; race; age; admission status; criminal history (prior convictions, juvenile criminal history, prior prison, prior misconducts, previous supervision, offence type, sentence length); education level; employment record; financial status; family/ marital background and current situation; living arrangement; criminal associations; substance use/abuse; emotional/personal; and attitude.
2. Institutional variables: court appearances;* institutional programs;* parole notifications; crowding; time served;** and protective custody (* empirical evidence is mixed; = empirical evidence is limited).

iii) Criterion measures of post-release outcome
Traditionally, studies addressing the topic of post release outcome have evaluated ex-prisoners in terms of recidivism measures.

These include: arrest; reconviction; parole violation; return to prison; new offenses; severity of offence; disposition; self-reported offence; death during the commission of offence; absconding; length of time on parole; various probation violations; and length of time to recommitment.

To date, there have been very few attempts to investigate post-release outcome in terms of criterion other than recidivism. One post-release outcome criteria other than recidivism is prosocial lifestyle (post-release employment).

iv) Factors linked with post-release outcome
Variables that are at least minimally predictive of post-release outcome include:

1. Intake/release variables: gender; race; age; release status; criminal history (prior convictions, juvenile criminal history, prior prison record, prior misconducts, previous supervision, offence type, sentence length); education level; employment record; financial status; family/marital background and current situation; living arrangement; criminal associations; substance use/abuse;
2. emotional/personal; and attitude. Institutional variables: visits;* crowding;* time served; and education/vocational training (*= empirical evidence is mixed; = empirical evidence is limited).

In sum, substantive links are to be found in the penological literature between a variety of factors and criterion measures.

i) Assessment tools
Objective assessment instruments for criminal justice decisions have been applied during (see Greenhalgh, Jordan, DeLand & Lund, 1979):

• pre-trial release;
• prosecutorial discretion;
• sentencing;
• institutional custody and transfer;
• parole release; and
• probation/parole supervision.

It is important to note that no scale or classification procedure can compensate for the lack of information on individual offenders.

Offender risk assessment techniques can be categorized into the following (see Cone, 1977):

• natural observation;
• analogue observation;
• self-monitoring;
• interviews;
• self-report; and
• ratings by others.

Current behaviour is a sample of responding to the assessment situation; inferences outside of that situation are not to be made without empirical justification (Nelson & Hayes, 1979). The following are current examples of offender classification instruments used in corrections.

ii) Institutional custody and transfer: The Custody Rating Scale
The Custody Rating Scale is an empirically derived scale used to assist in the initial penitentiary placement of offenders.

It is composed of two separate subscales or dimensions:

1. Institutional Adjustment (IA); and
2. Security Risk (SR).

Based on an offender's criminal history, age, offence type, prison history and personal adjustment, IA and SR score are calculated and the intersection of these scales classifies an offender into one of three custody levels (minimum, medium, maximum).

iii) Parole release: Statistical Information on Recidivism Scale
The Statistical Information on Recidivism Scale (SIR) for adult offenders was designed primarily for parole decision-making in Canada.

The items in the SIR Scale and the constructs they purport to measure were developed by Nuffield and her colleagues (1982). Information is gathered on 15 risk-related factors, using each offender's index offence, prior criminal history record and social history.

In practice, the SIR Scale is used to predict any re-arrest for an indictable offence within three years.

iv) Parole supervision: Community Risk/ Needs Management Scale
As part of the standards (Correctional Service of Canada/National Parole Board) for conditional release supervision, the Correctional Service of Canada's case management staff are required to use a systematic approach to assess the needs of offenders, their risk of re-offending and any other factors which might affect their successful reintegration into the community. In keeping with this standard, the Community Risk/Needs Management Scale is used to capture case-specific information on "criminal history" and a critical set of "needs" for classification while on conditional release (Motiuk 6.1 Porporino, 1989).

1. Criminal history risk assessment. In order to assess risk (of re-offending) systematically and consistently, case managers use the SIR Scale, an extensive review of the official criminal record, the National Parole Board's overall assessment of risk (low versus not low) and their judgment of criminal history risk based on a thorough review of an offender's criminal record.
2. Case needs assessment. The need areas selected for the case needs component of the Community Risk/Needs Management Scale are typical of those included in most other need assessment instruments used in various jurisdictions. A total of twelve need areas are covered:
   • academic/vocational skills;
   • employment pattern;
   • financial management;
   • marital/family relationship;
   • companions/significant others;
   • living arrangements;
   • behavioural/emotional stability;
   • alcohol usage;
   • drug usage;
   • mental ability;
   • health; and
   • attitude.

Although each need area is rated according to specified guidelines, an overall rating of "case needs" is simply the compilation of case manager judgments into three needs level groupings: low, medium and high.

While the Community Risk/Needs Management Scale was clearly intended to focus supervision resources (i.e., frequency of contact between case management and offender) and monitor changes in the offender's behaviour, attitudes and circumstance while under supervision, it essentially puts into practice a simple scheme that allows case managers to classify offenders.

The appropriate frequency of contact for parole supervision is determined by linking the two types of assessments — criminal history risk and case needs — in a matrix format.

In constructing a risk/needs management scale that would attend to the community supervision needs of certain special categories of offenders (i.e., sexual and mentally disordered), two additional special needs categories are included.

A special needs category of "other" is reserved for those who did not meet the aforementioned criteria but are viewed by case managers as meriting an override rating.

The purpose of this section is to present a framework for establishing the quality of objective risk assessment procedures.

It is recommended that traditional psychometric considerations be applied to offender classification systems — more specifically, an approach which emphasizes the psychometric concepts and procedures of reliability, validity, and practical utility (Cone, 1977).

i) Reliability
Reliability estimates are concerned with how much variation in measured phenomenon is attributable to inconsistencies in the measurement of that phenomenon (Anastasi, 1982).

In evaluating offender risk assessment instruments, the issue of whether ratings are stable and consistent arises.

Stability estimates
An accumulated body of evidence shows that over short-, mid-, and long-term periods, offender risk assessment ratings can be quite unstable. It should not be concluded, however, that observed variation in offender risk ratings are attributable to weaknesses in instrumentation. Decreasing stability estimates with time may be a function of the instruments reflecting responses to level/type interventions.

Essentially, offender risk assessment instruments should be stable in the very short-term and capable of reflecting downward shifts in level of risk with time under supervision.

Internal consistency
Reliability estimates based on the number of items and the average correlation of items provide an index of internal consistency for a risk instrument (Anastasi, 1982). Indices of internal consistency indicate the degree to which the items in a test intercorrelate.

For example, if all items in a risk assessment device were to perfectly correlate with one another, then all the items would be measuring the same thing. However, if all items in a risk assessment tool were loosely correlated with another, then they would not be measuring the same thing.

It should be noted that the items contained in an offender risk assessment instrument should be consistent to the extent they reflect sampling of content.

Inter-rater agreement
A good risk assessment tool should be reliable in the sense that two different raters will arrive at the same evaluation of a given offender (Megargee, 1977). Differences in ratings due to administration and scoring has been termed scorer variance or inter-rater reliability. Reliability estimates between raters may be evidenced through the percentage agreement in risk level ratings.

There should be at least 70% to 80% agreement between raters for an offender risk assessment tool to be considered adequate (Kazdin, 1977).

ii) Validity
In examining the psychometric-related differences in validity for offender classification systems, the issue of whether ratings are meaningful necessarily arises.

Objective risk assessment techniques for classifying criminal offenders may be reliable, but not necessarily valid. Instruments are reliable in general with respect to populations, however, they are only valid in relation to specified purposes (Glaser & Bond, 1981).

Completeness
Risk assessment tools should be complete in that the majority of offenders in a given correctional setting are classified.

Content validity
Content validity estimates indicate the extent to which an instrument has adequately covered the behavioural domain being measured. To ensure content validity, there must be both representativeness among items and appropriate procedures used in test construction (Nunnally, 1978).

Criterion-related validity
Criterion-related validity deals with estimates of performance in specified situations (Anastasi, 1982). Subsumed under criterion-related validity are:

1. concurrent validity;
2. correspondence with level/ type interventions; and
3. predictive validity.

1) Concurrent validity: Concurrent validity estimates provide a measure of both predictor and criterion taken at approximately the same time, or the assessment of present status (Ghiselli, Campbell & Zedeck, 1981). In contrast, predictive validity entails the prediction of future outcomes. For example, in concurrent validity, the question of whether an offender is presently "aggressive-psychopathic" arises, not whether the offender will become so at a later date. Significant differences in demographic and behavioural data provide important empirical support for the validity of level/type differentiations within a classification system.

2) Correspondence with level/type interventions: Offender risk assessments should exhibit an acceptable degree of correspondence between level/type ratings and the prescribed security/custody designations. Classification rates and proportional distributions of levels/types across samples are approaches which may be used to establish the appropriateness of offender classifications.

3) Predictive validity: Predictive validity becomes an important issue whenever risk assessment instruments are used for making decisions regarding future outcomes (Anastasi, 1982). The placement of offenders into differential security/custody levels as well as treatment regimes serve to illustrate the kinds of operational problems commonly experienced among correctional managers. Classification instruments are often utilized to predict the in-program (institution, probation or parole) performance and post program (sentence expiration) outcome of offenders. In addition, these assessment tools are also used to decrease errors in forecasting offender behaviour. Ideally, the best risk assessment instruments would be those that produce the maximum of accuracy and efficiency with the minimum of effort and costs.

Construct validity
Construct validity is the degree to which an instrument measures hypothesized constructs or the extent to which a measure is related to other measures in predictable and logical ways (Cronbach & Meehl, 1955; Brown, 1976; Quay, 1983).

Incremental validity
Incremental validity of an instrument is determined by its unique contribution to prediction (Sechrest, 1963). Offender risk assessment tools are deemed valid if they produce an increment in predictive accuracy and efficiency.

Social validity
Offender risk assessment tools are deemed to have social validity (Wolf, 1978) if they are: 1) needed by criminal justice agencies; 2) acceptable to both offenders and correctional administrators; and 3) provide consumer satisfaction.

Social validity raises issues such as costs, staff training, resistance to implementation in the field and consumer satisfaction.

Risk assessment instruments need to be both rapid and cost efficient, effective in correctional management and generalizable to other populations.

iii) Practical utility
Utility is taken to be the benefits attained and costs incurred during utilization. In classifying criminal offenders for differential level/type interventions, the question arises as to which instrument will make more valid predictions (positive and negative) with the least amount of serious errors (false negatives). This formulation may be converted into an index of expected utility (EU) or, in other words, the proportion of valid positives plus valid negatives minus the false negative percentage.

An index of predictive accuracy is the number of valid positives and valid negatives or, stated differently, the proportion of correctly identified recidivists and non-recidivists (Reiss, 195 1).

This index demonstrates those variables which correctly identify offenders who will eventually become recidivists (valid positives) and non-recidivists (valid negatives).

The relationship between prediction and actual outcome for recidivism has been adapted from Fischer (1983) and is shown below.

In addition, the concepts "sensitivity" (true positive rate) and "specificity" (true negative rate) can be used as indices of assessment performance (Lerner et al., 1985; Mossman, 1994, Somoza and Mossman, 1990).

Objective instruments for classifying criminal offenders have depended on interviews, social records and psychological testing as sources of information.

Traditionally, most risk assessment tools have relied heavily on one source of information, almost excluding all others.

One case in point are self-report "inventories." These inventories may be criticized from an offender's standpoint as having little social relevance to their situation (e.g. endorsing items such as "I used to like to play drop the handkerchief'). Self-report inventories also raise ethical and procedural concerns for correctional decision-makers, since offenders are essentially classifying themselves to various security/ custody levels and treatment regimes.

Similarly, there are "scales" which essentially use only one source of data, that is, criminal history. This poses a concern for offenders as they are unable to improve their classification or downgrade their risk status once sentenced to a period of incarceration. Furthermore, the instrument does not provide an assessment of offender needs which are deemed essential for decision-making in the delivery of social services.

Then, there are a variety of "checklists" which combine impressions about behaviour derived from interviews and official records. The format, however, does not afford offenders the opportunity to express both negative and positive circumstances surrounding their designations or labels. In addition, the notion of only one individual making the decision can pose limitations on the objectivity of the conclusions drawn.

In keeping with the view that personal interviews and social records are an acceptable means of obtaining offender information, the best approach would be a risk assessment process which systematically synthesizes information from interviews (with offenders and collaterals), social records, psychological tests, and other professional opinions.

Anastasi, A. (1982). Psychological Testing. Fifth Edition. New York: MacMillan Publishing Co., Inc.

Brown, F. G. (1976). Principles of Educational and Psychological Testing. Second Edition. New York: Holt, Rhinehart, & Winston.

Cone, J. D. (1977). "The relevance of reliability and validity for behavioral assessment,"Behavior Therapy, 8, 411-426.

Cronbach, L. J. & Meehl, P. E. (1955). "Construct validity in psychological tests," Psychogical Bulletin, 52,281-302.

Fischer, D. R. (1983). The Impact of Objective Parole Criteria on Parole Release Rates and Public Protection. Des Moines: Statistical Analysis Center, Iowa Office for Planning and Programming.

Ghiselli, E. E., Campbell, J. P. & Zedeck, S. (1981) .Measurement Theory for the Behavioral Sciences. San Francisco: W.H. Freeman and Company.

Glaser, R., & Bond, L. (1981). "Testing: Concepts, policy, practice, and research," American Psychologist, 36, 997-1000.

Greenhalgh, R. J., Jordan, A., DeLand, G. W. & Lund, L. J. (1979). Classification Instruments for Criminal Justice Decisions. Sacramento: United States Department of Justice, National Institute of Corrections.

Kazdin, A. E. (1977). "Artifact, bias, and complexity of assessment: The ABC's of reliability," Journal of Applied Behavior Analysis, 10, 141-150.

Lerner, J.A., Inui, T.S, Trupin, E.W. & Douglas, E. (1985). "Preschool behavior can predict future psychiatric disorders," Journal of the American Academy of Child Psychiatry, 24,42-48.

Megargee, E. I. (1977). "A new classification system for criminal offenders I: The need for a new classification system," Criminal Justice and Behavior, 4, 107-114.

Mossman, D. (1994). "Further comments on portraying the accuracy of violence prediction," Law and Human Behavior, 18,587-593.

Motiuk, L. L. (1991).Antececlents and Consequences of Prison Adjustment. A Systematic Assessment and Reassessment Approach. Doctoral Dissertation, Carleton University..

Motiuk, L.L. & F.J. Porporino. (1989). Field Test of the Community Risk/Needs Management Scale: A Study of Offenders on Caseload. R-06. Ottawa: Correctional Service of Canada.

Nelson, R. O., & Hayes, S. C. (1979). "Some current dimensions of behavioral assessment," Behavioral Assessment, 1, 1-16.

Nuffield, J. (1982) . Parole Decision-making in Canada: Research Towards Decision Guidelines. Ottawa: Communications Division.

Nunnally, J. C. (1978). Psychometric Theory. New York: McGraw-Hill Book Co.

Quay, H. C. (1983). Technical Manual for the Behavioral Classification System for Adult Offenders. Sacramento: United States Department of Justice, National Institute of Corrections.

Reiss, A. J. (1951). "The accuracy, efficiency, and validity of a prediction instrument,"American Journal of Sociology, 61,532-561.

Sechrest, L. (1963). "Incremental validity: A recommendation," Educational and Psychological Measurement, 23, 153-158.

Somoza, E. & Mossman, D. (1990). "Introduction to neuropsychiatric decision-making: Binary diagnostic tests, " Journal of Neuropsychiatry and Clinical Neurosciences, 2,297-300.

Wolf, M. H. "Social validity: The case for subjective measurement or how applied behavior analysis is finding it's heart," Journal of Applied Behavior Analysis, 1978, 11, 203-214.