Setting Cut-off Scores: A Matter of Judgement
A Matter of Judgement
Cut-off scores are set as part of identifying the best qualified candidates for a position. Since situations vary from one process to another, judgement is required in setting cut-off scores.
This guide will help you in making these judgements by providing information and advice on such matters as the factors to consider in setting cut-off scores; who should set cut-off scores; types of cut-off scores; and steps to follow in setting cut-off scores.
What are cut-off scores?
A cut-off score represents a standard of performance that is set in a selection process with the objective of identifying the best qualified candidate(s). In setting a cut-off score, you are deciding on the level of performance that a candidate must display to be considered further. Often the objective of identifying the best qualified candidate(s) will be achieved most efficiently by setting a standard of performance above just a minimally acceptable level.
Higher scores on selection instruments are usually associated with higher levels of job performance. The expression "more is better" captures this notion. The manager may want to consider only candidates showing higher levels of performance. Whatever the initial preference of the manager, he/she will want to consider several factors before making a cut-off score decision.
Factors to Consider in Setting Cut-Off Scores
In setting a cut-off score, it is crucial to consider the level of competence required to perform the job. Regardless of other factors, no cut-off score should be set below what would correspond to an acceptable level of job performance.
Other aspects for the situations must also be considered in setting appropriate cut-off scores, for example:
- the anticipated or actual size of the participant pool
- the prevailing labour market conditions
- the number of positions to be staffed
If labour market conditions are such that there are many well-qualified candidates, it would be reasonable to require a very high standard of performance.
Because factors other than the required level of competence can influence where the cut- off score is set, there is no such thing as a cut-off score which is applicable to all situations. For instance, even if similar qualifications and levels of job performance are required, the cut-off score could still vary from one selection process to the next because of differences in the availability of well-qualified candidates and in other relevant factors.
Who Should Set Cut-off Scores?
Cut-off scores should be set by people who have a good understanding of the position and the required level of job performance. Awareness of labour market conditions and of similar competitions in the past is a definite asset. Normally, the manager of the position to be staffed is the most appropriate person to set cut-off scores. Nonetheless, the opinion of others knowledgeable in the area is often useful in making the final decision.
Types of Cut-off Scores
Setting cut-off scores may be divided into two major types: performance-related and group-related. These two types of methods and their combination are described below. Additional methods for setting cut-off scores can be found in "Guidelines for Establishing Pass Marks" published by the Public Service Commission.
Performance-related cut-off scores
Performance-related cut-off scores are set by making a judgement about the test score or the level of the qualification that corresponds to the desired level of job performance. The following are examples of this type of cut-off score:
- On a test of typing speed and accuracy: 40 gross words per minute with no more than a 5% error rate.
- On a paper-and-pencil instrument measuring knowledge: 80 correct answers out of 100 questions.
- On a test of lifting strength: lifting a weight of 20 kg.
- On a qualitative rating scale for motivation: a rating of "adequate" or better.
- On a 5-point rating scale for initiative: a rating of 4 or better.
Group-related cut-off scores
Group-related cut-off scores are set relative to the performance of the candidates in a reference group. This reference group may be the present group of candidates, last year's group of applicants, or some other appropriate reference group.
The following are examples of relative cut-off scores:
- On a test of managerial knowledge: a score in the top 5 percent of the applicant group.
- On a written communication test: a score above the incumbent group average.
- On a rating scale for interpersonal relations: the top 10 candidates.
Combination of performance-related and group-related cut-off scores
Both methods may be used in combination in order to select the highest ranking candidates while ensuring that they demonstrate a minimum level of performance on the test. The following are examples of a combination of the two methods:
- On a test of technical knowledge: a score in the top 10 percent of the applicant group in as much as this score is not below a (performance-related) score of 70% correct answers on the test.
- On a rating scale for coping with stress the top 5 candidates who achieve at least a skill rating of 6 out of 10.
Please note that such a combination is implicit when setting group-related cut-off scores with PSC tests, as many of these have minimum cut-off scores. For instance with the General Administration Test, a group-related cut-off score may be expressed as "a score in the top 10 percent of the applicant group", but it follows that candidates scoring below the minimum of 35 over 65 will not be considered as this is the minimum score (see "Test Information Sheet", available at the Personnel Psychology Centre, for a listing of such scores).
Express Cut-off Scores in a Manner Most Useful to your Selection Process
Cut-off scores can be expressed in different ways: relative to test performance, relative to a reference group or to a combination of these two types of methods. Select the most appropriate type given the circumstances. In some instances, it may be useful to set a cut-off score based on performance. Take the example of a manager who is staffing an AS-03 position, Chief of Administrative Services. This position requires that the incumbent demonstrate a thorough knowledge of the organization, objectives and programs of the Department as it relates to the local office administrative functions. The manager determines what knowledge elements are essential for performing at an acceptable level in the position. He then establishes the cut-off score of 70% for the essential components.
In other instances, it may be most appropriate to set a cut-off score relative to a reference group. For example, a manager may need to fill 3 Computer Systems Administration (CS) positions which involve participation in an extensive training program in computer programming. Using a computer programmer aptitude test and anticipating a hundred applicants, the manager decides to set a cut-off score that will screen in the 30 best candidates (i.e. a selection ratio of 10 candidates per position).
Managers sometimes need to choose among the best candidates and make sure that these candidates achieve a minimum level of competence on the test. For example, the manager staffing a Civil Aviation Inspector position (AO-CAI-02) intends to administer a pencil-and-paper screening test designed to assess the "Ability to interpret and apply aeronautics rules and regulations". He anticipates many high calibre applicants and also wants to ensure that applicants who are allowed to proceed further in the competition show a high level of skill in order to protect public safety. He chooses a score in the top 10 percent of the applicant group as long as it does not go below a score of 70% on the test.
In summary, cut-off scores can be expressed in terms of a number of methods; relative to job performance relative to the performance of a reference group or to some combination of the two. The manager will determine the most appropriate type given the circumstances.
A Practical Approach to Setting Cut-off Scores
The following is a description of two basic steps to be used in determining a cut-off score.
Step #1: Consider the required level of job performance
Make a judgement as to the score on the assessment instrument that corresponds to the level of job performance that you consider acceptable (i.e., the lowest level of job performance that you consider satisfactory).
In making this judgement, you consider a variety of information sources; such as the performance of incumbents, the opinion of colleagues who know the positions, etc.
Step #2: Determine whether a Higher Cut-Off Score can be Justified on the Basis of External Factors
Having established the lowest level that you are willing to accept, determine whether a higher cut-off score can be justified on the basis of external factors, such as the pool of candidates, the prevailing labour market conditions, the importance of the work to be done, etc.
Example 1: Knowledge about filing systems
Step #1: To assess a filing qualification, which includes ability to file and knowledge of filing systems, a manager has developed a paper-and-pencil instrument and has set the pass mark at 75%, that is, the minimum level considered satisfactory.
Step #2: However, the manager is expecting a new filing system to be set up in their area and the conversion of old files will be the incumbents' responsibility. Therefore, she may require candidates to possess a better ability to file and more general knowledge of filing systems as well as specific knowledge of the new filing system. Consequently, it was determined that a cut-off score of 85% would be more appropriate in view of this information.
Example 2: Management Abilities
Step #1: In this case, management abilities are assessed by means of a Middle Management Simulation exercise. A five-point rating scale is used to assess the candidates' performance. The five points are "very poor", "poor", "adequate", "very good", and "excellent". Behavioural descriptions are developed for each of these ratings. In this case, the manager decides that the level of performance corresponding to the "adequate" rating will be the lowest level of performance that can be accepted.
Step #2: Based on the results of similar competitions held in the previous year which had made use of the same simulation, the manager expects that most candidates will obtain ratings of "adequate" or better. He/she decides therefore to require a rating of "very good" as your cut-off score for the simulation.
Be prepared to justify the cut-off score
Although you may not always have to justify the cut-off score that you have set, you should be prepared to explain why you set it where you did. It would be a good idea to write down your reasons while they are still fresh in your mind. Being able to give a clear explanation for your decision is also the best way to verify its reasonableness.
Other Issues in Setting Cut-off Scores
Should you allow exceptions?
Situations may arise where you will want to consider making an exception, that is, passing someone who did not meet the cut-off score. The problem in allowing such exceptions is that once the first exception has been made, where do you stop?
Exceptions also leave you open to criticism as to the impartiality of the assessment process. Therefore, it is generally not recommended to adjust cut-off scores after the fact unless there is a defensible rationale for doing so.
Should you allow candidates a second chance?
Should they feel that they cannot perform at their maximum level because of a physical or psychological indisposition, candidates are expected to ask for the test, simulation or interview to be rescheduled before the testing session is over. However, some candidates may ask for another chance after they have been informed of the results. Unless there is a defensible rationale for doing so, the principle of providing one opportunity to all candidates should prevail. Of course, if there is any evidence that testing conditions were unfair, a reassessment of all candidates should be automated.
Report a problem or mistake on this page
- Date modified: