Executive Group Position Evaluation Plan - September 2005

Job Evaluation Fundamentals

Overview of the Evaluation Process

Guide Chart for Evaluating Know-How

Practical / Technical / Specialized Know-How

The depth and range of Practical, Technical or Specialized Know-How required in a position is measured on the vertical axis of the Chart.

There are three important concepts to grasp in order to apply the Practical/Technical/Specialized scale correctly:

Managerial and Operational Know-How

The requirement for Managerial and Operational Know-How is measured on the horizontal axis of the Chart.

Managers in Executive Group positions must know how to do such things as plan, organize, motivate, co-ordinate, direct, develop, control, evaluate or check the results of others’ work. This management skill can be required in direct activities (e.g., by line managers), through consultative activities which require thinking like a manager (e.g., by staff specialists), or both (as in positions which manage staff operations).

There are two key concepts to bear in mind when evaluating this sub-factor:

Criticality of Human Relations

Criticality of Human Relations is measured, along with Managerial and Operational Know-How, on the Guide Chart’s horizontal axis. This final Know-How sub-factor integrates the assessment of the practical requirement for using human relations skills into the measurement of job content, that is, the degree to which establishing and maintaining effective interpersonal relationships is central to the position achieving its objectives.

The requirement for using human relations skills on the job is represented by three possible levels. For most Executive positions, because of their size and/or nature, the achievement of objectives truly hinges on the establishment and maintenance of effective interpersonal relations. However, this is not the case for every Executive job.

Therefore, evaluators should not automatically assign level 3 in Criticality to every Executive job. In assessing each Executive position, evaluators must weigh a variety of considerations in making their judgements, such as:

• The degree to which the executive values of leadership and motivation are both integral to the job and highly complex or difficult in nature.
• The importance of “service” and client contact (both internal and external) as integral elements of the job.
• The nature of the client relationship(s).

In assessing the significance of client contact, evaluators should consider such factors as the frequency and nature or intensity of these contacts. There is a significant difference in the Criticality of Human Relations between a case where contact is established simply to gather or exchange information and/or opinions and a case where contact is established and maintained to influence decisions, processes or behaviours which are crucial to the organization successfully achieving its goals.

It is also important to relate the nature of the job’s contacts to its objectives. Evaluators should avoid being misled by statements in job descriptions which ascribe contacts to a job that are not in keeping with its objectives and accountabilities.

There are three levels of Criticality:

Level 1

Common courtesy must be employed and an appropriate working relationship established and maintained with subordinates, colleagues and superiors in order to accomplish the position’s objectives. However, there is no significant need to influence others in carrying out assignments. Interaction with others is generally for the purpose of a straightforward information exchange or seeking instruction or clarification.

Level 2

In dealing with subordinates, colleagues and superiors, and in the course of some contact with clients inside and/or outside government, it is necessary to establish and maintain the kind of relationships that will facilitate the acceptance and utilization of the position’s conclusions, recommendations and advice. In order to achieve desired results, positions have to interact regularly with subordinates, colleagues and superiors and have some contact with clients. The nature of these contacts is such that tact and diplomacy beyond the demands of normal courtesy are required.

Level 3

Successful achievement of the position’s program delivery and/or service and/or advisory objectives hinges on the establishment and maintenance of appropriate interpersonal relationships in dealings with subordinates, colleagues and superiors and in ensuring the provision of service through substantive contact with clients inside or outside government. Skills of persuasiveness or assertiveness as well as sensitivity to the other person’s point of view are essential to ensuring the delivery of service. This involves understanding the other’s point of view, determining whether a behavioural change is warranted and, most importantly, causing such a change to occur through the exercise of interpersonal skills.

The key concept to remember when evaluating the Criticality of Human Relations is:

Combining the Know-How Elements

To this point, three independent decisions regarding Know-How have been made. For example:

The total weight of Know-How is derived from the combination of the three sub-factors. The values assigned to the sub-factors will lead the evaluator to a “cell” on the Chart. This cell will contain three numbers, representing three step values.

For example: the F II 3 cell reads:

• 350
• 400
• 460

Normally, a solid fit on all three sub-factors would lead you to select the middle number in the cell. The final decision about which of these numbers to choose to represent the job’s total Know-How requirement will be based on the degree of confidence in the validity of the cell selected.

Regardless of the number chosen, you should record any shadings in your evaluation (i.e., any “pulls” up or down). You can do this by using an arrow up or down beside the sub-factor, (such as F II 3 ()).

Checking the Step Relationships of a Know-How Evaluation

There are some “rules of thumb” that can assist you in making/validating your judgments. It is important to bear in mind that these are “rules of thumb,” not hard-and-fast rules. They should not be used as a substitute for thorough analysis of the job and interpretation of the Guide Charts.

As a “rule of thumb,” when you are considering a hierarchy of jobs in a job family, technical ladder or reporting structure, the number of steps in the Know-How score can give some insight into the vertical structure of the hierarchy:

One-step difference e.g., 460 to 528

A “one-step” difference generally indicates a point of compression in the structure, giving reason to question the need for the number of organizational “layers” found. For example: one-over-one situations where the subordinate’s job is virtually a replica of the superior’s job or is “just perceptibly” different.

Two-step difference e.g., 460 to 608

This is the typical or logical relationship/vertical distance in a reporting sequence.

Three-step difference e.g., 460 to 700

Three steps between levels are characteristic of reporting relationships in organizations with a broad span of control.

Four-step difference e.g., 460 to 800

This represents a significant difference in terms of job content on the Know-How factor, suggesting that a level may be missing in the organizational structure. Care should be taken to ensure that the void exists in reality and that it is not the result of an evaluation error.

Validating Against the Benchmarks

The evaluation of the Know-How factor should always make sense within the continuum of Executive Group positions. There is a need to ensure that the logic of that continuum remains intact over time.

The method for validating against the continuum is to “prove” the evaluation by finding several comparable reference positions from the standardized Benchmark positions. This is the key test of the validity of an evaluation. Generally, the Benchmark validation step is done after the position has been evaluated against all three factors. The process is outlined in Appendix C.

Guide Chart for Evaluating Problem Solving / Thinking

Problem Solving / Thinking Definition:

Problem Solving/Thinking is the original, self starting thinking required by the job to: (1) identify, (2) define, and (3) resolve a problem. "You think with what you know" - this is true of even the most creative work. The raw material of any thinking is knowledge of facts, principles and means. Ideas are put together from something already there. Therefore, Thinking is treated as a percentage utilization of Know How. There are two components :

Note: Each of the two components is marked with a certain number of dots. The number of dots corresponds to a section on the Guide Chart for Evaluating Problem Solving / Thinking.

The Thinking Environment:

The extent to which assistance or guidance is available from others or from past practice or precedents and the degree to which the position is required to identify situations where direction or precedents are not applicable. How well/poorly defined is the problem, issue, etc.?

The Thinking Challenge:

The novelty and complexity of the thinking to be done and the time pressures within which the thinking must be done.

Measuring Problem Solving / Thinking:

Problem Solving/Thinking measures the intensity of the mental process which employs Know How in analyzing, evaluating, creating, reasoning, arriving at and making conclusions. To the extent that Problem Solving/Thinking is circumscribed by standards, covered by precedents, or referred to others, the scope of the Problem Solving/Thinking is diminished, and the emphasis correspondingly is on Know How.

N.B.

The evaluation of problem solving/thinking should be made without reference to the job’s freedom to make decisions or take action; the scope and nature of the job’s decisions are measured on the accountability/decision making chart.

Click here for Guide Chart for Evaluating Problem Solving / Thinking

Freedom to Act

Freedom to Act is the most important of the three Accountability / Decision Making sub-factors. By examining the nature and extent of the controls - or the lack of the controls - that surround the job, it directly addresses the question of the job’s freedom to take action or implement decisions. Because of its importance, this sub-factor carries the most quantitative weight in the evaluation of a position’s total Accountability / Decision Making.

The controls placed on the position’s Freedom to Act can be supervisory or procedural, or both. A key concept to keep in mind when considering Freedom to Act is:

Since controls tend to diminish as you rise in the organization, Freedom to Act increases with organizational rank. However, while it is true that no job can have as much Freedom to Act as its superior, the evaluator should be wary of automatic slotting according to organization level alone.

Here are some broad guidelines that can help in assessing Freedom to Act:

• At the D level, positions are relatively free to decide how to achieve predetermined results under some direction from superior management. Positions at this level are subject to managerial approval of tactical objectives and periodic evaluation of results, generally on a quarterly or annual basis.
• At the E and F level, positions are relatively free to determine what the general results are to be. Managerial direction will be general in nature. Assessment of end results must be viewed over longer time spans (e.g., six months to a year or longer).
• At the G level, the what is communicated only in very general terms. Positions become subject to guidance rather than direction or control. Any job evaluated here is subject only to broad policy.

Impact

While the explanation of how to evaluate Impact and Magnitude is presented sequentially, these two sub-factors should always be considered together.

The Impact sub-factor measures the directness of the position’s effect on end results. The Impact levels are as follows:

I - Indirect:

The position provides information, recording or other supportive services for use by others. Activities are noticeably removed from final decisions / end results. The position’s contribution is modified by or merged with other support before the end result stage.

C - Contributory:

The position provides interpretative, advisory or facilitating services for use by others or by a team in taking action. The position’s advice and counsel are influential and closely related to actions or decisions made by others. Such an Impact is commonly found in staff or support functions which significantly influence decisions related to various units or programs. For example:

• A senior labour relations specialist makes recommendations and administers policies and practices which affect the use of the unionized human resources of a unit. Note: The Magnitude of these resources might be represented by the human resources costs (payroll) of the unit.

S - Shared:

The position is jointly accountable with others (usually one other) for taking action and exercising a controlling Impact on end results. Positions with this type of Impact have noticeably more direct control over action than positions evaluated at the Contributory level, but do not have total control over all the variables in determining the end result. In addition, Shared Accountability can be used to indicate that a position makes an extremely strong contribution to end results (stronger than its peers) but does not have a Primary Impact.

A basic rule is that Shared Impact does not exist vertically in an organization

(i.e., between superior and subordinate). However, Shared Impact can exist between peer jobs within the same organization or with a position from outside the organizational unit. Shared Impact suggests a degree of partnership in, or joint Accountability for, the total result. In this way it differs from Contributory Impact, where the position is only accountable for a portion of the end result.

• The departmental Project Manager could be considered to have a Shared Impact on all design and construction activities carried out by Public Works and Government Services Canada in the construction of a major facility.

Note: There are few situations in the Public Service where true shared accountability exists.

P - Primary:

The position has controlling Impact on end results, and the accountability of others is subordinate. Such an Impact is commonly found in managerial positions which have line accountability for key end result areas, be they large or small. For example:

• The Director of a research unit may have Primary Impact upon the research activities done by all sections of the unit. A subordinate Manager within the unit may be accountable for the research activities in a section of the unit. Both positions could be evaluated at the Primary level, but the level of Magnitude (the size of the unit or function or activity) would vary.

Magnitude

Magnitude measures the size of the area affected by a position. While it does give an indication of the “weight” to be assigned to the position, it is the least important of the three sub-factors used to determine the overall Accountability / Decision Making evaluation.

For measuring Executive Group positions, a common, quantifiable means or “proxy” for representing the diverse units, functions and programs that could be affected by the position must be identified. Dollars have proven to be the most widely applicable “proxy” for measuring the Magnitude to be assigned to a given position.

However, to make a logical, rational determination of Magnitude, the evaluator must remember that dollars are simply a proxy, not an absolute measure. The key concept behind this is:

Dollar ranges on the Guide Chart correspond to varying alignments of Magnitude/Impact/Freedom to Act. The Benchmarks provide evaluators with a wide variety of examples of different levels of Magnitude to assist them in the determination of this sub-factor.

The Magnitude continuum has seven degrees, from “Very Small” to “Largest.” These headings provide a rough idea of the appropriate Magnitude for the subject position. References to the appropriate Benchmarks will help refine this initial determination. In this way, evaluators can arrive at a reasonable determination of Magnitude and avoid jumping immediately to a premature consideration of budget dollars.

Evaluators should use the following process for applying the proxy to establish the appropriate Magnitude:

• Determine and describe (in words) what part(s) and/or function(s) of the organization the job affects, and the nature of the job’s effect on each of them. For example: the position controls a Branch.
• Once the part(s) and/or function(s) most appropriate to the job have been identified, think about the relative size of the part(s) or function(s) under consideration and describe these in words. For example: the Branch is very small, small, large, very large, etc.
• Once these relationships have been articulated, verify them and the “size” selected for the job against the dimensions of the Benchmark positions.

Choosing the Correct Impact / Magnitude Combination

An evaluation score may differ depending on the combination of Impact and Magnitude used. For instance:

• A function head (e.g., a Director General of Human Resources) may be seen to have a Contributory Impact on the operations of the Department or a Primary Impact on the operations of the Human Resources Branch.
• Very often the point totals available in these two slots will be the same. Where they are not the same, it is advisable to use the higher score to properly reflect the full job size as long as you are confident that your reasoning is correct.

The key is to find the combination of Impact and Magnitude that results in the highest legitimate evaluation. This is because it is vital to get the fullest, most complete measure of the position for these two sub-factors so as to properly reflect the job size. Table 1 on the following pages provides some guidelines for evaluating certain types of expenditures when these are used as the proxy.

Combining the Accountability / Decision Making Sub-Factors

The result of evaluating the three sub-factors is that the evaluation falls within a cell with three possible point values, each representing one step up in size.

Example: A cell with three possible point values, each representing one step up in size

The number chosen from the cell will depend upon your assessment of the relative strength of the job’s fit to sub-factor definitions. However, unlike the sub-factors of Know-How and Problem Solving/Thinking (which have a more or less equal weight in determining the factor’s score), in the case of Accountability / Decision Making, the fit of the Freedom to Act sub-factor is the most important one to consider.

There is a propensity for evaluators to forget this hierarchy of values and use Magnitude to drive the selection of the number in the cell. This is inappropriate. Considerations of Impact and Magnitude (the least important sub-factor) should be used to confirm the direction of the overall evaluation, as determined by considerations of Freedom to Act.

Checking the Accountability / Decision Making Evaluation

Validating Against the Benchmarks

The most important test of the validity of the evaluation is finding comparable reference evaluations in the standardized continuum of the Benchmarks, as outlined in Appendix C.

Using the Benchmarks

The Benchmark reference positions have two critical roles to play in the job evaluation process:

• They provide the necessary discipline of a constant set of reference points.

The Benchmark evaluations have been thoroughly checked to ensure that the job evaluation method was applied consistently and appropriately. As a result, they provide a constant standard against which to evaluate positions, making them the key tool for ensuring consistency in the application of the Guide Chart methodology, over time, throughout the federal government.

• They allow for flexibility within a disciplined framework.

Given the number and complexity of Executive Group positions to be evaluated, it would be both impossible and counter-productive to attempt to provide hard-and-fast solutions for every possible situation. By providing a sufficient number of constant Benchmark positions, but allowing evaluators to use their common sense in using them, both flexibility and discipline can be built into the process.

Copies of the Benchmark reference positions are found in Appendix C.

Process for Selecting Suitable Benchmarks

Sample Validations Using Benchmarks

Below are two examples of the process for using Benchmarks. Descriptive statements from the job description of the subject position and the sample Benchmarks are shown beneath. These statements are not the only relevant differentiating considerations. They are presented simply to illustrate the thinking process involved in using Benchmarks.

Sample Validation: Example B

Subject Position: Director Financial Policies, Systems and Accounting

Evaluation :

	Factors	Points
Know-How	F III 3	460
Problem Solving / Thinking	E 4 (50%)	230
Accountability / Decision Making	E 5 C	264
Total		954

Benchmarks Selected

Director General, Resource Management (6-O-2)

	Factors	Points
Know-How	G III 3	608
Problem Solving / Thinking	F 4 (57%)	350
Accountability / Decision Making	F 2 P	350
Total		1308

Director, Financial Management (5-M-1)

	Factors	Points
Know-How	F III 3	528
Problem Solving / Thinking	E 4 (50%)	264
Accountability / Decision Making	E 5 C	264
Total		1056

Regional Manager, Finance & Administration (4-O-3)

	Factors	Points
Know-How	F II 3	460
Problem Solving / Thinking	E 4 (43%)	200
Accountability / Decision Making	E 2 P	230
Total		890

Benchmark Comparisons Example B

	Major Challenge
Subject Position	A major challenge for the Director is to provide management with an integrated accounting and financial systems operation that has effective accounting mechanisms and control, and at the same time meets the requirements of Parliament and Central Agencies with respect to departmental initiatives in the areas of Economic Development and Trust and Loan Fund Management.
Benchmark 6-O-2	The position is expected to provide corporate leadership in the development, dissemination and implementation of policies, systems, processes, services and control mechanisms for the seamless implementation of the management reform underpinning the modern comptrollership and modern management functions affecting the management of the Agency’s financial, material and human resources.
Benchmark 5-M-1	The position’s challenge is to ensure the continued integrity of the financial management data, systems and procedures while ensuring the successful implementation of significant changes in financial management practices, information systems and accounting procedures arising out of the Modern Comptrollership Initiative and other financial modernization initiatives.
Benchmark 4-O-3	The position guarantees the integrity and reliability of budgetary control and reporting systems in the Region to ensure that probity and prudence are duly exercised in the handling of regional resources.

Example B Explanation: The Know-How Factor

Subject Job	Director General, Resource Management (6-O-2)	Director, Financial Management (5-M-1)	Regional Manager, Fin. & Admin. (4-O-3)
F III3 460	G III 3 608	F III3 528	F II3 460

1. While the subject position and Benchmark 5-M-1 are both rated F III 3, the Benchmark is a senior full-time financial officer and is thus seen as requiring higher degrees of professional and operational knowledge. Benchmark 6-O-2 reflects a degree of mastery required to provide executive direction at the first managerial level for the management of all the infrastructure support functions across the department. The subject position and Benchmark 4-O-3 reflect the depth of professional expertise required to provide executive level support in the delivery of the department’s programs.
2. The managerial expertise required for the subject position is less than that required for Benchmarks 6-O-2 and 5-M-1, which have department-wide functional responsibilities. The department-wide role of the subject position and the focus of its end results, although rated higher, are weighed as comparable with the regional focus of Benchmark 4-O-3 and receive the same numerical value.

Example B Explanation: The Problem Solving / Thinking Factor

Subject Job	Director General, Resource Management (6-O-2)	Director, Financial Management (5-M-1)	Regional Manager, Fin. & Admin. (4-O-3)
E 4 (50%) 230	F 4 (57%) 350	E 4 (50%) 230	E 4 (43%) 200

1. As the senior functional position in the department, Benchmark 6-O-2 is clearly involved in more demanding, extensive and varied issues than are the subject position or the other two Benchmark positions.
2. The departmental functional roles of the subject position and of Benchmark 5-M-1 are viewed as requiring a higher degree of innovation, creativity and integration than Benchmark 4-O-3.

Benchmark Comparisons Example B

Dimensions (Constant Dollars)

Subject Position	Division Employees:	50
	Division Operating Budget:	$500,000
	Departmental Operating Budget	$885 million
Benchmark 6-O-2	Branch Employees:	30
	Department Employees:	380
	Branch Operating Budget:	$800,000
	Agency Operating Budget:	$6.7 million
Benchmark 5-M-1	Division Employees:	48
	Division Operating Budget:	$484,000
	Department Operating Budget:	$177 million
Benchmark 4-O-3	Region Employees:	1,370
	Division Employees:	133
	Region Operating Budget:	$49 million
	Division Salary Budget:	$866,000

Example B Explanation: The Accountability / Decision Making Factor

Subject Job	Director General, Resource Management (6-O-2)	Director, Financial Management (5-M-1)	Regional Manager, Fin. & Admin. (4-O-3)
E 5 C 264	F 2 P 350	E 5 C 264	E 2 P 230

1. The ratings reflect the higher level of empowerment and contribution of Benchmark 6-O-2 in the achievement of expected end results and departmental objectives.
2. Note that while the magnitude levels of the subject position and Benchmark 5-M-1 could warrant a difference in their respective numerical ratings, their contribution to the achievement of the department’s goals is viewed overall as being of equal value.

			Likely Step Differences
Designation	Unit Diagram	Common Characteristics	Know How	% Problem Solving	Account-ability
“Normal” Superior/Subordinate Relationship		reasonable span of control Clear channels of communication Balance between subordinates Management delegation	2	1	3
Lean Staff		Broad span of control Communication processes fuzzy Imbalance between subordinates positions Management and decision-making centralized	3	1 or 2	3 to 5
Missing Level		Large span of control Communication of tasks only Large differentials in subordinates position Management and decision-making highly centralized	4	2	5 to 7
One Over One		One subordinate Superior and subordinate function as a team Usually a temporary “grooming” position for subordinate prior to taking over the top position Appropriate where critically of top job dictates (i.e. CEO) or where there is a need for a split between external focus and internal focus	1	1	2

Organization Check

A crucial test of the validity of the evaluation is whether it fits with the evaluations for other positions in the unit. This means that when you isolate each factor, the step differences between the subject job and the supervisor, peer and subordinate positions for each factor all make sense.

A common evaluation error is over-emphasizing the differences between peer positions and under-emphasizing the differences between superior and subordinate. Note, however, that there are no rules for determining the proper relationship between levels in an organization. Each case must be assessed on its own.

• a variety of superior/subordinate relationships that can make sense
• the importance of looking at the reality of the actual departmental structure when testing the validity of a new evaluation

In the final analysis, as throughout the evaluation process, informed common sense should be the tool for making and checking all judgments.

Inclusions

Notwithstanding the generality of the foregoing, it includes positions that have, as their primary purpose, responsibility for one or more of the following activities:

1. Managing programs authorized by an Act of Parliament, or an Order-in-Council, or major or significant functions or elements of such programs;
2. Managing substantial scientific or professional activities;
3. Providing recommendations on the development of significant policies, programs or scientific, professional or technical activities; and
4. Exercising a primary influence over the development of policies or programs for the use of human, financial or material resources in one or more major organizational units or program activities in the Public Service.

Exclusions

Positions excluded from the Executive Group are those whose primary purpose is included in the definition of any other group.

Classification Levels: Executive Group

The total points assigned through the evaluation process will determine the classification level for a newly evaluated position. Positions within the assigned point bands fall into compensation levels from EX-1 to EX-5. The bands are as follows:

Government of Canada Executive Group