Guide to Mitigating Biases and Barriers in Assessment
Table of contents
- Purpose
- Definitions: biases and barriers that disadvantage persons belonging to any equity-seeking group
- Guiding principles
- Principle 1: Awareness of self and others
- Principle 2: Accessibility
- Principle 3: Validity and reliability
- Principle 4: Standardization
- Principle 5: Transparency
- Principle 6: Fairness
- How to conduct an evaluation of biases and barriers
- Documentation
- Your feedback
Description
What's presented in this video:
00:19 – A mock interview scenario
00:45 – An overview of subsection 36(2) of the Public Service Employment Act
01:15 – A summary of the PSC's guide and tool to conduct an evaluation
02:04 – The PSC's guiding principles
02:37 – Awareness of self and others
04:06 – Evaluation of biases and barriers in 3 simple steps
04:39 – Step 1: Describe the interview method
05:27 – Steps 2 and 3: Identify potential biases and barriers, and learn how to take action to remove them or mitigate their impact
- 05:47 – Designing the interview
- 07:30 – Developing the interview questions
- 09:57 – Planning the administration procedures
- 11:44 – Planning the scoring method
- 13:16 – Planning the assessment board
14:06 – Documenting the method used, the biases and barriers identified and the actions taken
15:01 – Moving forward with interviews
16:12 – Key take-aways
Transcript
How to conduct an evaluation of biases and barriers of an assessment method as it relates to subsection 36(2) of the Public Service Employment Act (PSEA). Prepared by the Personnel Psychology Centre of the Public Service Commission of Canada (the PSC).
Sasha is a subdelegated hiring manager. She has a vacant position to staff and will use an external appointment process. In collaboration with a human resources (HR) specialist, she drafted an inclusive Statement of Merit Criteria and prepared the screening method. Right now, she is planning the interview and will evaluate it for possible biases or barriers that disadvantage members of equity-seeking groups.
Subsection 36(2) of the PSEA states that before an assessment method is used, an evaluation must be done to identify potential biases or barriers that disadvantage members of equity-seeking groups, and reasonable efforts must be made to remove them or mitigate their impact. The authority of subsection 36(2) is delegated to and within organizations, and may vary from one organization to another.
Sasha will evaluate her interview tool using the PSC Guide and Tool for Mitigating Biases and Barriers in Assessment.
The Guide describes the guiding principles and the 3 steps to follow while the Tool helps the user complete the 3 steps. The tool includes descriptions of over 40 potential biases and barriers and related mitigation strategies. Some of these potential biases and barriers may apply to any assessment method and others may apply only to one specific type of assessment tool, such as an interview.
Sasha wants to design the interview in a way that is fair for everyone. She will be proactive and will apply some of the mitigation strategies from the PSC tool as she develops the interview instead of doing it later in the process.
Sasha will follow the PSC's guiding principles closely. She will maintain a constant awareness of self and others and reflect on the reasons behind her decisions. She will make the interview accessible to various people and will make sure it measures the qualifications accurately and consistently. Standardization can help with this step when done properly. She will also be transparent and proactive. Ultimately, she wants to give everyone a fair chance to fully demonstrate their qualifications.
Sasha takes a moment to think of the first guiding principle: awareness of self and others. She reflects on her own experiences. She can remember her first government interview as a candidate. She was rather confused by all the steps and felt overwhelmed before the interview even began. At one point, she asked to clarify a question that seemed open to interpretation. The assessment board was reluctant and would not elaborate. She thought their attitude was because of her accent, and had the impression that they didn't want to hire her. She felt discouraged and gave short, incomplete answers to the remaining questions.
Sasha also remembers when, later as a hiring manager, she met a candidate who had attended the same high school as she had. They chatted a lot, sharing anecdotes about the school and community. She realizes that she easily engages with people who are similar to her and that she might be less prone to building the same rapport with those who are not. This can lead to a common rating error called affinity bias.
Sasha knows that everyone is biased in some way because it is how our brains work. They make shortcuts to process information, but these shortcuts can be prone to error and have a real impact on assessments. She will remember this when planning the interview and will integrate different perspectives throughout the process.
Sasha will complete the evaluation of biases and barriers in 3 simple steps found in the PSC guide.
Step 1 – she will write a description of the interview and how it will be administered. Step 2 – she will identify potential biases and barriers that may disadvantage persons belonging to any equity-seeking group.
Step 3 – she will take action to remove them or mitigate their impact.
She will keep track of what she has done in the PSC tool, which can also serve as documentation.
Sasha starts with Step 1, reflecting on her approach. She needs to assess 6 qualifications and plans to write 6 interview questions, one per qualification. She will have 4 behavioural questions which ask about past behaviours and 2 situational questions which ask about hypothetical situations. She will draw inspiration from past interviews.
The interview will last 1 hour and will be administered in person by an assessment board of 3 people. The board members will be instructed to assign ratings using a 5-point scale from "does not meet" to "exceeds expectations".
Sasha writes these key points in the PSC tool. She will update the interview description if her approach changes.
Sasha has described the interview method and will continue with Steps 2 and 3. She will identify potential biases and barriers and take action to remove them or mitigate their impact. She will start by considering aspects of the design, including the questions themselves, before looking at administration and scoring.
One of the first mitigation strategies in the PSC tool is to "Build diversity and inclusion into your plan and consult others". Doing this helps ensure the interview assesses qualifications that are related to the job and that reflect different ways of working effectively.
Sasha speaks with an HR specialist, who can give her advice. Together, they exchange ideas on best practices in interviewing and discuss ways to be more inclusive and flexible.
The HR specialist points out that some people may need more time to prepare answers that fully demonstrate their qualifications.
He asks whether the job often requires working under pressure or within tight deadlines. Sasha replies that this is not a requirement of the job. In that case, he suggests adding more time and including a preparation period. Sasha agrees with this approach. This will help ensure that the skills needed to respond during the interview are related to the job.
They also discuss various ways that people might demonstrate the qualifications on the job so these ideas can be considered when preparing the questions and scoring key.
Sasha examined the job requirements, added a preparation period and is considering different ways of working. This helps her mitigate the impact of 2 potential biases listed in the PSC tool: the qualifications are narrow or beyond what is required for the job; and the assessment captures only one way of demonstrating the qualifications.
Sasha feels confident about her plan and will now draft the interview questions.
Sasha is working on the interview question to assess Initiative. She uses a question from a past interview. The question is "Tell me about a time when you took initiative to resolve a policy issue on your own."
Reading the PSC tool, she realizes there are potential problems with this question. First, it may not accurately measure the qualification. She wants someone who can anticipate challenges and act without being prompted. This might be unclear when saying "took initiative". Also, the person could have shown initiative in many different contexts, but not policy. As a mitigation strategy, she will remove the word "policy" and will clearly define "initiative" in the definitions given to candidates and the scoring key.
She doesn't find any more potential biases in the section common to many assessment methods, so she looks at the items specific to interviews in the PSC Tool. The first potential bias for interviews is "too much emphasis on past work behaviours". As a mitigation strategy, she will encourage people to consider other past experiences by telling them the experience could come from various contexts such as work, volunteering or school.
There is also a risk of gathering incomplete information since there are no planned follow-up questions. Sasha will add follow-up questions so that the people responding can give more complete answers. These will include "who did you involve?", "what was the situation?" and "what was the outcome?".
Sasha asks a fellow manager to review the question. Her colleague points out that the term "on your own" may not be inclusive because it emphasizes individual action, and some people might value collective action and outcomes. She suggests removing "on your own" and adding a follow up question asking, "what steps did you take and why". She also offers to help Sasha draft a situational question if she wanted to give people the chance to show initiative without being restricted to past experience.
Sasha's revised interview question is now "Tell me about a time when you took initiative to resolve an issue" and includes 4 follow-up questions. She completes the same exercise for the other interview questions.
Once that is done, Sasha starts reviewing the administration procedures.
To limit the risk of inconsistent administration, Sasha and the HR specialist have prepared administration instructions, procedures, and best practices for the assessment board and test administrator based on past processes. This includes a script with introductory remarks to help build rapport and put people at ease.
Sasha knows the risks of having limited instructions for candidates. She does not want people to feel the same way she did during her first government interview, so she will be sure to give clear instructions about what is being assessed, the number of questions, the number of board members, the note taking procedure, the interview duration, etc.
Candidates will also be informed of the room set-up and given the opportunity to request adjustments if they prefer a different seating arrangement to being seated in front of a panel. This will help mitigate the potential bias of rigid seating structure and effects of implied hierarchy.
After some reflection, Sasha decides to give the option for virtual interviews because this may help reduce potential barriers to in-person administration. She updates the test material and administration procedures accordingly.
Finally, to mitigate the risk of insufficient time, Sasha will allot a separate preparation period where candidates read the 6 questions and have 40 minutes to prepare their answers. The board will then greet them, spend 5 to 10 minutes on introductory remarks, then give 40 minutes to answer the interview questions. Additional time would be reserved at the end for closing the interview.
Now that Sasha has the administration organized, she reviews the 5-point scale and scoring procedures.
She clearly defines what is expected at each level of performance in the 5-point scale. She asks the board members about different ways people could respond to the questions. She reminds them to consider all answers objectively, even if an approach is unexpected or not covered in the rating booklet. This helps her mitigate the impact of narrow rating criteria.
Sasha has prepared a rating booklet to remove potential bias related to a lack of structured rating. The booklet includes descriptions of the qualifications, the scoring key, examples of effective responses and space for taking notes. It also details the rating procedures such as rating each qualification individually before coming to a consensus, considering evidence for a qualification across all questions asked, and justifying each rating based on the notes taken.
Sasha will mitigate the impact of insufficient note taking by asking each board member to take detailed notes of what the person says rather than paraphrasing.
She will also address common rating errors by asking the board to rate responses shortly after the interview and to take breaks to reduce fatigue-related rating errors. She will also schedule only 1 or 2 interviews per day.
The structured rating will help ensure the validity and reliability of the results.
Sasha is now ready to select the board members. She wants to choose more than 3 people so there are always enough board members per candidate. She's thinking of inviting 2 managers and 2 job incumbents from her division, given their experience, familiarity with the job and availability.
The HR specialist reminds her of the bias in the PSC tool related to an assessment board being homogeneous or not properly trained. He can help her identify other qualified people by reaching out to their department's committee on equity, diversity and inclusion. Sasha agrees and says she will ask everyone to follow training on common rating errors, unconscious bias and cultural sensitivity. She will also check that ratings are consistent by reviewing a random selection of responses.
Sasha and the HR specialist prepare the documentation, following their departmental staffing policies. Sasha saves a copy of the interview material. She makes sure that potential biases or barriers identified are checked off in the PSC tool, including the mitigation strategies applied. She takes note of any additional actions taken and the different people consulted throughout the process.
The purpose of documentation is to show what method was used to evaluate the interview, and what potential biases or barriers were identified, including the actions taken to remove them or mitigate their impact.
The PSC does not prescribe a specific format for how to document the evaluation. Some organizations may choose to list observations and actions in a separate document, write brief narrative descriptions, or apply another format.
Sasha is ready to move forward with the interviews! She will keep the guiding principles in mind, committing to being transparent and fair when communicating with candidates. Sasha and the board will follow procedures as planned to support the standardization, validity, and reliability of the interview, while offering flexibility and accommodations to meet individual needs.
As a reminder, accommodation focuses on the individual needs of a person and can be requested at any point of the appointment process, while the evaluation of the assessment methods must occur before their use.
Despite her best efforts, an error or bias may still occur during the interviews. She will be sure to document the information and will work closely with HR to consider the situation objectively. She will carefully analyze any new information or seek feedback from others about the fairness of the interview before taking action.
When Sasha has completed the interviews, she will ask for feedback from candidates, board members and HR to identify future areas of improvement.
The appointment process is now over! Sasha has hired a qualified person who will be a great asset and fit for the team. She is content with the process and shares the following key take-aways with her colleagues:
It's not as difficult as she had imagined. Many of the strategies are easy to implement or she was already doing them.
A few changes can have a significant impact. For example, giving preparation time and thinking ahead about different candidate needs and experiences.
Using the PSC tool early on made it easier to implement good strategies and saved her time later.
Collaboration is key. HR was a vital support, and she was encouraged to see the knowledge and experience of others in her department and networks.
This will only get easier over time.
Overall, it was a positive experience. She learned about herself and her own potential biases. Speaking with others helped her integrate diverse perspectives and ultimately led to a fairer assessment.
She will continue learning and staying connected with others in support of inclusive staffing practices.
Thank you for following Sasha's story! See the link below to explore additional tools and resources on the Amendments to the Public Service Employment Act, and be sure to add it to your Favorites!
Amendments to the Public Service Employment Act (accessible only on the Government of Canada network)
How to conduct an evaluation of biases and barriers of an assessment method as
it relates to subsection 36(2) of the Public Service Employment Act (PSEA). Prepared by the
Personnel Psychology Centre of the Public Service Commission of Canada (the PSC).
Sasha is a subdelegated hiring manager. She has a vacant position to staff and will use an external
appointment process. In collaboration with a human resources (HR) specialist,
she drafted an inclusive Statement of Merit Criteria and prepared the screening method.
Right now, she is planning the interview and will evaluate it for possible biases or barriers that
disadvantage members of equity-seeking groups. Subsection 36(2) of the PSEA states that before an
assessment method is used, an evaluation must be done to identify potential biases or barriers that
disadvantage members of equity-seeking groups, and reasonable efforts must be made to remove them or
mitigate their impact. The authority of subsection 36(2) is delegated to and within organizations,
and may vary from one organization to another. Sasha will evaluate her interview tool using
the PSC Guide and Tool for Mitigating Biases and Barriers in Assessment.
The Guide describes the guiding principles and the 3 steps to follow while the Tool helps the
user complete the 3 steps. The tool includes descriptions of over 40 potential biases and
barriers and related mitigation strategies. Some of these potential biases and barriers
may apply to any assessment method and others may apply only to one specific
type of assessment tool, such as an interview. Sasha wants to design the interview in a way that
is fair for everyone. She will be proactive and will apply some of the mitigation strategies from
the PSC tool as she develops the interview instead of doing it later in the process.
Sasha will follow the PSC's guiding principles closely. She will maintain
a constant awareness of self and others and reflect on the reasons behind her decisions.
She will make the interview accessible to various people and will make sure it
measures the qualifications accurately and consistently. Standardization can help with
this step when done properly. She will also be transparent and proactive. Ultimately,
she wants to give everyone a fair chance to fully demonstrate their qualifications.
Sasha takes a moment to think of the first guiding principle: awareness of self and others. She
reflects on her own experiences. She can remember her first government interview as a candidate. She
was rather confused by all the steps and felt overwhelmed before the interview even began. At
one point, she asked to clarify a question that seemed open to interpretation. The assessment
board was reluctant and would not elaborate. She thought their attitude was because of her accent,
and had the impression that they didn't want to hire her. She felt discouraged and gave short,
incomplete answers to the remaining questions. Sasha also remembers when, later as a hiring
manager, she met a candidate who had attended the same high school as she had. They chatted a lot,
sharing anecdotes about the school and community. She realizes that she easily engages with people
who are similar to her and that she might be less prone to building the same rapport
with those who are not. This can lead to a common rating error called affinity bias.
Sasha knows that everyone is biased in some way because it is how our brains work. They
make shortcuts to process information, but these shortcuts can be prone to error and have a real
impact on assessments. She will remember this when planning the interview and will integrate
different perspectives throughout the process. Sasha will complete the evaluation of biases and
barriers in 3 simple steps found in the PSC guide. Step 1 – she will write a description of the
interview and how it will be administered. Step 2 – she will identify potential biases
and barriers that may disadvantage persons belonging to any equity-seeking group.
Step 3 – she will take action to remove them or mitigate their impact.
She will keep track of what she has done in the PSC tool, which can also serve as documentation.
Sasha starts with Step 1, reflecting on her approach. She needs to assess 6
qualifications and plans to write 6 interview questions, one per qualification. She will
have 4 behavioural questions which ask about past behaviours and 2 situational questions
which ask about hypothetical situations. She will draw inspiration from past interviews.
The interview will last 1 hour and will be administered in person by an assessment board of
3 people. The board members will be instructed to assign ratings using a 5-point scale from
"does not meet" to "exceeds expectations". Sasha writes these key points in the PSC
tool. She will update the interview description if her approach changes.
Sasha has described the interview method and will continue with Steps 2 and 3. She will
identify potential biases and barriers and take action to remove them or mitigate their impact.
She will start by considering aspects of the design, including the questions themselves,
before looking at administration and scoring. One of the first mitigation strategies in the
PSC tool is to "Build diversity and inclusion into your plan and consult others". Doing this helps
ensure the interview assesses qualifications that are related to the job and that reflect
different ways of working effectively. Sasha speaks with an HR specialist,
who can give her advice. Together, they exchange ideas on best practices in interviewing and
discuss ways to be more inclusive and flexible. The HR specialist points out that some people
may need more time to prepare answers that fully demonstrate their qualifications.
He asks whether the job often requires working under pressure or within tight deadlines. Sasha
replies that this is not a requirement of the job. In that case, he suggests adding
more time and including a preparation period. Sasha agrees with this approach. This will
help ensure that the skills needed to respond during the interview are related to the job.
They also discuss various ways that people might demonstrate the qualifications on the
job so these ideas can be considered when preparing the questions and scoring key.
Sasha examined the job requirements, added a preparation period and is considering
different ways of working. This helps her mitigate the impact of 2 potential biases
listed in the PSC tool: the qualifications are narrow or beyond what is required for the job;
and the assessment captures only one way of demonstrating the qualifications.
Sasha feels confident about her plan and will now draft the interview questions.
Sasha is working on the interview question to assess Initiative. She uses a question from
a past interview. The question is "Tell me about a time when you took initiative
to resolve a policy issue on your own." Reading the PSC tool, she realizes there
are potential problems with this question. First, it may not accurately measure the
qualification. She wants someone who can anticipate challenges and act without being
prompted. This might be unclear when saying "took initiative". Also, the person could have
shown initiative in many different contexts, but not policy. As a mitigation strategy,
she will remove the word "policy" and will clearly define "initiative" in the definitions
given to candidates and the scoring key. She doesn't find any more potential biases in the
section common to many assessment methods, so she looks at the items specific to interviews in the
PSC Tool. The first potential bias for interviews is "too much emphasis on past work behaviours". As
a mitigation strategy, she will encourage people to consider other past experiences by telling
them the experience could come from various contexts such as work, volunteering or school.
There is also a risk of gathering incomplete information since there are no planned follow-up
questions. Sasha will add follow-up questions so that the people responding can give more
complete answers. These will include "who did you involve?", "what was the
situation?" and "what was the outcome?". Sasha asks a fellow manager to review the
question. Her colleague points out that the term "on your own" may not be inclusive because it
emphasizes individual action, and some people might value collective action and outcomes.
She suggests removing "on your own" and adding a follow up question asking, "what steps did you
take and why". She also offers to help Sasha draft a situational question if she wanted
to give people the chance to show initiative without being restricted to past experience.
Sasha's revised interview question is now "Tell me about a time when you took initiative to
resolve an issue" and includes 4 follow-up questions. She completes the same exercise
for the other interview questions. Once that is done, Sasha starts
reviewing the administration procedures. To limit the risk of inconsistent administration,
Sasha and the HR specialist have prepared administration instructions, procedures,
and best practices for the assessment board and test administrator based on past processes. This
includes a script with introductory remarks to help build rapport and put people at ease.
Sasha knows the risks of having limited instructions for candidates. She does
not want people to feel the same way she did during her first government interview,
so she will be sure to give clear instructions about what is being assessed, the number of
questions, the number of board members, the note taking procedure, the interview duration, etc.
Candidates will also be informed of the room set-up and given the opportunity
to request adjustments if they prefer a different seating arrangement to being
seated in front of a panel. This will help mitigate the potential bias of rigid seating
structure and effects of implied hierarchy. After some reflection, Sasha decides to give
the option for virtual interviews because this may help reduce potential barriers to in-person
administration. She updates the test material and administration procedures accordingly.
Finally, to mitigate the risk of insufficient time, Sasha will allot a separate preparation
period where candidates read the 6 questions and have 40 minutes to prepare their answers.
The board will then greet them, spend 5 to 10 minutes on introductory remarks,
then give 40 minutes to answer the interview questions. Additional time would be reserved
at the end for closing the interview. Now that Sasha has the administration
organized, she reviews the 5-point scale and scoring procedures.
She clearly defines what is expected at each level of performance in the 5-point scale.
She asks the board members about different ways people could respond to the questions. She reminds
them to consider all answers objectively, even if an approach is unexpected or not
covered in the rating booklet. This helps her mitigate the impact of narrow rating criteria.
Sasha has prepared a rating booklet to remove potential bias related to a lack of structured
rating. The booklet includes descriptions of the qualifications, the scoring key, examples of
effective responses and space for taking notes. It also details the rating procedures such as
rating each qualification individually before coming to a consensus, considering evidence for
a qualification across all questions asked, and justifying each rating based on the notes taken.
Sasha will mitigate the impact of insufficient note taking by asking
each board member to take detailed notes of what the person says rather than paraphrasing.
She will also address common rating errors by asking the board to rate responses shortly after
the interview and to take breaks to reduce fatigue-related rating errors. She will also
schedule only 1 or 2 interviews per day. The structured rating will help ensure the
validity and reliability of the results. Sasha is now ready to select the board
members. She wants to choose more than 3 people so there are always enough board
members per candidate. She's thinking of inviting 2 managers and 2 job incumbents
from her division, given their experience, familiarity with the job and availability.
The HR specialist reminds her of the bias in the PSC tool related to an assessment board
being homogeneous or not properly trained. He can help her identify other qualified
people by reaching out to their department's committee on equity, diversity and inclusion.
Sasha agrees and says she will ask everyone to follow training on common rating errors,
unconscious bias and cultural sensitivity. She will also check that ratings are consistent by
reviewing a random selection of responses. Sasha and the HR specialist prepare the
documentation, following their departmental staffing policies. Sasha saves a copy of the
interview material. She makes sure that potential biases or barriers identified are checked off
in the PSC tool, including the mitigation strategies applied. She takes note of any
additional actions taken and the different people consulted throughout the process.
The purpose of documentation is to show what method was used to evaluate the interview,
and what potential biases or barriers were identified, including the actions taken to
remove them or mitigate their impact. The PSC does not prescribe a specific
format for how to document the evaluation. Some organizations may choose to list observations
and actions in a separate document, write brief narrative descriptions, or apply another format.
Sasha is ready to move forward with the interviews! She will keep the guiding principles
in mind, committing to being transparent and fair when communicating with candidates. Sasha and the
board will follow procedures as planned to support the standardization, validity, and reliability
of the interview, while offering flexibility and accommodations to meet individual needs.
As a reminder, accommodation focuses on the individual needs of a person and can
be requested at any point of the appointment process, while the evaluation of the assessment
methods must occur before their use. Despite her best efforts, an error or
bias may still occur during the interviews. She will be sure to document the information and will
work closely with HR to consider the situation objectively. She will carefully analyze any new
information or seek feedback from others about the fairness of the interview before taking action.
When Sasha has completed the interviews, she will ask for feedback from candidates, board members
and HR to identify future areas of improvement. The appointment process is now over! Sasha has
hired a qualified person who will be a great asset and fit for the team.
She is content with the process and shares the following key take-aways with her colleagues:
It's not as difficult as she had imagined. Many of the strategies are easy to implement
or she was already doing them. A few changes can have a significant
impact. For example, giving preparation time and thinking ahead about different
candidate needs and experiences. Using the PSC tool early on
made it easier to implement good strategies and saved her time later.
Collaboration is key. HR was a vital support, and she was encouraged to
see the knowledge and experience of others in her department and networks.
This will only get easier over time. Overall, it was a positive experience.
She learned about herself and her own potential biases. Speaking with others
helped her integrate diverse perspectives and ultimately led to a fairer assessment.
She will continue learning and staying connected with others in
support of inclusive staffing practices. Thank you for following Sasha's story!
See the link below to explore additional tools and resources on the Amendments to
the Public Service Employment Act, and be sure to add it to your Favorites!
A fair assessment is inclusive, accessible and measures the required qualifications.
Purpose
This practical guide is for human resources specialists and hiring managers. It explains how to evaluate your assessment to identify, remove or mitigate any potential biases and barriers. As these can occur at any stage of an assessment, including planning, administration and scoring, you need to make reasonable efforts to remove or mitigate their impact on members of equity-seeking groups.
- See the Public Service Employment Act
Use this guide and the Tool for Mitigating Biases and Barriers in Assessment (or another tool of your choice) to evaluate your assessment methods before conducting your assessments. The guide and related tool apply to assessments developed and administered by the hiring manager or third parties. It's best to evaluate your assessment as it is being developed, rather than afterwards. Don't forget to take into account how much effort and time is needed to apply the right strategies for a fair assessment.
Use this guide to support the Government of Canada's commitment to an inclusive public service.
Definitions: biases and barriers that disadvantage persons belonging to any equity-seeking group
See Definitions: biases and barriers that disadvantage persons belonging to any equity-seeking group.
Guiding principles
The following 6 principles define a framework that will help you make decisions on how to apply fair assessment methods:
- awareness of self and others
- accessibility
- validity and reliability
- standardization
- transparency
- fairness
These principles are integrated into the Tool for Mitigating Biases and Barriers in Assessment, and can be applied to all parts of an assessment method and its application.
Principle 1: Awareness of self and others
People are naturally likely to have biases, which are shortcuts our brain uses to process information and respond. These shortcuts are shaped by personal values and experiences, influencing how we see and interact with others. They are, however, prone to error. Awareness is the first step in ensuring effective mitigation of biases and barriers in assessments.
Why awareness of self and others is important
- Bias can shape how we judge a candidate
- Bias has a negative impact on members of equity-seeking groups
- Bias happens more often when we're tired or when we make judgments based on first impressions
Where to start
- Commit to learning and personal reflection: learn about inclusive hiring practices and reflect on your values and personal experiences that may influence your judgment of others
- Consult the Federal Public Service Inclusive Appointment Lens
- Consult the Inclusive Recruitment Toolkit (accessible only on the Government of Canada network)
- Learn about biases and barriers within your department or agency from employment systems reviews, surveys, audits, reports, publications and data
- Engage with employee diversity networks to learn of lived experiences in assessment
- Learn to spot and avoid common rating errors
- Have the courage to approach others if they are being biased
Awareness requires constant personal reflection, a desire to learn and the courage to change.
Principle 2: Accessibility
Accessibility is about how easily people can navigate and interact with the assessment method and its environment. People should be able to perform at their best without barriers related to technology, scheduling, language, or any other factor unrelated to the assessment criteria.
Why accessibility is important
- Encourages people to apply and succeed (helps prevent people from screening themselves out)
- Allows people to show their true performance
- Produces more accurate and reliable results
How to improve accessibility
- Proactively support practices of inclusion by design, and choose accessible assessment methods
- Ask yourself whether the assessment is in plain language, neutral, adaptable and inclusive for different equity-seeking groups
- Learn more about accessibility criteria and available tools, such as the Digital Accessibility Toolkit and the Government of Canada Workplace Accessibility Passport
- Be transparent and have an open discussion about the type of assessment and work accommodation available
- Provide assessment accommodation measures promptly
Design to allow everyone to show their true performance.
Principle 3: Validity and reliability
A fair assessment measures qualifications with accuracy (validity) and consistency (reliability).
Why validity and reliability are important
- A valid and reliable tool allows for a more accurate evaluation of merit
- A valid tool directly measures the qualifications that are needed for the job
- A reliable tool produces comparable results across time, in different assessment environments and with different assessment boards
How to improve validity and reliability of assessments
- Look for and evaluate criteria that are relevant to the job
- Select an appropriate assessment method and seek expert help, as required
- Pilot the assessment method to ensure it assesses criteria accurately and consistently
- Stay vigilant: avoid factors that can fatigue candidates and assessors
- Reduce administration errors (such as incomplete instructions and inconsistent timing)
- Provide a comfortable testing environment that is free of distractions
- Ensure consistent use of the rating method across assessment boards
Measure qualifications accurately and consistently to improve selection decisions.
Principle 4: Standardization
When standardizing an assessment, the same method is used in the same way for each person.
Why standardization is important
- Promotes fairness by minimizing biases that aren't related to the assessment
- Reduces variations in testing conditions to better capture and compare performance
- Helps preserve validity and reliability by keeping testing conditions consistent
How to ensure standardization
- Use the same instructions, test content, administration, and rating method
The goal of standardization is to support a fair assessment of all candidates. However, any standardization that doesn't serve fairness should be reconsidered. Flexibility may be required to ensure all candidates have an equal opportunity to perform at their best.
You can show flexibility in standardized assessments by:
- proposing flexible schedules
- this will give people more opportunities and a better chance to prepare
- using multiple assessment tools to measure the same qualification
- this will give you more sources of information about each person's performance
- offering assessment accommodation measures on a case-by-case basis, while keeping standardization in the nature and level of the qualification being assessed
- offering diverse options and non-traditional assessment methods (for example, pre-recorded interviews, work samples)
- this can create more opportunities for people to perform at their best (see guidance on flexibility in assessment)
Ensuring key elements of the assessment method are standardized is central to a fair assessment.
Principle 5: Transparency
People perform best when they understand what is being assessed and how they will be rated. Documenting and communicating the reasons behind each decision (for example, choice of assessment method or rating method) can help ensure integrity and transparency.
Why transparency is important
- Increases trust in the staffing process
- Helps candidates understand the complex assessment process
- Promotes a positive assessment experience and reduces test anxiety
What to communicate to candidates
- Information about the assessment process (for example, qualifications being assessed, type of questions and length)
- Tips on how to prepare and how to structure responses
- Flexibilities that don't require assessment accommodation measures (change in assessment date, different response formats)
- Types of assessment accommodation measures that might help reduce specific barriers (use of assistive technology, extra time), and how to ask for them
- Ways to ask for more information (providing contact information, inviting to orientation sessions, sharing links to additional information, or recruiting packages)
Be clear and proactive. Tell candidates what to expect.
Principle 6: Fairness
The goal of an assessment is to give everyone the chance to perform at their best. Fairness is a fundamental concept to be considered from the start of an assessment process. The inclusive appointment lens and fairness review checklist can help you build fairness into each step of the assessment process.
Why fairness is important
- Gives equal opportunity to all candidates, including members of equity-seeking groups
- Fosters trust in the staffing process and promotes a positive assessment experience
- Supports the legal defensibility of an assessment
How to improve fairness
- Give all candidates a fair opportunity to demonstrate their qualifications and offer assessment accommodation measures when required
- Make sure the qualifications assessed are required for the position
- Consider the various ways people learn and perform on the job when you choose or design your assessment method
- Ensure the assessment method provides valid and reliable measures of the qualifications
- Check that the administration of the assessment and its rating method aim to be impartial and unbiased
Give everyone a chance to fully demonstrate their qualifications.
How to conduct an evaluation of biases and barriers
This section outlines the 3 steps to evaluate any assessment method and its application.
- Describe the assessment method and its application
- Evaluate the assessment method and its application to identify potential biases and barriers
- Take action to remove or mitigate biases and barriers
According to the Public Service Employment Act, the evaluation of biases and barriers must be conducted before using an assessment method. The Tool for Mitigating Biases and Barriers in Assessment can help you to fulfil this requirement. It includes a list of biases, barriers, and mitigation strategies that are common across assessment methods or specific to screening, written tests, interviews, reference checks, narrative assessments and performance reviews. Note that you may also use or develop your own tools on biases and barriers in assessment.
The guide and related tool present many biases and barriers that may occur in assessments. Other biases and barriers may exist, and some strategies may not be suitable for all situations. Consider taking note of biases or barriers and mitigation strategies that aren't included in this guide and the related tool. When determining what strategies to apply or action to take, keep in mind the job requirements and the merit criteria.
1. Describe the assessment method and its application
For each assessment method, consider describing:
- the qualifications and how they will be assessed
- the administration process
- the rating method and how decisions are made
- the people involved in the different stages (planning, administration, and scoring), and any training that may have been required or taken
2. Evaluate the assessment method and its application to identify potential biases and barriers
Identify sources of potential biases and barriers in your assessment, how the assessment is conducted and all related material. Pay attention to any aspect of the assessment that might not uphold the Guiding principles.
Aspects to consider when conducting your evaluation:
- information provided to candidates before the assessment
- instructions and questions
- rating guide
- assessment board training
- administration manual
- any other related materials and processes
Look for biases and barriers that may occur during:
- planning (for example, qualifications are too narrow or beyond what is required for the job, or the language is too technical)
- administration (for example, limited preparation time or untimely assessment accommodation measures)
- scoring method (for example, unstructured rating process or homogenous assessment board)
A description of common biases and barriers is available in the Tool for Mitigating Biases and Barriers in Assessment.
3. Take action to remove or mitigate biases and barriers
Once you've identified the potential biases and barriers, reasonable efforts must be made to remove or mitigate their impact. Use professional judgment to decide what actions to take and whether they are satisfactory. You can involve colleagues, assessment specialists or representatives of equity-seeking groups to gain different perspectives.
Consider different actions you can take when planning:
- the design (for example, build diversity and inclusion into the work plan and review material for accessibility and fairness concerns)
- the administration (for example, create flexible schedules, give ample preparation time and the same instructions to everyone)
- the scoring (for example, prepare a rating booklet and review borderline ratings with other board members)
Common strategies are listed in the Tool for Mitigating Biases and Barriers in Assessment.
Documentation
As you proceed with the evaluation, make note of which guide, tool or reference document was used as the basis or lens to identify biases and barriers in each of your assessment methods and their application. You should also report on the actions you took to remove or to mitigate the impact of biases or barriers identified.
As the hiring manager, collaborate with your human resources specialist responsible for the process to act promptly if issues arise during the assessments. You may also gather feedback after the assessments (from candidates, board members, for example) to learn from the experience and apply any lessons learned to future assessments.
Your feedback
This guide and the related tool are made for you. We're looking for your reactions and feedback on how to improve them. Please complete the short feedback survey.