Enhance fairness and reduce bias in the content of assessment tools

What does the checklist cover?

The questions in the Fairness Review Checklist fall under five areas:

• readability
• inclusiveness and representativeness
  
• neutrality
• format accessibility
• alternative formats

Benefits of using the checklist

• enhances the validity and fairness of tests and tools
• reduces the use of stereotypes and potential negative representation of groups
• helps identify and eliminate test content that could pose barriers to test-takers
• helps identify and eliminate test content that is irrelevant to the qualifications being assessed
• reduces the likelihood of tests and tools being challenged
• enables test developers to consider fairness-related issues, which can be very challenging and/or costly to address later
• helps reduce accessibility accommodation requests

Scope not limited to employment equity groups

We encourage you to consider diversity when reviewing documents, as doing so will ensure the assessment tools are as fair as possible for all test-takers.

The Fairness Review Checklist will help you identify:

• content that could be problematic for employment equity (EE) groups: Women, Indigenous persons, visible minorities, and persons with disabilities
• fairness issues that go beyond the EE groups

Contributing to fairness

Fair assessment occurs when:

• the qualifications being assessed are necessary for the job
• the instruments used to assess the qualifications are sound
• all test-takers are provided with an equal opportunity to demonstrate their qualifications

Other best practices that contribute to fairness include:

• consultations with EE groups during test development
• proper and standardized test administration
• setting appropriate pass marks

Please note: This checklist can also be useful when reviewing other types of documents for fairness.

Fairness Review Checklist

1. Readability

• 1.1. Is all text written concisely and as clearly as possible?
• 1.2. Does the text contain uncommon words, given the target level?
• 1.3. Is the material free of slang or local expressions?
• 1.4. Is the material free of idioms or colloquial expressions that might be less familiar to test-takers whose first language is not English?
• 1.5. If the text has been translated, is it a high-quality translation? Are there errors that have an impact on the flow of the text and the test-takers’ ability to understand it? For example, are there errors related to spelling, grammar, syntax, punctuation, typography or vocabulary (literal translations form French)?
• 1.6. Is the material free of acronyms or abbreviations?
• 1.7. Is the material free of all kinds of jargon (for example, government-specific, political, business, and technical)?
• 1.8. Is the terminology correct and appropriate when referring to specific groups?
• 1.9. Can the active voice be used instead of the passive voice?
• 1.10. Is the material free of double negatives?

2. Inclusiveness and representativeness

• 2.1. Can the vocabulary and concepts in the material be understood by test-takers from diverse educational backgrounds and occupational backgrounds?
• 2.2. Are terms defined broadly enough to capture diverse ways of doing things?
• 2.3. Examine the names, positions, roles of characters in the test content. Are people from diverse backgrounds and genders presented in a balanced way?
• 2.4. Is the material free of terms or expressions that are only associated with people from a specific cohort or generation?
• 2.5. When terms or expressions are associated with people from specific regions, are the selected terms the most well-known and commonly used by people in the target region?
• 2.6. Are certain groups excluded in collective terms such as “we” or “management”?

3. Neutrality

• 3.1. Is the material free from bias, inappropriate assumptions, or stereotypes in terms of gender, culture, religion, beliefs, etc.?
• 3.2. Are issues related to employment equity and diversity presented in a balanced and neutral way? Is the order in which questions or test items are presented fair to all groups?
• 3.3. Is the material free of references specific to a particular culture, such as movie, TV show, or cartoon characters, entertainment or sports personalities, which may impede understanding for members of other cultures?
• 3.4. Is the material free of inappropriate and unnecessary reference to gender, ethnicity, religion, etc.?
• 3.5. Are the tone and nuance of each word or expression positive or neutral?
• 3.6. Is the material free of controversial topics that might cause unnecessary distress?
• 3.7. Does the material contain content that might cause unnecessary embarrassment or discomfort?

4. Format accessibility

• 4.1. Is the spacing sufficient between sections (at least two lines) or paragraphs (at least one line)?
• 4.2. Is a font type without serifs used (e.g., Arial, Verdana, Tahoma rather than Times New Roman)?
• 4.3. Is the font size at least 12 points?
• 4.4. Is the text aligned to the left margin?
• 4.5. Are colours and shading avoided?
• 4.6. Are the margins at least 2.5 cm?
• 4.7. Is the structure consistent throughout the document?
• 4.8. In documents where there is a space provided to write an answer, is there a limit on the number of words or characters allowed? If yes:
  1. Does the limit take into account the differences between the two official languages in terms of writing and is it an evidence-based limit (for example, pilot tests)?
  2. Does the limit allow test-takers to fully demonstrate their ability?
  3. Is this limit suitable for different writing styles (for example, literary, direct and concise styles)?

5. Alternative formats

• 5.1. Is the instrument developed in a manner that lends itself to multiple formats for persons with disabilities?
• 5.2. Can the information in this instrument be easily processed when presented in a different format or modality (for example, auditory)?
• 5.3. Does the instrument include information presented in a visual format such as graphs, charts, tables, calendars, figures, or maps that may not be easily processed when delivered in another format (for example, screen reader)? If so, can this information be removed or modified?
• 5.4. Is the necessary information regarding the availability of accommodation provided?