Anonymized Recruitment Pilot Project

Results and Next Steps

Table of contents

• Background
• Pilot objective
• Overview of methodology
• What we looked at
• Key findings
• Additional findings
• Challenges and considerations
• Next steps
• Appendix A: Participating organizations

Background

• In April 2017, in collaboration with the Office of the Human Resources Officer (OCHRO), the PSC undertook an Anonymized Recruitment Pilot Project (ARPP)
• In recent years, numerous jurisdictions have undertaken studies and initiatives to explore how anonymization can influence hiring decisions.
  • Initiatives range from pilot projects and studies (e.g. Australia) to full implementation (e.g UK Civil Service)
• Results of these initiatives have been mixed with some showing benefits of anonymization and others either failing to observe benefits or showing an adverse impact on some sub-groups.

Pilot objective

• The objective of the pilot was to determine:
  1. if concealing personal information which could lead to the identification of a candidate’s origin, from job applications had an impact on the screening decisions made by reviewers; and
  2. if an impact is observed, whether it translated into a benefit or a disadvantage for EE group members
• Participating organizations (see Appendix A) and sample:
  • 27 external processes from 17 volunteer organizations were included in the pilot
    • 2226 candidates
    • 685 candidates who self-declared as members of visible minority (30.8%)
    • Overall, 54 reviewers (2 reviewers per process) resulting in 4452 independent screening decisions
• Given the small number of candidates who self-declared as a member of Aboriginal peoples (n: 73 candidates or 3%) or as a person with a disability (n: 102 candidates or 5%), the analysis was restricted to visible minorities.
• Given that gender could easily be determined from applications submitted in French, the ARPP team did not remove any references made to gender.

Overview of methodology

The anonymization process was divided into five phases:

1. For each process, participating departments sent applications to the PSC.
2. The PSC then randomly distributed selected applications across 4 groups (See Table 1).
3. Applications requiring anonymization were assigned to trained anonymizers for redaction.
4. Redacted applications were subsequently quality assured via a second trained anonymizer.
5. Applications were returned to the organizations for assessment by reviewers.

Distribution of applications for each process

Group 1 (25% of applications)

• Reviewer A: Traditional
• Reviewer B: Traditional

Group 2 (25% of applications)

• Reviewer A: Traditional
• Reviewer B: Anonymized Recruitment

Group 3 (25% of applications)

• Reviewer A: Anonymized Recruitment
• Reviewer B: Traditional

Group 4 (25% of applications)

• Reviewer A: Anonymized Recruitment
• Reviewer B: Anonymized Recruitment

For each process, two reviewers were asked to assess all applications independently and were instructed not to consult each other during their assessment.

The personal information in a number of these application was redacted. Redacted information included:

• Last name, first name, initials and any other references to the candidate’s name
• Citizenship and country of origin
• Mailing address(es) and telephone number(s)
• Educational institutions
• Any references to organizations, businesses and establishments where general training and professional experience were acquired
• Languages spoken and written
• Any references to geographical locations, other than those related to a professional association
• Any references that may compromise employment equity, other than to the female gender
• Any references to religion
• Any references to publications (university or other)

What we looked at

An analysis was undertaken to assess the effect of various factors on the applicants’ screen-in rates. The following variables were included in the analysis :

• Assessment methods (Anonymized and Traditional);
• Visible minority status;
• Occupational category;
• Experience in the Federal Public Service;
• Number of essential qualifications required;
• Order in which applications are reviewed.

Key findings

There was no net benefit or disadvantage of the AR assessment method for visible minorities. However, AR significantly reduces the rate of being screened-in for all other candidates.

Additional findings

Candidates with Federal Public Service experience have higher screen-in rates. This effect exists for all candidates across assessment methods.

The effect of Federal Public Service experience is essentially the same for visible minorities and all other candidates.

Challenges and considerations

• Anonymizing candidates’ information proved to be labour intensive
  • Depending on position being staffed more or less info is to be redacted
  • Letters and CV need to be carefully reviewed
• Complete anonymization is not always possible or preferable
  • Risk of removing skill related information
• There are some inherent limitations to the pilot
  • Hiring managers were aware of their participation in the pilot project which may have influenced their screening decisions
  • No matter how good quality control is – maybe some information left behind
• Making inferences to the whole of the public service
  • Pilot project relied on volunteer organizations and a non-random selection of recruitment processes
  • Pilot project must be considered as one source of information on the potential benefits of AR in our organizational context

Next steps

To complement the results of the pilot project, audit work will be undertaken to explore drop off rates of EE groups in a larger and random selection of recruitment processes.

As AR may represent a viable option in some circumstances, solutions will need to be explored to alleviate any additional operational burden associated with AR should organizations opt to implement this screening method.

Appendix A: Participating organizations

The scope and methodology of the pilot have been designed to explore the impact of Anonymized Recruitment in collaboration with these seventeen (17) participating organizations:

AAFC

Agriculture and Agri-Food Canada

ESDC

Employment and Social Development Canada

ECCC

Environment and Climate Change Canada

DFO

Fisheries and Oceans Canada

GAC

Global Affairs Canada

IRCC

Immigration, Refugees and Citizenship Canada

INAC

Indigenous and Northern Affairs Canada

INFC

Infrastructure Canada

DND

National Defence

NRCan

Natural Resources Canada

PBC

Parole Board of Canada

PSC

Public Service Commission

PSPC

Public Services and Procurement Canada

RCMP

Royal Canadian Mounted Police

StatCan

Statistics Canada

TC

Transport Canada

TBS

Treasury Board of Canada Secretariat