Evaluation of the Recruitment of Policy Leaders Program

Report Summary

PDF Version

1. Purpose of the evaluation

1. The evaluation objective was to assess and report on the relevance, effectiveness and efficiency of the Recruitment of Policy Leaders program (the program) development, administration and achievement of outcomes. The evaluation provides a neutral, independent view on how the program supports government objectives and priorities, and information on the performance and continued relevance of the program as currently designed and implemented.

2. Program background

2. In 2000, the Auditor General of Canada noted, “In light of the increased competition for university graduates, the [Public Service] Commission needs to be a more aggressive recruiter. There has to be better promotion of the public service as a career choice.” ^{Footnote 1} The same year, the Government of Canada’s Committee of Senior Officials Sub-Committee on Recruitment commented on the pressure this environment was putting on the federal public service, noting “we face substantial competition for any recruits, in particular highly skilled Canadians who have global opportunities for employment.” Concerned by diminishing policy capacity after years of cuts and the absence of targeted recruitment strategies, a number of deputy ministers launched an initiative to recruit Canadians with international graduate degrees and related work experience to reverse the talent “brain drain” from Canada. ^{Footnote 2}

3. In 2001, the Privy Council launched a pilot project titled the Recruitment of Outstanding Canadians. The goal was to attract Canadians in post-graduate studies programs abroad to federal public service jobs. Small recruitment teams comprised of Deputy Minister Champions interviewed candidates for the most part on their home campuses. Candidates who met broad-based merit criteria were placed in a pre-screened inventory and went through a series of interviews led by deputy ministers and/or assistant deputy ministers in Ottawa and organized by past recruits. Successful candidates were typically appointed to middle and senior level policy analyst positions.

4. In October 2004, a committee of deputy ministers made the pilot permanent. The goal was to hire and retain top-level talent who could become future leaders by “attracting high-caliber individuals to policy-oriented federal public service middle to senior level positions.” ^{Footnote 3} In December 2004, the program was renamed the Recruitment of Policy Leaders, and the Public Service Commission of Canada assumed administration responsibilities for program delivery. Initially, deputy ministers continued to serve as champions, and program alumni volunteers continued to screen applications, participate in interview processes, and serve as mentors.

5. Since 2004, the program has evolved. The statement of merit requirement has changed incrementally and the recruitment approach has progressed over time. Initially, the program sought to attract Canadians studying abroad who had been awarded significant competitive international scholarships (such as Rhodes and Commonwealth). Today, the focus is on recruiting Canadians, whether studying abroad or in Canada, who demonstrate exceptional academic, employment, as well as extra-curricular performance.

6. A number of innovations have been trialed, including a secondary pool of qualified individuals for lower level policy positions. A second trial in this area was established for the 2017–18 recruitment campaign and continues as of May 2019. Known as the Emerging Talent Pool pilot, this initiative was designed to maximize program benefits by developing a framework to hire candidates who were not placed in a partially assessed inventory but could make an important contribution to renewing policy capacity. In 2017–18, 20 candidates were qualified through the Emerging Talent Pool pilot. As of April 9, 2019, 9 were hired into positions at the EC-4 or equivalent level.

The Recruitment of Policy Leaders program in 2018–19

7. A Deputy Minister Champion appointed by the Clerk of the Privy Council oversees the program and is responsible for program direction, promotion and marketing of qualified candidates to Deputy Minister Colleagues across the federal public service. The current champion is the Deputy Minister of Global Affairs Canada.

8. The Recruitment of Policy Leaders Executive Committee and officials from the Public Service Commission’s Service and Business Development Sector support the Deputy Minister Champion and help promote the program, screening potential candidates and managing the budget for travel, accommodation and any other expenses related to recruiting and placing candidates into the federal public service. Recruitment of Policy Leaders alumni continue to have an important role in recruiting and supporting successful candidates.

Program resources

9. Program expenditures consist of direct and indirect costs. Direct costs include Public Service Commission staff dedicated to program administration in the National Capital Region (1 PE-5 at 30%, a PE-3 at 70% and an AS-2 at 100%). Indirect costs include allocations, such as travel and support expenditures from hiring organizations, as well as employee time. For example, a typical interview involves an assistant deputy minister and 2 alumni at the EC-6 or EC-7 level or higher. The Deputy Minister Champion’s Office also allocates resources to support the administration and delivery of the program. Appendix 1 provides information on program expenditures from 2008–09 to 2018–19.

The recruitment process

10. The 5-step recruitment process is outlined below in Figure 1 and described in the following paragraphs.

Figure 1 – Recruitment of Policy Leaders multi-stage recruitment process

11. Steps 1 and 2: Applicants are screened according to whether they have a doctorate, master’s degree, or professional degree in law complemented by an undergraduate degree. Applicants must also have received a notable scholarship or significant distinctions at the graduate level, have relevant policy experience, and have demonstrated extra-curricular leadership achievements. In 2018–19, formal screening materials were reviewed by the Public Service Commission’s Personnel Psychology Centre for use by program officials throughout the recruitment process (that is, the framework for screening applications, conducting interviews and conducting reference checks). While some aspects of screening materials, such as interview questions, have varied, the screening and assessment process continues to follow the original format. At this stage, applicants who pass the first step are invited to submit a writing sample that is assessed as part of the screening process.

12. Step 3: Applicants who pass the initial screening are invited to interviews. Senior federal public service executives lead these interviews and are supported by 2 program alumni. Interviews are conducted by telephone or in person. The interview assesses candidates against established statement of merit criteria and core competencies.

13. Step 4: Reference checks are performed for applicants who pass the interview phase. Approximately 30 to 50 candidates have reference checks performed each year. Applicants who pass the fourth step are placed in a partially assessed inventory and may remain in the inventory for 2 years.

14. Step 5 and beyond: Annually, once the partially assessed inventory has been established, the Deputy Minister Champion presents candidate biographies to colleagues at a deputy minister breakfast meeting. Candidates who are looking for positions are then paired with an alumni mentor. Mentors are volunteers who are committed to contributing to the continued success of the program. They provide guidance, advice and assistance to successful candidates included in the partially assessed inventory for job placement purposes within the federal public service.

15. Mentors work with departmental liaison officials to support the matching of candidates with hiring managers. They work with the Deputy Minister Champion’s Office to promote successful candidates to hiring managers across the federal public service. As well, candidate biographies are posted on a GCpedia (Government of Canada’s internal wiki) page (accessible only on the Government of Canada network).

16. During the 2-year matching period, hiring managers may invite candidates from the partially assessed inventory for an interview to conduct further assessments against specific qualifications, requirements, or criteria. Hiring managers have discretion on the level of the position offered to successful candidates. Both parties also negotiate salary and payment for agreed-upon relocation expenses.

17. Table 1 provides an example of the multi-stage recruitment process using data from the 2017–18 campaign. It shows the number of applicants and the steps for developing the final list of qualified candidates in the partially assessed inventory, as well as the number of candidates who are hired through the program.

Table 1 – Recruitment of Policy Leaders program multi-stage recruitment process, 2017–18 campaign

Recruitment process	2017–2018
Total number of applications received	1 633
Total number of applications sent for senior screening Footnote 4	1 200
Total number of applications sent for primary screening Footnote 5	825
Total number of applications sent for secondary screening Footnote 6	258
Total number candidates invited to submit a written sample	190
Total number of candidates invited to an interview	107
Number of qualified candidates	27
Number of hires	21

3. Evaluation scope, questions and methodology

Scope

18. The scope covered the period from the previous evaluation in 2008 to December 2018. The Emerging Talent Pool pilot was not included in the scope, as it was in the early stages of implementation.

Evaluation questions

Relevance

Program need

Is there a continuing need for the Recruitment of Policy Leaders program?

Program alignment

Are the program objectives aligned with government priorities and roles and responsibilities?

Effectiveness

Immediate outcomes

Are qualified candidates placed in policy positions in the federal public service?

How does the federal government leverage Recruitment of Policy Leaders recruits?

Intermediate outcomes

Do alumni make a meaningful contribution to the federal public policy framework?

Do alumni exercise leadership on the public policy framework?

Is the program aligned with federal public service diversity objectives?

Ultimate outcome

Do the alumni influence and shape Canada’s public policy positions to address emerging national and global challenges?

Efficiency

Are there better, alternative ways (for example, design changes) or lessons learned for achieving the same results?

Are the appropriate governance structures and processes in place to support the effective delivery of the program?

Methodology

19. The methodology (see Appendix 2 for a detailed description and limitations) outlined below was used to answer the evaluation questions developed in consultation with key stakeholders based on a revised program logic model (see Appendix 3). The evaluation matrix (see Appendix 4) details the evaluation indicators and the multiple lines of evidence employed. Data from these sources were analyzed and used to develop conclusions and recommendations.

20. The following methods were employed to answer the evaluation questions:

• Document review. We reviewed strategic departmental and policy documents (including Public Service Commission Annual Reports, Departmental Plan, Clerk’s Annual Reports, organizational human resources plans). Other key documents were reviewed (for example, similar recruitment programs in other federal departments or other jurisdictions, and publications on employment of policy analysts).
• Literature review. We conducted a literature review of documents that provided further insights and perspectives on relevant staffing issues and policy capacity building challenges.
• Review of administrative and performance data. We obtained, reviewed and analyzed program administration and performance data from a variety of sources. We performed data analysis on 260 Recruitment of Policy Leaders alumni who had been hired by federal departments and agencies subject to the Public Service Employment Act between April 2006 and March 2018. We performed the data analysis in collaboration with the Data Services and Analysis Directorate within the Public Service Commission’s Oversight and Investigations Sector. Official data from the Public Service Commission captured 232 Recruitment of Policy Leaders hires during that period, and an additional 28 individuals were identified and included in this analysis by drawing on a GCpedia list of self-identified Recruitment of Policy Leaders program hires and an improved matching method. A detailed explanation of the methodology used in this report is provided in Appendix 2.
• Interviews. We interviewed 20 key informants, including deputy ministers, senior executives, Public Service Commission officials, hiring managers, Recruitment of Policy Leaders program secretariat members, candidates and mentors.
• Surveys. The Public Service Commission’s Evaluation Division collaborated with the Data Services and Analysis Directorate to conduct 2 surveys as part of this evaluation.
  • One survey was sent to 65 senior policy executives across the federal public service at the Assistant Deputy Minister, Associate Assistant Deputy Minister, and Director General level, with a response rate of 23% (15).
  • A second survey was sent to 574 former Recruitment of Policy Leaders candidates from the 2008–18 period, with a response rate of 43% (248). As shown in the following chart, 180 respondents were hired through the program, with 154 currently employed in the federal public service and 26 who left after being hired (see Graph 1).

Graph 1 – Candidate survey respondent population

4. Evaluation findings

21. This section presents the findings according to the 3 evaluation issues of relevance, effectiveness, and efficiency as outlined in the Treasury Board Directive on Results.

4.1 Relevance

Graph 2 – EC population (indeterminate and term) and new EC hires 2006–18 ^{Footnote 15}

26. A study on the state of policy capacity revealed a shortage of policy professionals with technical expertise.^{Footnote 16} In the study, 67.4% of survey respondents agreed there were such shortages. It found that the ability to keep up with the demand for policy professionals had fallen short, and that shortages in policy staff were becoming an issue. Strengthening policy capacity appeared to be an area where the federal public service could improve by focusing on recruitment initiatives.

Evolution of the federal public service policy capacity gap

27. The policy capacity gap identified in the 1990s focused on a number of areas. As summarized in the Fellegi-Anderson Report: “There are risks for the longer-term because of restraint and cutbacks. The community is not renewing itself as it should through recruitment and career development. These difficulties will require a strong focus on personnel issues by policy managers. Junior officers have expressed concerns that policy managers do not pay sufficient attention to personnel management.” ^{Footnote 17}

28. Ten years later, in August 2008, a report by Allen Sutherland found that there is a “generally acknowledged talent shortage in many parts of the policy community. Quality analysts are in high demand, as can be seen by the accelerated promotion rates and expectations.” The report also found that “all newcomers, including the best and brightest, are being hired primarily on their potential not their current capacity; however formidable their resumes, this potential needs to be fostered through additional training, but most growth will occur primarily through exposure, learning by doing, and conscious career development.” ^{Footnote 18}

29. In July 2016, the Policy Community Co-Champions issued their report to the Clerk of the Privy Council of Canada on the Policy Community Project. This report assessed the state of policy within the federal public service and identified recommendations to enhance, strengthen and transform the policy community. It identifies a shifting landscape within which policy is developed and implemented, and identifies a need to re-evaluate how policy work is conducted, as well as the tools that support the provision of policy advice. ^{Footnote 19} “These and other factors are creating new conditions and expectations for policy making and challenge the public service to be networked, agile and innovative to enable the continued provision of high-quality policy advice.” To address some of the challenges facing the community, the report makes a number of recommendations. One of the recommendations is for “fundamental changes, such as modernizing the recruitment of policy professionals with tools to effectively recruit entry-level, mid-career and senior-level professionals, establishing a community of practice and holding an annual policy workshop.”

30. Based on the analysis, there is a need to continue building federal public service policy capacity, and action is being taken. A number of recruitment programs and initiatives have been established to enhance policy capacity. These include:

• Recruitment of Policy Leaders program
• Advanced Policy Analyst Program – Government of Canada wide (funding partners)
• University Recruitment Program – Finance Canada
• Post-Doctoral Research Program (science-based departments)
• Policy Officer Recruitment Program (Department of National Defence)
• Policy Analyst Recruitment and Development Program (Natural Resources Canada)
• Treasury Board of Canada Secretariat Policy and Program Analyst Post-Secondary Recruitment Campaign
• Post-Secondary Recruitment Program – Policy, Economics and Social Sciences

31. The Recruitment of Policy Leaders program must be viewed as part of the overall federal public service policy capacity renewal effort. The program is designed to attract higher-level candidates who can be placed directly into mid-to-senior level policy positions. This is in line with policy community work that has been undertaken in recent years that identified a need for programs that can bring in policy analysts at all levels.

Are program objectives aligned with government priorities, roles and responsibilities?

32. The need to attract, recruit and retain high-calibre talent to enhance policy leadership and innovation has increased in recent years as part of public service renewal.^{Footnote 20} As shown in Graph 3,^{Footnote 21} in the federal public service as a whole there is a widening gap between employees who are over and those who are under 40 years of age. For the EC category, which contains most of the policy community, although the variance is not as pronounced as for the entire public service, the number of ECs who are under 40 years of age has decreased since 2013, while the number of those who are above 40 years of age has increased (see Graph 4 ^{Footnote 22}). This trend in EC category demographics demonstrates a need to ensure that renewal efforts are successful.

Graph 3 – Federal public service demographics (2006–18) ^{Footnote 23}

Graph 4 – EC category demographics (2006–18) ^{Footnote 24}

33. In addition to renewing the policy group in order to address capacity gaps, innovative approaches are required in order to recruit candidates with the right skills, as outlined in the 2016 policy community report to the Clerk. There was a consensus among interviewees that program hires are highly qualified, with strong skill sets and abilities from diverse educational backgrounds. It was indicated they bring fresh perspectives and new solutions to address increasingly complex public policy challenges, such as climate change, artificial intelligence, threats to democracy, and terrorism. Some remarked that the program aligns with public service priorities relating to renewal and retaining talent in Canada.

34. As well, according to the Canadian Council for Public-Private Partnerships, the government is increasingly delivering programs and policies in partnerships with non-government delivery agents, and such arrangements require complex skill sets and competencies. ^{Footnote 25} Some interviewees indicated that the program attracts people with policy expertise outside government (that is, people who worked closely with Canada research chairs or other innovative knowledge organizations), which helps them work within complex, multi-layered structures, and interact with a broad range of internal and external stakeholders.

35. The evaluation found that program objectives are aligned with Government of Canada priorities, roles and responsibilities. The program is aligned with the direction provided to public servants. In Blueprint 2020, the Clerk of the Privy Council stated that the public service must hire “the best and brightest people with the skills needed to develop evidence-based options and advice for the Government and to provide effective support to Canadians in times of change.” The Clerk also encouraged federal departments to find innovative ways “to ensure that the Public Service has the competencies and leadership skills needed to harness the best talent and the brightest ideas, wherever they may be found, to meet the evolving needs of Canadians.” ^{Footnote 26}

36. Innovative approaches to recruitment and development must be carried out in accordance with human resources legislation, regulations and policies. The current Recruitment of Policy Leaders program design meets the spirit of the government’s human resources management framework and the New Direction in Staffing, which was launched in 2016. The New Direction in Staffing provides the federal public service with flexibility “to customize their approaches to staffing, based on their own day-to-day realities.” ^{Footnote 27} The policy framework streamlines staffing rules and policies, and encourages managers to exercise their discretion when making staffing decisions, while meeting the simplified policy requirements in ways adapted to their organizations.

37. The Recruitment of Policy Leaders program is a targeted government-wide program that aims to recruit and retain exceptional graduates into mid to senior-level policy positions. These candidates have the potential to shape the future of Canada’s public policies landscape. As such, the program’s objectives support the Clerk’s call for a federal workforce equipped to contribute to the public service capacity to respond to a myriad of a global and complex social, economic, environmental and security challenges.

4.2 Effectiveness

Are qualified candidates placed in policy positions in the federal public service?

38. As one of the recruitment programs designed to enhance policy capacity, the Recruitment of Policy Leaders program is aligned with government priorities. The data that this evaluation has been able to obtain, review and analyze indicates that there are opportunities to improve program delivery to achieve results. The extent to which the program addresses the policy capacity deficit is unclear considering that the program’s “ability to fulfill this function is being jeopardized by the falling number of recruits who are successfully placed into positions within government.”^{Footnote 28} We will explore this question in this section of the report.

39. Each year, an average of 1 638 candidates submit applications; 37 qualify in partially assessed inventories; and 21 successful candidates are hired into mid- to senior-level policy positions. However, appointments have been declining since 2010-11 compared to the period from 2006–07 to 2010–11 (see Graph 5).

Graph 5 – Recruitment of Policy Leaders applicants, candidates and hires, 2007–08 to 2017–18

40. The decrease in appointments since 2011–12 reflects the broader federal public service trends of reduced recruitment during the Deficit Reduction Action Plan period and slowly increasing recruitment since that period. The question that should be considered is why annual partially assessed inventories of 27 to 56 highly qualified individuals have not led to more hiring during this time period. Is it due to job offers being turned down, or because job offers were never made? Our survey results indicate it is a combination of these 2 factors. But regardless of the cause, the decrease in the appointment of candidates from the partially assessed inventories raises questions about the extent to which the federal public service is benefiting from the program.

41. Available data shows that 6 departments hired more than half of all Recruitment of Policy Leaders alumni (see Table 2). Federal departments that employ a large number of ECs, such as National Defence, Transport Canada, and Statistics Canada, as well as regional-based federal organizations, have not been using the program to hire policy leaders.

Table 2 – Recruitment of Policy Leaders hires by department and agency from April 2006 to the present ^{Footnote 29}

Department or agency	Recruitment of Policy Leaders hires (2006–2007 to 2017–2018)
Global Affairs Canada (including Canadian International Development Agency)	50
Environment and Climate Change Canada	30
Employment and Social Development Canada	22
Indigenous and Northern Affairs Canada	20
Natural Resources Canada	15
Health Canada	15
Canadian Heritage	13
Public Health Agency of Canada	10
Innovation, Science and Economic Development	9
Immigration, Refugees and Citizenship Canada	9
Public Safety Canada	9
Privy Council Office	8
Agriculture and Agri-Food Canada	6
Treasury Board of Canada Secretariat	5
Finance Canada (Department of)	5
Canada Border Services Agency	5
Justice Canada (Department of)	5
Fisheries and Oceans Canada	4
Infrastructure Canada	3
Immigration and Refugee Board of Canada	2
National Defence (public service employees)	2
Transport Canada	2
Canada School of Public Service	1
Civilian Review and Complaints Commission for the RCMP	1
Office of the Chief Electoral Officer	1
Offices of the Information and Privacy Commissioners	1
Public Services and Procurement Canada	1
Other departments and agencies	6

Graph 6 – Survey respondent category at time of appointment

44. For the 15 survey respondents who obtained federal public service jobs but not through the Recruitment of Policy Leaders program while in a partially assessed inventory, 50% indicated they obtained employment outside of the policy domain and were appointed into various occupational groups, such as research, commerce officers and law practitioners. This attests to the fact that the program attracts multidisciplinary individuals who may be suited to a variety of positions within the federal public service.

45. Approximately 85% of the 180 alumni who responded to our survey believed they were equipped to conduct policy work at the time they were hired. Most alumni interviewed observed they needed between 6 months to a year to understand the federal government environment in order to be fully effective. In general, while alumni believed they had the tools to conduct policy work upon initial hire, it was acknowledged that there is a learning curve for new recruits, as they adapt to working in the federal public service.

46. Alumni survey respondents identified their current position at the time they completed the survey. The job classification and level distribution of the 154 survey respondents indicates a concentration at the EC-6 level (54), followed by the EC-7 (38) and EX-1 (17) levels. In addition, approximately 11% of respondents occupy positions in a variety of other job classifications, such as administrative services, program administration, legal, and foreign service (see Graph 7).

Graph 7 – Alumni survey respondent category as of December 2018

Retention and support

47. From 2006–07 to 2017–18, the program retention rate was 79%, which is comparable to the retention rate for the EC-5, -6 and -7 control group at 79%. This retention rate exceeds the initial target of 50% set when the program was established. It is also interesting to note the consistency in retention between the program alumni and the EC-5, -6, -7 control group before and after 2012 (see Graph 8).

Graph 8 – Comparison of Recruitment of Policy Leaders and overall EC-5, -6 and -7 retention rates

48. Twenty-six survey respondents had accepted a Recruitment of Policy Leaders position and later left the federal public service. Approximately half of those who resigned identified work environment, administration and lack of career progression as main contributors to their decision. These respondents identified issues such as difficulty in balancing career progression and field expertise and believed they needed to become generalists in order to advance their federal public service careers. The other half left to pursue other employment opportunities due to a lack of suitable employment outside the National Capital Region or internationally, and due to having a leave without pay or educational leave expire.

49. The survey results showed that alumni had differing views on the career support they received once they joined the federal public service. Half of the 154 alumni survey respondents who are still working in the federal public service were very satisfied or extremely satisfied with the support they received, more specifically: achieving full potential (55%), professional development (56%) and guidance and supervision (48%). Those who were unsatisfied with these elements of their experience with the program identified training and mentoring as areas for improvement. These subjects are discussed in the efficiency section of this report.

Do Recruitment of Policy Leaders alumni exercise leadership on the public policy framework?

50. In order to examine the extent to which alumni exercise public policy leadership, the evaluation focused on 2 elements: the program’s ability to attract individuals who will become future leaders and the degree to which alumni influence and shape Canada’s public policy positions to address emerging national and global challenges.

Management and leadership roles

51. The evaluation assessed the management and leadership roles alumni have in the federal public service by analyzing the group’s career progression. While promotion to the executive level was not a stated goal of the program at the beginning, the number of Recruitment of Policy Leaders promoted provides an indicator of whether alumni are taking on policy leadership roles. We obtained data from 2 sources to perform this analysis given there was no overall data management strategy in place to administer the program. The first was through Public Service Commission information systems, which contain data for departments and agencies that operate under the Public Service Employment Act. The second data set was obtained through the survey, which included responses from 154 program alumni currently working in the federal public service.

52. The data demonstrates that alumni are being promoted after initial hire. 64% of the 154 alumni survey respondents currently working in the federal public service have obtained promotions. Since 2006–07, approximately 1 in 10 alumni have obtained executive level positions (see Graph 9). This is slightly higher than the 6.4% for EC-5, -6 and -7 control group employees. However, it is taking alumni more time to attain executive level positions than their control group peers (see Graph 10).

Graph 9 – Comparison of Recruitment of Policy Leaders to control group promotions to executive

Graph 10 – Comparison of days to promotion, Recruitment of Policy Leaders versus control group

53. The results demonstrate that the program is attracting qualified candidates who can advance in their federal public service career and obtain leadership positions.

54. Graph 11 provides an overview of the current occupational category and level of the 260 alumni who were hired since 2006–07. The data demonstrates that a number of alumni have become executives and the rate of promotions is consistent with the data obtained in the survey of applicants. Also, the data confirms that alumni are primarily employed in the EC classification.

Graph 11 – Group and level of the 260 Recruitment of Policy Leaders alumni as of December 2018

55. A cohort analysis on promotions was performed for alumni hired from 2006–07 to 2017–18 and compared to the EC-5, -6 and -7 control group. The results demonstrate that Recruitment of Policy Leaders cohort performance was higher than the control group during the period from 2006–07 to 2009–10. The data indicates that until the 2009–10 cohort, Recruitment of Policy Leaders recruits had an overall performance advantage in terms of reaching the EX-1 to EX-3 levels (see Graph 12 and 13). However, after this cohort, the control group has been more effective in reaching the executive ranks. The high performance of earlier Recruitment of Policy Leaders cohorts could be attributable to the fact that they have served longer in the federal public service.

Graph 12 – Comparison of EX attainment per Recruitment of Policy Leaders cohort

Graph 13 – Comparison of EX attainment per control group cohort

Influencing and shaping Canada’s policy positions

56. In general, most interviewees recognized that there is a learning curve for new recruits as they adapt to the federal public service work environment, but they have the competencies and abilities to make a contribution. One interviewee said that candidates are selected based on merit, skills and understanding of public policy issues. Once the transition/adaption period is over, they make an immediate impact on policy development.

57. Program alumni occupy a variety of job categories and levels and contribute to a wide range of public policy files including: national security, foreign policy and capacity building in other countries, public pensions and income security, social innovation, immigration, justice, and Indigenous issues such as reconciliation. The 154 alumni survey respondents who currently work in the federal public service identified the types of policy they were working on. When broken out by the current Cabinet committee structure, ^{Footnote 30} there is an even distribution of policy areas that alumni have contributed to (see Graph 14). This demonstrates that alumni conduct policy work that covers the full spectrum of government priorities.

Graph 14 – Policy areas worked on by Recruitment of Policy Leaders alumni

58. One key informant summarized program alumni contribution to, and influence on, federal public policies in the following words: “When there are critical policy issues, there are always alumni working on these files, because of their ability for community building and achieving results. For example, alumni have founded Policy Ignite, which has reached thousands of people, so their contribution is very significant.” Also, program alumni organized a Policy and Leadership Conference to convene the Recruitment of Policy Leaders program community on topics of shared interest.^{Footnote 31} This included an agenda item on the future of the program and the future of work.

59. Based on the evaluation survey results, 58% of the alumni indicated they strongly agreed that they influence and shape Canada’s public policy position (see Graph 15). One alumni stated in testimonials that the most significant and exciting role they had was representing Canada abroad. This person reported being proud to receive unexpected thanks on behalf of Canada for contributions by the Canadian government and/or Canadian organizations and individuals that had a positive impact on the lives of people abroad. Other participants mentioned their excitement and pride to sit in on Cabinet meetings and provide advice to Clerks of the Privy Council and Prime Ministers.

Graph 15 – Alumni assessment of their impact on influencing and shaping policy

60. Half of the senior policy executives who responded to the evaluation survey believed that alumni were exercising leadership and influencing and shaping Canada’s policy positions. This is consistent with information provided by one key informant who claimed that half of the federal public service deputy ministers supported the program at its creation.

Do program hires make a meaningful contribution to federal public policy?

61. The evaluation found that 73% of the 15 senior policy executives who responded to our survey believed that the program contributes to building federal public service policy capacity; 26.7% did not know, or did not believe, that the program made a significant impact. Senior policy executives highlighted in interviews and survey comments that when compared to other new recruits and employees, program hires have skills conducive to high quality policy work, and exhibit greater knowledge of the federal government machinery, including the public policy development process, and the expenditure management system.

62. Senior policy executives recognized the program and its alumni as having a well-established reputation within the federal government. Almost all key informants (senior policy executives, hiring managers, program alumni) interviewed were satisfied with the program’s performance and believed that the program had been successful in recruiting high potential individuals and retaining them for policy roles.

63. Program applicant survey respondents strongly supported the program, regardless of whether they were hired. 99% of the 180 alumni survey respondents and 79% of the 68 survey respondents who were not offered jobs indicated that they were in favour of the program. Written comments included in survey responses identified the following program strengths, in addition to attracting high-calibre talent:

• it allows candidates to enter the federal public service at the mid or senior policy analyst level, which opens doors for applicants who are mid-career
• the selection process, once a candidate has been qualified in an inventory, is more flexible and efficient than regular staffing processes
• the Recruitment of Policy Leaders Network provides opportunities for career development, leadership and promotion

64. Overall, both senior policy executives and alumni who responded to the evaluation surveys had positive views on how the program had made a meaningful contribution to enhancing the policy capacity of the federal public service.

Is the program aligned with federal public service diversity objectives?

65. Canada has a legislative framework that supports diversity and inclusion (the Canadian Human Rights Act, the Employment Equity Act, the Canadian Multiculturalism Act, and the Official Languages Act). The final report of the Joint Union/Management Task Force on Diversity and Inclusion, released in 2017, expresses the Government of Canada’s commitment to building a diverse public service that reflects Canadian society and that is a model of inclusion for employers. The 2017 report stresses that “In successful organizations, diversity and inclusion are not optional: greater diversity and inclusion enable organizations to leverage the range of perspectives needed to address today’s complex challenges.” ^{Footnote 32} Also, fostering diversity and inclusion, and embedding them in all levels of the public service, are among the Clerk’s priorities.^{Footnote 33} The goal is to build a federal public service that reflects Canada’s diverse population by attracting and retaining the best talent from all cultures, identities and abilities across generations.

Employment equity groups

66. Overall, alumni representation of 3 (women, members of visible minorities, and persons with disabilities) of the 4 employment equity groups matches or exceeds labour market availability rates in Canada (see Graph 16). The exception was the hiring of Aboriginal peoples (3.1%), which was slightly lower than the availability of 3.4%. The greater concentration of positions in the National Capital Region (see section 71 and following on regional representation) could be one of the explanations for the under representation of Indigenous people in the program.

67. There are many good initiatives that could be considered for improving diversity and inclusion within the program. One of them is the Indigenous student ambassadors’ initiative at Crown-Indigenous Relations and Northern Affairs Canada and Indigenous Services Canada. The Clerk’s 2018 report to the Prime Minister showcased this initiative, which seeks, among other things, to “promote student hiring programs and assist candidates with the application process.” Other examples include the Ontario Internship Program, a comparable recruitment program for Ontario’s public service, which conducts an annual demographic survey to ensure the program aligns with Ontario’s public service strategic initiatives related to workforce representation.

Graph 16 – Recruitment of Policy Leaders employment equity representation rate comparison

68. The results of the evaluation’s gender-based analysis (GBA+) demonstrate that female alumni are performing well relative to their male counterparts and the overall EC control group (see Graph 17). Furthermore, female alumni reached the EX level quicker than the control group, 2 543 days on average versus 2 101 days (see Graph 18). However, the data also demonstrates that female alumni who self-identified as members of visible minorities were offered employment at the EX minus 3 level (EC-4) or lower in higher proportions (12.5.%) than alumni who had not self-identified as being part of an employment equity group (4.5% for all women and 3.2% for men).

Graph 17 – Alumni promotion to executive level by sex

Graph 18 – Alumni days to obtain an executive level position by sex

Official languages

69. The evaluation found the program was not performing as well as the overall public service in terms of employing Francophone Canadians. Data obtained revealed that the representation of Francophone Recruitment of Policy Leaders hires is higher than Francophones in the control group, but lower than Francophones in federal government departments and agencies subject to the Public Service Employment Act (see Graph 19). The representation of Anglophone Recruitment of Policy Leaders hires is higher than their representation in the federal public service and lower than the control group.

Graph 19 – Alumni first official language

70. In terms of bilingualism, data on second language evaluation shows that 55% of alumni whose first language is English have an official language profile of BBB or higher in French, which is similar to 57% for the EC-5, -6 and -7 control group. The proportion of Francophone alumni with an official language profile of BBB or higher in English is 75% compared to 85% for the control group.

Regional representation

71. The evaluation found, based on data obtained for 198 of 260 program hires from 2006–07 to 2017–18, that the representation of recruits from universities in Ontario, Quebec and Nova Scotia was consistent with national post-secondary student enrolment data. To further diversify recruitment, an opportunity was identified to increase outreach efforts across the country to ensure that students from across Canada are aware of the program. For example, based on 2016–17 university enrolment figures, there is a need to attract more candidates from provinces other than Ontario and Quebec, particularly from western Canada. Furthermore, the fact that 38.4% of the 198 alumni obtained their highest level of education abroad is consistent with Statistics Canada data showing that 4 out of 10 Canadian students obtained their PhDs from universities outside Canada (see Graph 20).

Graph 20 – Location of study for 198 of 260 alumni from 2006–07 to 2017–18

72. In terms of location of hire, the evaluation found that nearly 97% of alumni obtained positions within the National Capital Region. This compares to 89.6% for the EC-5, -6 and -7 control group who are employed within the federal public service. Greater concentration of Recruitment of Policy Leaders program positions in the National Capital Region may limit opportunities for less mobile candidates. In fact, a number of interviewees and survey respondents indicated that some alumni left the federal public service because they did not have the opportunity to work in a regional setting. This being said, several key informants noted that most policy positions are within the National Capital Region, and that it may be challenging to increase the number of policy positions in the regions, which are more focused on service delivery to Canadians.

4.3 Efficiency

73. This section provides an assessment of the current governance and program delivery mechanisms in place to support Recruitment of Policy Leaders administration.

Are appropriate governance structures and processes in place to support the effective delivery of the Recruitment of Policy Leaders program?

Program governance

74. As multiple stakeholders are involved in delivering the program, it is important that each has a clear understanding of their roles, responsibilities and accountabilities with respect to ensuring the success of the program.

75. There are numerous actors involved in delivering the program. These include Public Service Commission officials, program Executive Committee members, alumni, the Deputy Minister Champion, hiring managers, and human resources personnel. Given the large number of officials involved in the administration and delivery of this program, the program could benefit from a formal memorandum of understanding clearly identifying roles and responsibilities to support the administration and delivery of the program.

76. The program was designed as part of the solution to support the renewal of policy capacity in the federal public service. There is a need for proper coordination of all activities that support the program to ensure that it achieves success. The evaluation found that the current governance model could be enhanced. Survey respondents and key informants mentioned that the program may become unsustainable due to a heavy reliance on volunteers, the absence of a dedicated team to plan and administer the program, and the lack of permanent funding.

77. One important area where governance improvements are required is data management and performance measurement. At the beginning of the evaluation, data and information that could be used to support program decision making and measure success were not centrally located. As a result, the data generated for this evaluation came from multiple sources and systems. The evaluation team was informed throughout the project that there are data integrity issues and that complete information — on number of hires, hiring organizations, current location of alumni and complete program costs — was difficult to obtain.

78. There were 2 good practices identified in our research that may be of value to program leadership as they contemplate governance changes:

• the 3-part governance structure under the Advanced Policy Analyst Program includes a steering committee, an alumni advisory board and a secretariat ^{Footnote 34}
• the policy community is supported by 2 champions appointed by the Clerk of the Privy Council and a small office led by a director ^{Footnote 35}

Alumni network

79. The alumni network is viewed as an important program feature and was identified as the top program strength by 32% of the 154 alumni currently working in the federal public service who responded to our survey. It was also listed as one of the top 3 strengths of the program by 14% of survey respondents who were not alumni. These results demonstrate the significant role that alumni have in sharing information about career opportunities and progression. However, comments raised in the survey and interviews stressed that the network could be improved and used more effectively.

80. Some potential improvements mentioned by respondents were to strengthen the network’s capacity to support new participants and existing alumni learning and development. Some survey and key informant comments included the potential to partner with other policy community initiatives (for example, the Advanced Policy Analyst Program), and to make program decision making more transparent and inclusive.

81. These improvements were seen as important to respondents, because the continued success of the program is heavily reliant on a robust alumni network. The network is the first place many alumni turn to for advice on career progression, learning opportunities and overall support. This has evolved over the years due to the lack of structured support provided to alumni once they are hired into the federal public service.

82. In fact, 24% of the 154 alumni survey respondents still working in the federal public service identified the lack of professional development possibilities (training and mentoring after appointment) as one of the main areas where program delivery could be improved. Other specific areas include improving new hire on-boarding, increasing access to second language training, establishing formal mentoring partnerships post-hire, as well as creating more possibilities for career progression, which could include formal or informal management training.

83. The lack of consistent support to alumni for on-boarding and training was also raised as a key program delivery challenge by senior policy executives during interviews. One key informant stated that there was a need to support program recruits by better managing their expectations, matching them with work they are qualified to deliver on by placing them in the right organization, and providing them with the right training. Other key informants and survey respondents believed that alumni should not be provided preferential treatment such as second language training for career advancement, and should be treated in the same way as other public servants with potential who have not come through the program.

84. One approach identified for consideration to further develop alumni leadership competencies was to include rotational assignments to a central agency as part of the program. Policy work requires on-the-job experience, policy-course specific training, as well as research and analytical skills developed at university. Rotational assignments to one or more central agency may help alumni develop and improve skills that are essential for the federal public service, such as inter- and intra-disciplinary collaboration, interpersonal skills and relationships with various stakeholders. The evaluation also found that the competency framework currently being developed by the policy community is a tool that alumni should use to support their professional development. ^{Footnote 36}

85. With regard to training and development opportunities, the Cross-functional Policy Mobility Program and Canada’s Free Agents programs are 2 recent innovative initiatives that could potentially support alumni in their pursuit of professional growth. The Cross-functional Policy Mobility Program is an 18-month program that is managed by the policy community and designed to help policy analysts gain experience in different policy functions across the public service.^{Footnote 37} Participants are provided opportunities in areas where they have no prior work experience.

86. As an experimental model, Canada’s Free Agents program offers mobility and flexibly to federal public servants who are interested in contributing to innovation and problem-solving methods and practices that would help address increasingly complex and rapidly evolving challenges.^{Footnote 38} Free agents are offered lateral assignments to work on files that match their skills and interests. Before being accepted in this program, candidates are assessed against “a set of 14 attributes that are internationally recognized as useful for public sector problem solving, such as empathy, action orientation, team orientation, resiliency and outcomes focus.” ^{Footnote 39} These are 2 examples of existing programs that alumni have access to, which could assist in their personal and professional growth and development.

Mentors

87. Mentors are alumni who volunteer to support successful candidates in partially assessed inventories. They play a central role in the recruitment and placement processes. In fact, volunteerism is one of the core foundations of the original program.

88. While mentors distinguish the program from other federal recruitment initiatives, the program’s reliance on volunteers was described by survey respondents and key informants as a source of both strength and weakness. For some, the mentor model is effective and should be kept as a cornerstone of the program. Mentors are seen as playing a key role in recruitment activities (screening and interviews), placement processes for people in partially assessed inventories, and provide support to new alumni as required.

89. 72% of respondents to the candidate survey had positive experiences with their mentors. Conversely, almost 20% did not have positive experiences, and opportunities for improvement were identified in both written survey comments as well as during interviews (see Graph 21).

Graph 21 – Recruitment of Policy Leaders survey respondent satisfaction with the mentoring process

90. The 20% of survey respondents who were dissatisfied with their mentor experience had observations on this issue that were summarized by the following comment: “the program is only as good as its weakest mentor.” The methods used to find matches for people in the partially assessed inventories may not be the best approach for these excellent candidates to obtain policy jobs in the federal public service in their area of expertise, or where their talents could be used to advance Canada’s policy agenda. The survey results showed that the mentoring model was an area for improvement for both alumni (19%) and former candidates who are not alumni (25%). For example, it was suggested that there may be better ways to match the skillsets between the mentors and candidates, and that service standards could be developed to ensure that mentors respond to candidate needs in a timely manner.

91. The evaluation found that overreliance on volunteers during periods of fiscal restraint such as the Deficit Reduction Action Plan may affect mentors’ availability to properly help candidates in partially assessed inventories to obtain employment. A number of senior policy executives felt that the current program structure has limitations and that it may have reached its capacity to properly place qualified individuals into policy leader positions. Particularly, the reliance on volunteer mentors to support applicant screening, interviews and the placement of candidates into federal public service positions is perceived as an issue. Some key informants also raised concerns about the impact that a potential increase in the number of partially assessed candidates could have on finding mentors, and on the number of people hired under the program.

92. Some senior policy executives stated that the reliance on mentors at the working-level was not the best way to match highly qualified candidates and get them hired into the right positions. As well, the program’s reliance on individuals rather than institutions may negatively affect the sustainability of the program and its functioning, especially in cases when there is a sudden departure of key individuals.

93. The evaluation also found that, over time, mentors have become less engaged. This was seen as being due to the fact that mentors are volunteers, and their support for the program may not be reflected in their performance agreements. Many survey respondents mentioned that there were mentors who did not appear to have time to focus on helping them get a job. Often times they waited days to hear back from their mentors and ultimately felt that they could have been better supported. Some interviewees also reported that the lack of engagement may be more pronounced with alumni who are executives, because they may no longer have time to volunteer and properly support candidates in pre-qualified inventories.

94. Finally, there is limited capacity within the Public Service Commission and the Recruitment of Policy Leaders program secretariat to fully support mentors and enhance the overall effectiveness and efficiency of the program. There is an opportunity for the program to evolve and become more structured administratively to improve the support provided to both mentors and candidates in partially assessed inventories. The establishment of a full-time secretariat, either within the Deputy Minister Champion’s office or the policy community initiative, was presented as a measure that could mitigate governance risks and provide stability over program administration, which would lead to improved program results.

Are there better, alternative ways or lessons learned (for example, design changes, cost) for achieving the same results?

Program funding structure

95. The program is unique in its structure and approaches, and is not administered like other federal recruitment programs. Unlike the Advanced Policy Analyst Program, funding is not based on a cost-sharing ratio by participating organizations. The Public Service Commission funds its administration of the program, and the Deputy Minister Champion’s office allocates a certain amount to support program activities. Instead of a central budget holder overseeing complete program costs, individuals within a number of departments contribute to recruitment and appointment efforts, and many of these costs are non-monetary in nature and viewed as opportunity costs of participants’ time. It is difficult to identify the program’s full cost due to the fact that the funds expended to administer and deliver the entire program are not centrally tracked.

96. Program administration costs consist of several components, both within and outside of the Public Service Commission. Some costs are readily identifiable, while others are not. Program-related costs identified in Appendix 1 do not reflect full program delivery costs. There are additional costs that are not tracked. These include the time devoted to program administration and delivery allocated by the Deputy Minister Champion, alumni (including mentors), senior executives and the alumni who volunteer as part of the recruitment, screening and interview processes. The evaluation estimates that the annual value of time spent by assistant deputy ministers on the program is approximately $250,000. The estimated annual value of the time spent by alumni administering and delivering the program is approximately $315,000.

97. Consultations with stakeholders revealed that permanent funding has been a longstanding issue, which has not been fully resolved since the 2008 evaluation that concluded that the program needed permanent funding and should be hosted within a federal organization. The current evaluation also found that the program should be housed in a department or functional area that has a connection to the broader policy community given the importance of the potential for the program to contribute to federal public service policy capacity renewal. At the same time, the Public Service Commission should continue playing its role as a partner in renewing the policy community. Several interviewees indicated that permanent funding would allow for the creation of a secretariat, preferably filled by alumni, to provide strategic support to the Deputy Minister Champion, the Recruitment of Policy Leaders Executive Committee and alumni network.

Program administration – the hiring processes

98. The program has evolved over the past 10 years. The original design to attract top-talent remains intact, however there has been a shift in the types of program applicants and in the way that the program is delivered.

99. Half of the 248 survey respondents indicated that they would have considered the federal public service as a career choice even if the program had not existed. And only 29% indicated that they would not have. This proportion is higher than the 18.5% of those who stated that they would not have considered the federal government if they had not heard of the Advanced Policy Analyst Program. ^{Footnote 40}

100. As well, 32% of survey respondents had prior federal public service work experience, mostly through indeterminate, term or casual employment. A number of respondents also mentioned having federal experience gained through student employment programs (see Graph 22). This finding is not consistent with the design assumption that the program attracts people who would not have normally considered working in the federal public service. This demonstrates how the previous public service experience of people applying for the program has evolved over time.

Graph 22 – Breakdown of survey respondents with previous federal public service work experience prior to applying

101. The program recruitment and selection process was identified by a number of interviewees as an important factor that contributes to the program’s positive and well-established reputation in universities across the country and among federal public service hiring managers who use the program. The selection process was described by 14% of the alumni and 20% of the former candidates surveyed as a key program strength. Respondents considered it to be relatively more flexible and efficient than other staffing process mechanisms. This said, several key informants reported that the hiring process is too long and that good candidates find jobs elswhere by the time they qualify for the partially assessed inventories.

102. A number of interviewees emphasized that the program recruitment process is not currently designed to test all of the skills that potential managers need, such as the ability to supervise, develop employees and work collaboratively in a team environment. The evaluation team learned that some managers were reluctant to hire Recruitment of Policy Leaders candidates in partially assessed inventories for senior policy positions, because they believed these candidates might not have the required management skills. When c are hired at the EC-6 and EC-7 levels, they may not benefit from the development aspects of growing through the levels. A number of key informants suggested that the program consider 2 recruitment streams: one for candidates who want to excel as policy analysts in their chosen field; and another for candidates interested in growing into management and executive level positions.

103. The evaluation found that alumni generally managed their careers on their own or in collaboration with their immediate managers. Thus, when compared to other recruitment programs such as the Advanced Policy Analyst Program or the Policy Analyst Recruitment and Development Program, alumni were not benefiting from support structures to navigate their careers through training and development opportunities and short-term assignments. The lack of overall support mechanisms for alumni is a factor in the outcomes that have been achieved by program cohorts, particularly since 2010.

104. A number of survey respondents identified screening and recruitment practices as key program weaknesses. This may be due to concerns raised by key informants and survey respondents about the use of mentors, the fact that the program is supply- rather than demand-driven, and time to staff. The survey identified time to staff as an area that could be improved to meet candidate expectations. It is important to note that almost 73.5% of Recruitment of Policy Leaders appointments occurred within candidates’ first year of eligibility in pre-qualified inventories. However, 15% of Recruitment of Policy Leaders hires took more than 1.5 years (see Graph 23). This was due in part to the fact that some people in partially assessed inventories were completing their studies and unable to start working immediately.

Graph 23 – Recruitment of Policy Leaders hiring timeline survey results

105. It is important to note that the program was not designed to be demand-driven, and as a result, forecasting to identify the number of available positions for new recruits has been inconsistent. The approach has been to place exceptional candidates in partially assessed inventories, and match their skills to jobs where they can advance policy solutions. An alternative program delivery approach used in recruitment programs in other jurisdictions (the Ontario Internship Program, the Manitoba Public Service Leaders in Training Program, and the Presidential Management Fellows for the United States Public Service) offers a 2-year internship leading to a permanent or term position. In the case of the Ontario Internship Program, the placement rate following the internship is approximately 80%.

106. The current supply-driven model depends greatly on:

• hiring managers’ knowledge of both the program and methods to appoint people from partially assessed inventories
• the quality of the candidates in the partially assessed inventories
• available funds to hire at any point during a fiscal-year

The current model also has an impact on the highly qualified individuals who in many cases remain in partially assessed inventories for a long time, affecting their career decisions. These factors have contributed to the program’s overall level of success in placing candidates in departments and agencies.

107. The evaluation found that consideration could be given to assessing demand before launching the recruitment process. Both the Advanced Policy Analyst Program and the Internal Audit Recruitment and Development program use this approach. Assessing potential demand would increase awareness of the program, keep it top of mind for hiring managers when they complete their annual planning exercises, and potentially increase the number of qualified people who are hired through the program. This could also contribute to better managing applicant expectations, which several interviewees raised as an opportunity for improvement.

108. Senior policy executive survey respondents suggested re-evaluating and modernizing the way that candidates are supported in finding jobs once they are in the partially assessed inventory. Technology could be used to support program administration and operations in placing successful candidates, in program promotion and recruitment, and in monitoring and reporting. Recruitment processes should be renewed to keep up with technological advances and to focus on a new generation of tech- and social media-savvy candidates.

109. While the program helps renew policy capacity to a certain degree, there is more that can be done to strengthen the program’s role within the policy community as a key vehicle to attract, recruit and retain policy leaders. There is a risk that if the program continues to operate as currently administered and action is not taken to address the issues identified in this report, the program may become less relevant.

Program marketing and promotion

110. The evaluation found areas for improvement in terms of program outreach and marketing. Candidate survey results (see Graph 24) indicated that the most important sources of information about the program were word-of-mouth (35%) followed by Government of Canada websites (31%).

Graph 24 – Recruitment of Policy Leaders program applicant responses to how they first learned about the program

111. The survey also found that senior policy executive level of involvement with the program was low, as 60% of the 15 respondents had not been previously involved in the program. In terms of those executives who knew about the program, one survey respondent summed up the situation: “There hasn't been much communication on this issue (Recruitment of Policy Leaders program) within my organization (or perhaps at the government level). This is the first time I have actually clicked on the link to the website.” The current level of awareness of senior policy executives about the program indicates that there is room for improvement in how the program is marketed across the federal public service. While most interviewees agreed that the deputy minister breakfast presentation should continue in order to maintain program awareness at that level, they suggested that the program be promoted at every level, from assistant deputy ministers to hiring managers to human resources advisors across departments.

112. In terms of program awareness, 65% of senior policy executive survey respondents had a good understanding of the program, 21% had some understanding of the program, and 14% did not have an understanding of the program. This level of awareness suggests there is room for improvement in marketing the program to senior executives and key stakeholders as a means to further enhance federal public service policy capacity. This is important given that the program’s goal is to bring top talent into mid to senior level policy positions to benefit from their expertise and diverse skill set.

113. Most senior policy executives interviewed suggested there may be a need to enhance internal communications to senior executives across the policy community to increase awareness of the program. They were aware that no other recruitment program recruits Canadian graduates who have been awarded significant competitive scholarships (such as the Rhodes, Commonwealth and Fulbright scholarships) and who demonstrate exceptional academic and employment achievements. Internal communications should also stress the potential and the quality of the individuals in inventories, and the value that Recruitment of Policy Leaders candidates bring to the Government of Canada. As well, this outreach should target the human resources community, so that they can assist hiring managers in human resources planning and recruitment initiatives.

114. The way that respondents to the candidate survey first learned about the program suggests that the external outreach approach should be rethought in order to increase program success. Some interviewees stressed that a more structured and fully marketed program would ultimately be beneficial for both hiring departments and potential candidates. In particular, it would ensure that a wider breadth of high-calibre individuals are aware of the program, and thus possibly apply to work for the federal public service. As well, approximately 10 years ago, the program undertook an environmental scan to identify where Canadians were studying abroad. There may be value in renewing this exercise.

5. Conclusion

115. The program began as a pilot in 2001 and became permanent in December 2004. It was developed to address 2 particular challenges at the time — a competitive market for top graduates and a need to re-build federal public service policy capacity after a decade of decline. Between 2006–07 and 2010–11, an average of 31 recruits were hired per year. The placement rate of successful candidates in the partially assessed inventories was 67% and the promotion rate of alumni to executive levels has been strong.

116. Since 2011–12, the program has not achieved the same levels of success. Between 2011–12 and 2017–18, an average of 15 recruits were hired per year, with a low of 8 in 2013–14. Federal budget reductions implemented in 2011–12 and 2012–13 may have affected the program’s ability to generate employment opportunities for those in the inventories. The placement rate of successful candidates was 43%, and the promotion rate of alumni to executive levels lagged behind the EC-5, -6 and -7 control group.

117. The program’s operating environment has evolved over the past 15 years. In the past, the program sought candidates who would not have considered working in the federal public service. Today’s applicants have previous federal public service work experience, and the evaluation survey results indicate that half of the respondents would have considered applying for federal public service work even if the program had not existed. The retention rate is also increasing. The original program goal was to achieve a retention rate of 50%. Over the years, the retention rate has risen from 73% between 2006 and 2011, to 92% from 2012 to 2018.

118. The program’s governance was also affected by federal budget reductions in 2011–12. When the program was launched in December 2004, a Public Service Commission budget of approximately $700,000 was planned to deliver the program. Over time, the funding levels decreased. In 2011–12 the Public Service Commission decided to include the program as part of its budget reductions, which led to the current level of funding and support to administer the program.

119. The absence of a formal governance structure, the lack of a stable funding model, and a general low level of awareness of the program among senior policy executives has affected program design, delivery and success. These factors, combined with a lack of information on program performance, have contributed to the decreased success of the program, particularly as it relates to the hiring of successful candidates from the partially assessed inventories.

120. While program performance has not been optimal since 2011–12, there remains a need for this type of program as part of the overall recruitment of policy analysts and leaders to support continued efforts to renew and strengthen the federal public service policy community. However, there is a risk that if the program continues to operate as currently administered and action is not taken to address the issues identified in this report, the program may become less relevant.

121. The 2016 Policy Community Report to the Clerk of the Privy Council identified the challenges facing the policy community and the efforts that are needed to address them. The Recruitment of Policy Leaders program is an important component of one of the action items identified to address recruitment challenges to address capacity. For this reason, it is important that the program update its value proposition, and solidify its role within the policy community continuum. There is a great opportunity to leverage work already done by the policy community, such as the development of the competency model for policy analysts and the implementation of the Cross-functional Policy Mobility Program, which provides participants with short-term assignment opportunities in areas where they have no prior work experience.

6. Recommendations

122. This report makes 6 recommendations, designed to address the governance, administration, data management and performance issues that have existed for some time. We also recommend that in 2022–23, stakeholders conduct a thorough follow-up of the implementation of the actions developed to address the recommendations.

123. Recommendation 1: It is recommended that a formal governance structure be established that includes a lead department or functional area that is responsible for administering the program with the support of the Public Service Commission. This should be formalized in a memorandum of understanding memorandum of understanding signed by all relevant stakeholders.

124. Recommendation 2: It is recommended that the program intent and value proposition be reviewed and updated to ensure that program officials understand the current operating environment, that the program remains relevant and that highly qualified Canadians are hired.

125. Recommendation 3: It is recommended that a formal data collection and performance measurement strategy be developed that standardizes the way data is collected, captured and analyzed to support evidence-based decision making and performance reporting. The strategy should be included in the memorandum of understanding established for the program.

126. Recommendation 4: It is recommended that an outreach strategy be developed to inform policy executives, hiring managers, and human resources professionals of the benefits of the program. This should include consultations prior to the annual recruitment campaign to support forecasting potential positions within hiring organizations and the resources required to manage the annual recruitment campaign.

127. Recommendation 5: It is recommended that alumni mentors be provided training and on-going support to fulfill their roles and responsibilities related to helping candidates in partially assessed inventories obtain federal public service policy positions.

128. Recommendation 6: It is recommended that an orientation/on-boarding program and support structure be established for alumni to help them navigate their federal public service career. The support structure could include post-hire mentors, rotational assignments and specific training activities.

7. Management response and action plan

Management response and action plan
Recommendation	Response and planned action	Office of primary interest	Estimated completion date
Recommendation 1: It is recommended that a formal governance structure be established that includes a lead department or functional area that is responsible for administering the program with the support of the Public Service Commission (PSC). This should be formalized in a memorandum of understanding (MOU) signed by all relevant stakeholders.	A Recruitment of Policy Leaders (RPL) Secretariat will be established (1.5 Full Time Equivalents [FTE]), funded by the DM Champion (1FTE) and the Canada School of Public Service (0.5 FTE plus associated overhead) for a two-year period. The RPL Secretariat will report to an RPL Executive Advisor and will be housed within the Canada School of the Public Service ( CSPS). As part of the governance, the PSC will continue to support the hiring process and the RPL secretariat and alumni will maintain and enhance their current roles. The RPL Secretariat, in collaboration with the RPL Executive Committee, PSC, and Policy Community will develop an MOU, signed by all parties, that documents governance, funding, data and performance measurement collection and reporting strategies. An assistant deputy minister (ADM) advisory panel will be established to provide advice on the annual recruitment campaigns and be part of the overall governance framework.	The RPL Executive Committee will work with the CSPS to establish an RPL Secretariat.   The RPL Secretariat, will take the lead on this action item. The work will be conducted in collaboration with key program stakeholders.	December 31, 2019
Recommendation 2: It is recommended that the program intent and value proposition be reviewed and updated to ensure that program officials understand the current operating environment, that the program remains relevant and that highly qualified Canadians are hired.	The RPL Secretariat, in collaboration with key program stakeholders, will review the current and medium term operating environment to ensure that RPL is effectively attracting candidates and the processes in place support efficient hiring from the inventories. The review will take into account the alignment of the program to broader policy recruitment needs. The results will be shared with the DM Champion, ADM advisory panel, and key program stakeholders and inform future recruitment campaigns.	The RPL Secretariat will take the lead on this action item. The work will be conducted in collaboration with key program stakeholders.	April 1, 2020
Recommendation 3: It is recommended that a formal data collection and performance measurement strategy be developed that standardizes the way data is collected, captured and analyzed to support evidence-based decision making and performance reporting. The strategy should be included in the memorandum of understanding established for the program.	The PSC National Recruitment Directorate, in collaboration with the PSC Data Services and Analysis Directorate and other program stakeholders, will develop a data collection and reporting strategy that is commensurate to the size of the program. The aim will be to support evidence-based decision making and the production of an annual report.  The development of the annual report will be led by the RPL Secretariat in consultation with the PSC and other key program stakeholders.	The PSC will take the lead on this action item. The work will be performed in collaboration with the RPL Secretariat and key program stakeholders as required.	Strategy in place by April 1, 2020. First annual report by June 2021.
Recommendation 4: It is recommended that an outreach strategy be developed to inform policy executives, hiring managers, and human resources professionals of the benefits of the program. This should include consultations prior to the annual recruitment campaign to support forecasting potential positions within hiring organizations and the resources required to manage the annual recruitment campaign.	The RPL Secretariat, in collaboration with the RPL Executive Committee and key program stakeholders, will develop an outreach strategy to increase hiring manager awareness of the program and placement of individuals from the pools. The strategy will also include ways to improve the reach of the program, particularly to increase the number of Francophone candidates. The RPL Secretariat will consult with other relevant communities (e.g. Policy Community, Free Agents) to share good practices and ensure ongoing program alignment with broader policy community priorities. This strategy will be presented to the ADM steering committee and DM Champion and used to support future RPL recruitment campaigns.	The RPL Secretariat will take the lead on this action item. The work will be performed in collaboration with key program stakeholders.	June 1, 2020
Recommendation 5: It is recommended that alumni mentors be provided training and on-going support to fulfill their roles and responsibilities related to helping candidates in partially assessed inventories obtain federal public service policy positions.	The RPL Secretariat, in collaboration with the RPL Executive Committee and key program stakeholders, will develop guidance and key competencies to identify, train, and support RPL alumni to effectively perform the mentoring role and maximize placement.	The RPL Secretariat will take the lead on this action item. The work will be performed in collaboration with key program stakeholders.	April 1, 2020
Recommendation 6: It is recommended that an orientation/on-boarding program and support structure be established for alumni to help them navigate their federal public service career. The support structure could include post-hire mentors, rotational assignments and specific training activities.	The CSPS, in collaboration with the RPL Secretariat, RPL Executive Committee, and Policy Community will develop an orientation package for new RPL hires that includes information related to policy work, the policy community and opportunities to help them navigate their federal public service careers. In addition the CSPS will put together a curriculum of courses that are offered through the CSPS that would support RPL hires in the first year of their career and other training that they could take over time to build their policy competencies. The RPL Program would however remain predominantly a recruitment program.	The CSPS will take the lead on this action item. The work will be performed in collaboration with the RPL Secretariat, RPL Executive Committee, and Policy Community.	September  2020

Appendix 1: Recruitment of Policy Leaders program expenditures ^{Footnote 41}

Year	PSC salary ($)	PSC non-salary ($)	DM Champion non-salary ($)	Total ($)	Number in inventory	Number placed	Cost/ placement ($)
2008–09	126,000	0	101,638	227,638	39	33	6,898
2009–10	149,035	1,771	127,819	278,625	56	20	13,931
2010–11	148,667	0	67,890	216,557	40	34	6,369
2011–12	151,927	1,497	54,805	208,229	42	21	9,916
2012–13	133,118	0	0	133,118	29	12	11,093
2013–14	136,363	0	0	136,363	28	8	17,045
2014–15	139,668	543	0	140,211	39	13	10,785
2015–16	143,748	1,802	30,612	176,162	30	17	10,362
2016–17	147,106	2,449	58,973	208,528	30	12	17,377
2017–18	148,756	4,691	66,726	220,173	27	21	10,484
2018–19	168,698	1,728	0	170,426	30

Appendix 2 - Methodology and associated limitations

Methods / sources	Associated limitations
Document review Program documents PSC documents Previous evaluations (RPL and the Advanced Policy Analyst Program) Government of Canada documents Media articles
Literature review exploring the following themes Policy capacity Policy recruitment Policy leadership Policy innovation
Interviews Deputy ministers (3) Senior executives: Director General and above (3) PSC Officials (3) Hiring managers (3) RPL Secretariat members (2) Former candidates (2) Mentors (3) Policy community stakeholders (1)	Almost all interviewees had been involved to some capacity with the RPL program. Mitigation measures: Other lines of evidence, for example the surveys as well as the administrative data, allowed to triangulate interview data with information independent of program participants, managers or champions
Surveys Former RPL candidates Senior policy executives	The list of RPL applicants that was provided to the evaluation team included individuals who had applied to the program since its inception. It included individuals who had been involved in the program prior to the scope period, which begins in 2008. Mitigation measures: Survey data permitted the identification of respondents who had applied to the RPL prior to 2008. This allowed the evaluation team to adjust the data analysis to factor out respondents who were not in scope.
Administrative data analysis PSC staffing data Statistics Canada data	PSC staffing data might have gaps due to lack of a systematic mechanism for tracking RPL recruits. Mitigation measures: The starting point of our analysis was the PSC’s internal data holdings, which identified a total of 232 RPLs during the period of interest. The chart below displays a breakdown by fiscal year of the RPLs identified in the PSC’s internal data holdings (“Official RPL”). In order to provide a more complete picture of the RPL universe, at the request of Internal Audit and Evaluation, the Data Services and Analysis Directorate(DSAD) used a GCpedia list (provided by the program) to capture additional RPL hires that had not been historically identified in the PSC’s datasets. As well, DSAD undertook a review of its RPL identification methodology. As a result of these methodological choices, DSAD identified 28 additional RPLs for the period under study. The breakdown by fiscal year of these newly identified RPLs is offered in the chart below. As such, the data used in this study is different than official statistics. Using the latest methodology, we have revised the official statistic for RPL hires in 2017-2018. RPL applicant survey data was used to reduce the margin of error in PSC staffing data.

Appendix 3 - Recruitment of Policy Leaders program logic model

Appendix 4: Evaluation matrix

Evaluation questions	Indicators	Methods/Source
Relevance
Continued need for the program
1.1 Is there a continuing need for the Recruitment of Policy Leaders (RPL) program?	1.1.1 Level of demand for the RPL program recruits	Document review Review of administrative and performance data Key informant Interviews Survey Literature review
Program alignment
1.2 Are the RPL program objectives aligned with government priorities?	1.2.1 Degree of alignment of the RPL program outcomes with government priorities with respect to building policy capacity within the federal public service	Document review Key informant interviews Literature review
1.3 Is the RPL program consistent with the federal public service roles and responsibilities in the area of human resources staffing?	1.3.1 Degree of compliance of the RPL program with the Public Service Employment Act and regulations, as well as other relevant legislation	Document review Key informant interviews Literature review
Effectiveness
Immediate outcomes
2.1 Are qualified candidates placed in policy positions in the federal public service?	2.1.1 Extent to which RPL recruits are placed in policy positions in the federal public service	Document review Review of administrative and performance data Key informant interviews Survey Literature review
2.2 How does the federal government leverage RPL recruits?	2.2.1 Extent to which RPL recruits expertise, knowledge and networks is leveraged in the federal public service	Document review Review of administrative and performance data Key informant interviews Survey Literature review
Intermediate Outcomes
2.3 Do alumni make a meaningful contribution to the federal public policy framework?	2.3.1 Extent to which RPL participants contribute to the development and implementation of innovative public policies	Document review Review of administrative and performance data Key informant interviews Survey Literature review
2.4 Do alumni exercise leadership on the public policy framework?	2.4.1 Evidence that alumni are in leadership positions 10 years after entry in the federal public service	Review of administrative and performance data Key informant interviews Survey
2.5 Is RPL program aligned with federal public service diversity objectives?	2.5.1 Extent that the program fosters diversity which is compliant with the Employment Equity Act and other relevant legislation and/or guidelines	Document review Review of administrative and performance data Key informant interviews Survey
	2.5.2 Extent that the program facilitates diversity in leadership which is in compliance with Employment Equity Act and other relevant legislation and/or guidelines	Document review Review of administrative and performance data Key informant interviews Survey
	2.5.3 Degree of difference in career progression of alumni from employment equity groups compared with the public service as a whole	Document review Review of administrative and performance data Key informant interviews Survey Literature review
Ultimate outcomes
2.6 Do the alumni influence and shape Canada’s public policy positions to address emerging national and global challenges?	2.6.1.Evidence that the alumni influence Canada’s public policy positions	Document review Review of administrative and performance data Key informant interviews Survey Literature review
	2.6.2 Extent to which the alumni shape public policy positions	Document review Review of administrative and performance data Key informant interviews Survey Literature review
Efficiency
3.1 Are there better, alternative ways (for example, design changes, cost) or lessons learned for achieving the same results?	3.1.1 Evidence of alternative models for recruiting policy analysts in other federal departments, national or international jurisdictions	Document review Review of administrative and performance data Key informant interviews Survey Literature review
3.2 Are the appropriate governance structures and processes in place to support the effective delivery of the program?	3.2.1 Appropriateness of governance structure to support the delivery of the RPL program	Document review Review of administrative and performance data Key informant interviews Literature review