Evaluation of the Research and Policy Initiatives Assistance Program

Evaluation Report
Approved on
Internal Audit and Evaluation Bureau

Table of Contents

Executive Summary

This report presents the results of the evaluation of the Research and Policy Initiatives Assistance (RPIA) Program, conducted by the Internal Audit and Evaluation Bureau of the Treasury Board of Canada Secretariat (the Secretariat) from to . This evaluation was included in the Secretariat’s approved Five-Year Departmental Evaluation Plan for 2012–13 to 2016–17.

This evaluation was conducted in accordance with the Treasury Board Policy on Evaluation and complies with section 42.1 of the Financial Administration Act, which requires departments to review the relevance and performance of all ongoing grants and contributions programs at least once every five years. To extend the RPIA Program for another five years, a review of this program is required by the end of fiscal year 2013–14.

Conclusions

A. Relevance

The RPIA Program is relevant. Through third party organizations, it responded to the government’s continuing need to develop and access ways of streamlining government operations and enhancing the effectiveness of public service management practices. The program addressed this need by supporting ad hoc initiatives to improve public policy research and enhance public service management practices.

i. Continuing Need for the Program

The RPIA Program met beneficiaries’ needs to a great extent. Beneficiaries expressed interest in participating in the program if it were extended. The program has also met sponsors’ needs, although only two sectors used the program in the last five years.

ii. Alignment with Government Priorities and the Secretariat’s Strategic Outcome

The program’s final outcomes were aligned with both government and the Secretariat’s priorities, given the common focus on sound management practices, increased accountability, and risk management. The RPIA Program’s objectives and initiatives were thematically aligned with the government’s priorities of returning to balanced budgets and making government operations more efficient.

iii. Alignment with Federal Roles and Responsibilities

There is a role for government to strengthen the effectiveness of public service practices, to improve service delivery to the public. The evidence demonstrated that the RPIA Program is aligned with this role by supporting small-scale, ad hoc research initiatives that contributed to the advancement of public service management.

B. Performance

The RPIA Program achieved the short- and medium-term outcomes articulated in its logic model. However, the lack of performance measures and the inconsistent use of project evaluation grids created difficulty in readily demonstrating evidence of the program’s impacts. The absence of a formal central administrator or program manager led to challenges for sponsors in understanding roles and responsibilities. Sponsors had an uneven understanding of the program, resulting in incomplete tracking of performance measures and inconsistent use of project evaluation grids. A low level of program awareness contributed to the lack of use by some sectors.

The program’s terms and conditions (renewed in 2009) were based on the 2000 version of the Policy on Transfer Payments rather than the 2008 version. This resulted in a lack of alignment with a number of current policy requirements, such as the need to implement performance measurement, service standards, and risk-based assessments for monitoring, as well as requirements to integrate official languages requirements and to post program information on the Secretariat’s external website.

i. Effectiveness

There was evidence that the RPIA Program’s immediate outcomes were achieved to some extent. A more systematic approach to reporting on impacts of initiatives would aid in a clearer demonstration of the program’s overall ability to achieve its stated outcomes.

The program achieved its intermediate outcomes. Through its initiatives, it expanded the Secretariat’s knowledge in areas of leadership, innovation, and risk management. The Secretariat benefited from rich partnerships with prominent national and international organizations.

ii. Economy and Efficiency

The outcomes were achieved in the most economical manner possible. Compared with similar public service programs, the RPIA Program had the lowest overhead costs. It also leveraged resources (financial and professional) from third party organizations, and donations from other entities.Footnote 1 Although minimal resources were expended to administer the program, a more focused program management approach in line with the program’s low materiality and risk would promote even more efficient operations.

Some aspects of the RPIA Program detract from its efficiency, namely, the terms and conditions, which impose heavy administrative requirements on sponsors and beneficiaries, as well as the requirements for the Sector Head, Chief Financial Officer and Deputy Head to approve all initiatives.

iii. Design Alternatives

The RPIA Program’s design is flexible; however, a risk-based approach to funding (e.g., removing the requirement for Chief Financial Officer and Deputy Head approval of low‑risk, low‑materiality initiatives with long-standing third party organizations) could reduce administrative burden on managers, sponsors, and beneficiaries, and result in faster implementation of initiatives.Footnote 2

Recommendations

  1. It is recommended that Corporate Services Sector, in consultation with the Office of the Comptroller General, update the terms and conditions of the RPIA Program and monitor the terms and conditions on an ongoing basis, to ensure conformance with any future updates to the Treasury Board Policy on Transfer Payments.
  2. It is recommended that Corporate Services Sector, in conformance with the Treasury Board Policy on Transfer Payments, post RPIA Program templates, guides, and information products on the Secretariat’s external website for third party organizations and sponsors.
  3. It is recommended that the Corporate Services Sector implement a performance measurement strategy calibrated to the size and risk of the RPIA Program.
  4. It is recommended that the Corporate Services Sector develop options for the designation of a program manager for the RPIA Program, to ensure that materials are up to date, to make the program more visible, and to monitor program outcomes by implementing the performance measurement strategy.
  5. It is recommended that the Corporate Services Sector use a risk-based approach for approvals, to increase the efficiency of the RPIA Program and to reduce administrative burden.

1. Introduction

This report presents the results of the evaluation of the Research and Policy Initiatives Assistance Program (RPIA Program), conducted by the Internal Audit and Evaluation Bureau of the Treasury Board of Canada Secretariat (the Secretariat) from to . This evaluation was included in the Secretariat’s approved Five-Year Departmental Evaluation Plan for 2012–13 to 2016–17.

This evaluation was conducted in accordance with the Treasury Board Policy on Evaluation and complies with section 42.1 of the Financial Administration Act, which requires departments to review the relevance and performance of all ongoing grants and contributions programs at least once every five years. To extend the RPIA Program for another five years, a review of this program is required by the end of fiscal year 2013–14.

2. Program Profile

2.1 Background

Established in 2003, the RPIA Program is a class contribution program to support initiatives that contribute to the advancement of public policy research, thereby furthering the Secretariat’s mandate and objectives. Treasury Board ministers provided the Secretariat and its managers with this class contribution authority under the 2000 Policy on Transfer Payments. Following an evaluation in 2009 (for the period 2003–04 to 2007–08), the program’s terms and conditions were renewed for a further five-year period. The current terms and conditions expire on .

Prior to the RPIA Program, sectors prepared individual Treasury Board submissions to fund these low-dollar value and low-risk research initiatives. Given the low materialityFootnote 3 of these initiatives, Treasury Board submissions were seen as inefficient and expensive. As a result, this class contribution program was established to consolidate these individual initiatives and to provide a cost-effective funding option for sectors to partner with third party organizations in promoting targeted policy research.

The Secretariat’s RPIA Program has been used to transfer monies from the federal government to eligible recipients (third party organizations) for policy and research initiatives. These initiatives have included workshops, forums, conferences, and studies in subject areas that are consistent with the Secretariat’s mandate and that have contributed to the advancement of public policy research and public service management.

Eligible recipients include individuals; groups of individuals; privately owned organizations; not‑for-profit organizations; universities and colleges; cooperatives; provincial, territorial, and municipal governments and government agencies; provincial Crown corporations; international organizations; non-government corporations; and financial institutions.

Sectors access the program on an as needed basis. There are no annual minimum funding amounts targeted by the program. The program has a stacking limitFootnote 4 of 100 per cent, and the Secretariat may contribute up to 50 per cent of total eligible costs of an approved project, up to $500,000 over the life of the initiative. Although there is no annual maximum amount for the program, the Vote 1 limit in the Secretariat is $5 million.

2.2 Governance and Roles and Responsibilities

The RPIA Program has no central program administrator. It is a funding authority that allows sectors to pursue targeted research initiatives supporting the improvement, advancement, and promotion of public service management practices. Under the program’s terms and conditions, use of this funding authority requires the approval of a Sector Head, the Chief Financial Officer, and the Secretary of the Treasury Board. Corporate Services assists in reviewing applications and in determining that proposed initiatives are appropriate and eligible for program funding.Footnote 5

Sponsoring managers are responsible for assisting in the collection of performance measurement information to demonstrate program impacts.

The Sector Head ensures that contribution agreements comply with the Policy on Transfer Payments and with the terms and conditions of the RPIA Program prior to being recommended for approval by the Secretary. The Sector Head and the Chief Financial Officer ensure that the RPIA Program is the appropriate vehicle to fund the initiative so that only proposals conforming to the terms and conditions of the program are supported, and that management of approved initiatives abides by the terms and conditions of the program and the obligations and funding limits outlined in the contribution agreement.

Third party organizations (the beneficiaries) are responsible for implementing initiatives in accordance with the terms and conditions of the agreements and all applicable laws.

2.3 Resources

As shown in Table 1, over the five-year period of the evaluation (2008–09 to 2012–13), the RPIA Program funded 11 initiatives, totalling $442,209. A detailed overview of the total cost of these initiatives is provided in Appendix B.

Table 1. Value of RPIA Initiatives by Year (2008–09 to 2012–13)
Year Beneficiary and Initiative Initiative Value ($)
2008–09 Public Policy Forum, “Good Government is Good Politics” 28,366
2009–10 Public Policy Forum, “Adapting Public Policy to a Changing World” 20,000
2010–11 Public Policy Forum, “A Vision Too Far?” 20,000
2011–12 Public Policy Forum, “On Authenticity – How the Truth Can Restore Faith in Politics and Government” 20,000
2011–12 Institute on Governance, “Agents of Parliament Project” 33,443
2011–12 Organisation for Economic Co-operation and Development (OECD), “Communicating and Measuring Progress in Regulatory Reform” 150,000
2011–12 Centre for the Study of Living Standards, Festschrift Volume, “New Directions for Intelligent Government in Canada” 5,000
2011–12 International Risk Governance Council, “Public Sector Governance of Emerging Risks” 50,000
2011–12 OECD, “The 2011 Meeting of the Network of Senior Officials from the Centers of Government” 63,000
2012–13 OECD, “Development of the Observatory for Public Sector Innovation” 32,400
2012–13 Public Policy Forum, “Insights on the Relevance of the Cooperative Model for Public Service in Canada” 20,000
Total paid by the Secretariat 442,209

The average value of an initiative between 2008–09 and 2012–13 was $40,200.82. Table 2 shows that the average annual value of initiatives fluctuated from a low of $20,000 in 2009–10 and 2010–11 to a high of $53,573.83 in 2011–12.

Table 2. RPIA Program Expenditures from 2008–09 to 2012–13
Year Number of Initiatives RPIA Initiative ($) Average $ per Initiative
2008–09 1 28,366 28,366.00
2009–10 1 20,000 20,000.00
2010–11 1 20,000 20,000.00
2011–12 6 321,443 53,573.83
2012–13 2 52,400 26,200.00
Total 11 442,209 40,200.83

2.4 Logic Model

Figure 1 displays the logic model for the RPIA Program, which is a graphical depiction of how the activities and outputs of the program contribute to immediate, intermediate, and final outcomes.

Figure 1: Logic Model for the RPIA Program
Logic Model for the RPIA Program. Text version below:
Figure 1 - Text version

This graphic illustrates the logic model of the Research and Policy Initiatives Assistance Program (hereafter the RPIA Program). The logic model shows that the final outcomes of the RPIA Program are "Contribution to the advancement of public service management and public policy research, and Contribution to the effectiveness of public service management practices, reporting, and service delivery to the public."

The RPIA Program has two intermediate outcomes which aim to contribute to its final outcomes and are presented as follow: "Enhanced departmental capacity to produce thorough and relevant analysis, and Increased contribution to knowledge related to emerging policy issues of interest to the department.

The program's three immediate outcomes aim to contribute to the intermediate outcomes and are presented as follows: Access to expanded knowledge base on subjects of interest, Better informed policy context and expanded community of interest, and Increased leverage of resources or other capacities from external partners toward departmental objectives.

The program has three output clusters which are intended to lead to the achievement of the immediate outcomes. The first cluster of outputs includes "Contribution agreements; tools; guidance; and Terms and Conditions. The second cluster of outputs contains: "conferences; workshops; seminars; and other knowledge and outreach Activities. The last cluster of outputs includes: "research; analysis; and data collection.

Three activities are intended to lead to the clusters of outputs. The first activity that is undertaken by Corporate Service Sector is expected to lead to the first cluster of outputs and refers to "Establishing expedient and flexible contribution agreements. The second and third activities which are undertaken by Sponsoring Managers are intended to lead to the production of the second and third cluster of outputs. These are: "Identifying and seizing opportunities to lever external partners, and Scanning academic and professional communities for activities that could further key departmental objectives.

3. Evaluation Context

This section describes the evaluation’s purpose, scope, data collection methods, and limitations. The Secretariat’s Internal Audit and Evaluation Bureau conducted the evaluation but used the services of Goss Gilroy Inc., Management Consultants for the jurisdictional review.

3.1 Purpose and Scope

The purpose of the evaluation was to assess the core evaluation issues related to relevance and performance in accordance with the Treasury Board Policy on Evaluation. Relevance was examined in terms of the continuing need for the RPIA Program, the alignment of the RPIA Program with government priorities and the Secretariat’s strategic outcome, and the alignment with federal roles and responsibilities. Performance was assessed in terms of effectiveness, economy, efficiency, and design alternatives.

The evaluation covered departmental spending on the RPIA Program from 2008–09 to 2012–13.

A complete listing of the evaluation issues and questions addressed in this report is provided in Appendix A.

3.2 Evaluation Methodology

The evaluators used an evaluation matrix that identified the evaluation questions, indicators, data sources, and detailed lines of evidence or research methods. The findings, conclusions, and recommendations are based on the collection, analysis, and triangulation of information derived from the following lines of evidence:

1. Document Review

The evaluators reviewed the eleven initiatives sponsored during the five-year review period. They also examined project management materials, communication products, the program’s terms and conditions, and the Treasury Board submission that were prepared at the times the program was created and extended.

To aid in their analysis, the evaluators developed a document review matrix, which gave a synopsis by document and by evaluation issue. In addition, the evaluators examined the RPIA Program’s terms and conditions against the 2000 and 2008 versions of the Policy on Transfer Payments. The evaluation findings on relevance and performance were summarized in a technical report.

The evaluators consulted documents such as the Financial Administration Act and departmental corporate reports and policies to ascertain the program’s relevance and need, as well as its alignment with government and the Secretariat’s priorities.

2. Group Interviews

To streamline the evaluation approach and calibrate the evaluation to the overall low materiality and risk of the RPIA Program, the evaluators held group interviews by respondent type whenever possible. They formed seven interview groups that clustered respondents into three categories: program managers, contribution sponsors, and technical experts.Footnote 6 The interviews were semi‑structured, with open-ended questions that allowed individuals working in similar areas to build on one another’s observations. To ensure coverage for those missing the group interview, evaluators interviewed some respondents individually. To simplify reporting, all interview evidence was reported at the group level rather than an individual level and all groups were given equal weight.

The evaluation team gathered evidence by evaluation issue using an interview matrix, which supported the team during the analysis and reporting. Table 3 describes the structure of the interviews and the overall number and types of interviewees.

Table 3. Interview Structure
Interview Group Respondent Type Sector Number of Participants
1 Program manager Corporate Services 5
2 Program manager Corporate Services 1
3 Sponsor Priorities and Planning 5
4 Sponsor Priorities and Planning 1
5 Sponsor Regulatory Affairs 2
6 Expert Corporate Services 2
7 Expert Office of the Comptroller General 2
Total number of participants 18

The following scale expresses the relative weight of grouped responses:

  • “All” reflects the views of every respondent group;
  • “Majority” reflects the views of at least four but fewer than seven interview groups; and
  • “Few” reflects the views of at least two but fewer than four respondents.

3. Beneficiary (Third Party Organizations) Questionnaire

In collaboration with program sponsors, evaluators conducted an email survey of the six beneficiary organizations receiving RPIA funding from 2008–09 to 2012–13. The questionnaires collected information about the beneficiaries’ views of the program’s impacts, the beneficiaries’ levels of satisfaction with the program, ease of use of the program, the program’s visibility, and the beneficiaries’ ability to leverage other resources. Five of six beneficiaries answered the questionnaire.

4. Directors’ Coordinating Committee Questionnaire

The evaluation research team emailed a questionnaire to the Secretariat’s senior and executive directors via the Directors’ Coordinating Committee. Evaluators identified this committee’s membership as the main and potential users of the RPIA Program. The questionnaire sought to determine the extent to which the RPIA Program was needed, understood, accessible, and effective. Nine of twenty-one directors responded to the questionnaire.

5. Jurisdictional Review

This line of evidence offered an assessment of other Government of Canada programs with similar public service management and public policy research objectives. The purpose of the jurisdictional review was to identify any possible improvements or alternatives to the present design that could reduce costs while increasing the likelihood of achieving outcomes. The review found three fully comparable and five partially comparable programs.

3.3 Limitations

The evaluators attempted to compile a complete document inventory for all 11 initiatives (e.g., contribution agreements, end-of-project deliverables and reports, and briefing materials); however, they found only one project evaluation grid discussing the achievement of an initiative’s immediate outcomes. Consequently, the evaluators were not able to conclude with a high degree of confidence that the initiatives contributed to the achievement of the RPIA Program’s intermediate and final outcomes.

To mitigate this gap, the evaluators examined final reports or conference brochures relating to the initiatives whenever possible. They reviewed these documents alongside the views of beneficiaries and explored any gaps in files directly with sponsors.Footnote 7

Because of a lack of information within the Corporate Services Sector, the evaluators were unable to gather data about organizations that were unsuccessful in receiving RPIA funds. Instead, evaluators conducted interviews with contracting and procurement experts to examine alternatives to the use of the program.

4. Findings

4.1 Relevance

4.1.1 Continuing Need for the Program

This evaluation issue examined the extent to which the RPIA Program continues to address a demonstrable need and is responsive to the needs of Canadians.

Evaluation Question 1.1 – Continuing Need for the RPIA Program

  • To what extent does the RPIA Program address a continuing need?
  • Is the program responsive to departmental and beneficiaries’ needs?

Finding

There is a continuing need for the RPIA Program. Evidence from the document review, interviews, and the beneficiary questionnaire demonstrated that the RPIA Program responds to beneficiaries’ needs regarding ways to streamline government operations. Although only two sectors in the Secretariat used the RPIA Program over the review period, the program supported promotional activities such as forums for discussion on important policy areas, contributing to the advancement of public policy.

A majority of beneficiaries responding to the questionnaires reported that their projects contributed to the achievement of the RPIA Program’s immediate outcomes to a great extent or to a very great extent.

Addressing a Continuing Need

The document review confirmed a need for government to work toward more effective, streamlined operations. Specifically, Blueprint 2020 encourages the federal government to reflect on how government does business, in order to pursue continual improvement. Economic Action Plan 2013 asks departments to continue examining ways of streamlining government operations. Given that the ultimate outcome of the RPIA Program is to advance and make public service management more effective, the RPIA Program supported these broad themes.

The review of program documentation found that the RPIA Program addressed a continuing need of government to develop and access ways of streamlining operations and enhancing the effectiveness of public service management practices. The RPIA Program addressed this need by providing a mechanism to support ad hoc research initiatives that aim to improve public policy, service delivery to the public, and public service management practices.

The ongoing need for the RPIA Program was expressed by the majority of interviewees, who agreed that the program allowed sectors to establish ad hoc initiatives with external organizations, to create networks, and to enable critical thinking on policy research that benefits Canadians. Only two sectors used the RPIA Program during the review period; however, this limited use could be attributed to low awareness of the program; in the questionnaire administered to directors, five of nine reported being unaware of its existence.Footnote 8

All beneficiary organizations responding to the questionnaire stated that the program assisted them in meeting their information and research needs to a great extent or to a very great extent.

Responsiveness to Departmental and Beneficiary Needs

RPIA initiatives addressed and responded to beneficiaries’ needs for information on government management practices, regulatory reform, and improved service delivery to the public. The majority of beneficiaries agreed that the RPIA Program was useful in the work they do to shape public policy and that they would continue participating in the program if it were extended.

Evidence from the document review shows that the RPIA Program supported research and policy initiatives aimed at improving public service management practices.

There is evidence that opportunities exist for greater use of the program. A few respondents to the Director’ Coordinating Committee questionnaire stated that they would use the RPIA Program if it were extended.

4.1.2 Alignment with Government Priorities and the Secretariat’s Strategic Outcome

This evaluation issue assessed the alignment of the RPIA Program’s objectives with government priorities and with the Secretariat’s strategic outcome.

Evaluation Question 1.2 – Alignment with Government Priorities and the Secretariat’s Strategic Outcome

  • Do RPIA Program objectives align with government and Secretariat priorities?
  • To what extent are RPIA initiatives consistent with Secretariat priorities?

Finding

Evidence from the document review and interviews demonstrated that the RPIA Program objectives were aligned with both government priorities and the Secretariat’s priorities. Moreover, RPIA funding supported initiatives that were consistent with select sector and Secretariat priorities.

A review of program documents showed that the objectives of the RPIA Program are to support initiatives that advance public policy research and the effectiveness of public service management practices. These objectives are aligned with the Secretariat’s priorities, as articulated in its annual Reports on Plans and Priorities, of strengthening governance, accountability, and management practices; ensuring that the management regime operates effectively, provides opportunities for risk-based decision making, and enables innovation; strengthening risk management capacity and adopting risk-based approaches in daily business; ensuring value for money; and advancing initiatives to modernize government operations.

The document review also showed that RPIA Program objectives were aligned with the government’s priorities of returning to balanced budgets and establishing a more efficient government. Economic Action Plan 2013 calls for new ways to standardize, consolidate, and transform the way government does business, to improve services and achieve efficiencies.

The review of the RPIA Program files demonstrated that RPIA-funded projects supported research and workshop objectives on accountability; good practices for managing risk, innovation and leadership; and progress in regulatory reform. These objectives were aligned with the Secretariat’s priorities.

The majority of the interview groups also agreed that the RPIA Program is relevant to the Secretariat given that the program’s objectives are aligned with the sponsor sectors’ mandates and the Secretariat’s objectives regarding the management of government administration.

4.1.3 Alignment with Federal Roles and Responsibilities

This evaluation issue looked at the roles and responsibilities for the federal government in delivering the RPIA Program.

Evaluation Question 1.3 – Alignment with Federal Roles and Responsibilities

  • Is there a role for the federal government in improving, advancing, and promoting public service management practices that contribute to the advancement of public policy research and public service management?

Finding

Evidence from the document review showed that there is a role for government in strengthening the effectiveness of public service practices, to improve service delivery to the public. The RPIA Program supported this role by facilitating ad hoc research initiatives that contributed to the advancement of public service management.

The evaluators concluded from the document review that the federal government plays a significant role in advancing and promoting public service management practices that contribute to the advancement of public policy research and public service management. Economic Action Plan 2012 and Economic Action Plan 2013 state that the role of the federal government is to transform the way government does business. The Secretariat, in supporting Treasury Board ministers, strengthens the way government is managed and ensures value for money in government spending and results for Canadians.

The Secretariat’s Integrated Business Plan 2013–16 states that “the Secretariat is playing a central role in key reforms to increase the efficiency and productivity of the public service, including in the areas of human resource management, financial and expenditure management, and leveraging Information Technology.” RPIA Program objectives support the advancement of public service management practices by supporting research initiatives that contribute to the advancement of public policy research and public service management and that enhance the effectiveness of public service management practices, reporting, and service delivery to the public.

Conclusion: Relevance

The RPIA Program is relevant. Through third party organizations, the program responds to the government’s continuing need to develop and access ways of streamlining government operations and enhancing the effectiveness of public service management practices. The program addresses this need by supporting ad hoc initiatives to improve public policy research and enhance public service management practices.

4.2 Performance

4.2.1 Effectiveness

This evaluation issue assessed the extent to which the RPIA Program achieved its expected outcomes regarding program design and performance targets, reach, and the contribution of outputs to outcomes.

Evaluation Question 2.1 a. – Program Design

  • Are the program design and eligibility criteria appropriate?
  • Are the roles and responsibilities of the program components clear? Appropriate?
  • Is the program’s communications strategy adequate?
  • To what extent does the RPIA Program’s performance measurement system meet management’s information needs?
  • To what extent is the RPIA Program aligned with the Policy on Transfer Payments?

Finding

There were mixed views among interview groups on the appropriateness of the program’s design and eligibility criteria.

The absence of a formal central administrator or program manager led to challenges within sectors in understanding roles and responsibilities and to inconsistent performance monitoring and reporting.

Other challenges related to the lack of program tools, guides, and promotional materials, which hampered program performance and created barriers to access. The program was largely used by those knowledgeable about the program and has relied on word of mouth.

The evaluation found that the RPIA Program’s terms and conditions were based on the 2000 version of the Policy on Transfer Payments rather than the 2008 version, resulting in a lack of alignment with some current policy requirements.

Although interviewed managers emphasized that direct benefits to the Secretariat from contributions were not permitted, they reported having limited opportunities to observe program impacts because their roles did not involve implementing initiatives. For managers, indicators of the program’s value were its stable demand year over year, repeat users of the program, and the high profile of third party beneficiary organizations.

Program Design, Eligibility Criteria, and Roles and Responsibilities

Interview groups had mixed views of the appropriateness of the RPIA Program’s design. Three of seven groups favoured the program design for its lack of centralized, formal oversight, believing it provided for a streamlined, flexible process. The other groups were critical of the design, citing heavy administrative burden and steep learning curves. The program’s design requires interaction with select individuals and may involve iterations to ensure that proposals are well drafted. Two groups of sponsors stated that the program operated in a complex fashion and that it was labour intensive to establish an agreement. These groups reported that the program’s approval process was overwhelming for third party organizations and in one case had resulted in a withdrawal of interest.

Similarly, interview groups expressed mixed views on the clarity of the RPIA eligibility criteria and terms and conditions. Both groups of managers stated that the eligibility criteria were clear and readily available. Eligibility criteria were reported to be fair by managers in all interview groups, and these criteria were seen to be broader than the criteria of comparable programs in other departments. A majority of managers said that the broadness of the criteria gave sectors the flexibility to advance their research projects. This point of view was not shared by sponsors, who reported seeking help from other sponsors before approaching program managers for information. One group of sponsors said that the eligibility criteria were neither restrictive nor unreasonable. This group stated that delays occurred because the nature of the initiative involved working with external organizations that had unique processes, approvals, and authorities. Another group stated the eligibility criteria were challenging and unreasonable when dealing with multi-year projects and international organizations.Footnote 9

Interview groups were also divided on the appropriateness and clarity of the roles and responsibilities of financial advisors and sponsors of potential initiatives. Two groups said that the roles and responsibilities were clear; a third group suggested that roles and responsibilities of sponsors and administrators needed to be clarified; a fourth group said that the program was delivered as designed, which was appropriate; and a fifth group stated that the approval process needed to be simplified by removing the requirement for approval by the Deputy Head and the Chief Financial Officer.

When compared with similar programs,Footnote 10 the RPIA Program’s design was found to be more flexible and convenient for sponsors to use for small-scale, policy-related projects. Other programs had longer application forms, more detailed project selection procedures, and longer delays in receiving funds.

Communications Strategy

A majority of respondents to the Directors’ Coordinating Committee questionnaire stated that they were unaware of the existence of the RPIA Program. However, they indicated that the RPIA Program appeared to be a viable option for their research initiatives and that they were interested in learning more about it.

The jurisdictional review found that the RPIA Program was fully comparable to programs operating in the Department of Finance Canada, the Atlantic Canada Opportunities Agency, and Industry Canada, as it relies less on promotion and more on sponsors’ awareness of the program. This finding was corroborated by the review of program files, which demonstrated that the majority of initiatives were sponsored by one sector and by the same sponsors.

The document review found that the communications strategy for the program was based on a 2013 presentation by Corporate Services to all financial analysts. This presentation was also posted on the Secretariat’s InfoSite, to ensure that all employees were able to obtain more information about the program. Although managers stated that the program was well communicated, additional probing in the interviews revealed that the presentation had been delivered only once and had not been updated since. One group of managers added that it would be a good practice to have an updated presentation given to financial analysts every two years. Neither the evaluators nor one group of experts could find the RPIA Program’s templates (terms and conditions, contribution agreement template, submission/recommendation form, and project evaluation grid) on the Secretariat’s external website.

Two of the three groups of sponsors indicated that the application forms were not clear and that written instructions would be useful. Evaluators noted that even though information on the RPIA Program was posted on the Secretariat’s InfoSite, no guidelines, samples, or written instructions on how to fill in forms were provided.

There were mixed views among interview groups on the adequacy of the financial management guidelines for RPIA Program initiatives. Although both groups of program managers emphasized that they advise sectors, track resources, and administer funding, most of the sponsor groups identified a need for more guidance and admitted that they were using their own knowledge and guidance materials to create new initiatives. In their view, financial coordination was ad hoc.

A review of the program’s management response to the recommendations from the previous evaluation found no evidence that the Corporate Services Sector had increased the program’s visibility either by posting program materials on the Secretariat’s external website or by preparing a communications strategy that would inform sectors of the program’s existence.

Performance Measurement

Performance measurement is a mandatory component of all programs and is to be implemented at the time of a program’s design. Under the Policy on Transfer Payments, monitoring of performance must take into account the program’s materiality and must reflect risks specific to the program, the value of funding in relation to administrative costs, and the risk profile of the recipient.Footnote 11 In their review of RPIA Program documents, evaluators found no evidence that performance measures were in place and minimal evidence of performance measurement data. Although the document review shows that the responsibility for collecting performance measurement rests with sponsors, interviewed sponsors were not aware of this role.

Sponsors stated that performance measurement would create a burden for beneficiaries if it required asking beneficiaries to demonstrate outcomes. One group of sponsors cautioned that if a performance measurement system were implemented, it should not require input from beneficiaries because they were already overwhelmed by the administrative burden the program imposed. Managers also questioned whether formal performance measurement would be needed if sponsors were encouraged to hold back 10 per cent of funds until the end of the planned activity.

When evaluators asked managers to describe the type of information needed to manage the program, one group mentioned the project evaluation grid, which was recommended, but not required, to be administered at the close of the initiative. This grid requires sponsors to provide a description of the initiative and to identify the actual outcomes versus the planned outcomes in the proposal.

In the review of program files, evaluators found only one case in which the project evaluation grid was completed. One group of sponsors reported that they had never been asked to complete the grid and were unaware of it. They stated that they would have filled it in for beneficiary organizations had they been asked.

Alignment with the Policy on Transfer Payments and the Directive on Transfer Payments

The evaluators analyzed the 2000 and the 2008 versions of the Policy on Transfer Payments and associated directive as part of a comparative review of the program’s terms and conditions. The evaluators found that the program’s terms and conditions had not been updated at the time of the program’s renewal in 2009 and, as a result, did not conform to the 2008 policy and directive.

Section 6.5.2 of the Policy on Transfer Payments refers to establishing, at the time of the program design, a performance measurement strategy and maintaining and updating it throughout its life cycle, to effectively support an evaluation or review of relevance and effectiveness of the transfer payment program.

Sections 6.5.7 to 6.5.9 of the Policy on Transfer Payments refer to risks levels, engagement of applicants and recipients, and the implementation of departmental service standards.

Section 6.1.4 of the Directive on Transfer Payments refers to the availability of program information to potential recipients via the Secretariat’s website.

Section 6.5.1 of the Directive on Transfer Payments refers to the implementation of a risk-based assessment regarding the monitoring and reporting of recipients.

Appendix D, item 10 of the Directive on Transfer Payments refers to the description of official languages requirements in order to respect the obligations of the Government of Canada as set out in Part VII of the Official Languages Act and states that services and benefits are made available in both official languages.

One group of managers supported having a formal policy review to assure alignment, now that a new policy has come into force.

Evaluation Question 2.1 b. – Achievement of Immediate Outcomes

  • To what extent is the RPIA Program achieving its immediate outcomes?

Finding

Although a more systematic approach to reporting on the impacts of contributions would result in a clearer demonstration of the program’s overall ability to achieve its stated outcomes, there was evidence to suggest that the program’s immediate outcomes were achieved to some extent.

The program contributed to the expansion of the Secretariat’s knowledge in areas of leadership, innovation, and risk management. Working with third party organizations also contributed to a better-informed policy context in regulatory management, oversight, and management accountability and to expanded communities of interest by supporting dialogue among officials, experts, and academics.

Expanding the Secretariat’s Knowledge

A review of project documents such as funded research papers, final reports, contribution agreements, and the evaluation report grid found that the RPIA Program had provided access to some extent to an expanded knowledge base on subjects of interest to the Secretariat. The RPIA Program had funded lectures on adapting public policy in a changing world and had addressed issues such as public service reform, leadership, and innovation. Moreover, RPIA funds had provided the Secretariat with access to research on fields of interest to the government, such as communicating and measuring progress in regulatory reform and exploring good practices for managing emerging risk and corresponding management actions in large, complex public and private sector organizations. The RPIA Program had also helped a sponsoring sector promote dialogue on innovation, fiscal constraint, horizontal governance, and long-term challenges among senior government officials from OECD countries, identifying lessons learned and best practices.

Beneficiaries responding to the questionnaire stated that their projects had contributed to the achievement of the RPIA Program’s immediate outcomes to a great extent or to a very great extent. Similarly, through their development and implementation of initiatives, sponsors reported an increase in their sectors’ capacity to do public policy research. This group stated that without this program some activities would not have taken place.

Interview groups held mixed views on whether this immediate outcome was achieved solely as a result of the RPIA Program. The majority of sponsor groups indicated that lectures and workshops gave the Secretariat access to renowned specialists and subject-matter experts to some extent and strengthened the Secretariat’s partnerships with organizations of stature. One group of managers suggested that the same information and partnerships might be realized for international projects through professional service contracts and memberships in professional organizations.Footnote 12

A Better-Informed Policy Context and an Expanded Community of Interest

The majority of respondents stated that the RPIA Program contributed to the outcomes of a better-informed policy context and an expanded community of interest through non‑governmental partnerships and communication of policy contexts nationally and internationally.

From the file review of contribution agreements and the document review of research papers and final reports, the evaluators concluded that the RPIA Program supported regulatory management and oversight and management accountability policy initiatives, as the funded initiatives fell under these themes. The evaluators found documented evidence that one initiative “improved, advanced, and promoted public service management practices by supporting a dialogue among high-level officials on how government could effectively respond to policy and governance challenges in the context of fiscal constraint.”

Evidence from the file review demonstrated that the RPIA Program expanded the “community of interest” in regulatory management, the government’s role, risk management, policy, and innovation. In one case, the program funded a research initiative that explored the theme of good practices for managing emerging risks, drawing on the expertise of academics, expert practitioners, and representatives from other governments. In another case, the program funded a research initiative that brought together regulatory experts from international governments and academia to promote debate on regulatory policy evaluation, resulting in a practical framework that could be applied in OECD countries.

The majority of beneficiaries confirmed in their questionnaires that the program contributed to their organizational objectives by:

  • Furthering their research, expertise, and understanding;
  • Being instrumental in the collection of data for analysis across several countries; or
  • Allowing them to draw on expert knowledge, lessons learned from other countries, and the expertise of leading academics, to further their public policy mandates.
Increased Leverage of External Partners’ Resources and Other Capacities Toward Achieving Departmental Objectives

All sponsors affirmed that the Secretariat leveraged beneficiaries’ resources to a great extent. These resources were typically financial, expertise, and access to partners’ networks. One beneficiary was able to attract additional financial resources from other countries, as well as to expand its partnerships with other international organizations, to access international expertise and knowledge from academics because of the program.

The document review showed that beneficiary organizations were extremely resourceful in how they organized, planned, networked, and implemented their RPIA initiatives. Partnerships with beneficiaries built sponsor sectors’ capacity in these areas.

Evaluation Question 2.1 c. – Achievement of Intermediate Outcomes

  • To what extent is the RPIA Program achieving its intermediate outcomes?

Finding

The RPIA Program has contributed to some extent to the achievement of its intermediate outcomes of increased knowledge on emerging policy issues and of enhanced departmental capacity to produce thorough and relevant analysis. However, it was found that this contribution was limited to influencing policy development and creating awareness. Documented evidence of intermediate outcomes was largely unavailable.

The majority of interview groups agreed that the program contributed to a great extent to increasing the Secretariat’s knowledge of emerging policy issues, and to some extent to enhancing departmental capacity to produce thorough and relevant analysis. Interviewees stated that the contribution to these outcomes was indirect or limited because initiatives alone do not make policy. Rather, the results of initiatives are used to influence policy development by bringing awareness of a changing environment.

Similarly, the majority of beneficiaries reported achieving intermediate outcomes to a great extent or to a very great extent.

The document review demonstrated that even though the application forms collected information on anticipated outcomes regarding how the Secretariat’s capacity would be enhanced and how the sectors’ knowledge would increase as a result of the initiatives, there was limited information in the files to demonstrate the actual impacts. Evaluators found that project evaluation grids, which would have provided a narrative for the capture of these impacts, were not systematically completed by sponsors.Footnote 13

Evaluation Question 2.1 d. – Achievement of Final Outcomes

  • To what extent is the RPIA Program achieving its final outcomes?

Finding

The evaluation found that the program contributed to a very great extent to the advancement of public service management of public policy research, and to a great extent to the effectiveness of public service management practices, reporting, and service delivery to the public. Evaluators did not find verifiable evidence of the achievement of these outcomes in program files.

The majority of interview groups believed that the program contributed to some extent to the achievement of the effectiveness of public service management by virtue of long-standing research partnerships with international organizations. These respondents were less certain of the extent to which the achievement of final outcomes could be attributed solely to the program. Nevertheless, the majority of beneficiaries responding to the questionnaire reported that their projects contributed to a great extent or to a very great extent to the achievement of the RPIA Program’s final outcomes.

As previously noted, evaluators detected some gaps in the documentation of actual versus planned results, making a verifiable assessment of impacts difficult. However, there was evidence that all 11 initiatives proceeded according to their proposals, that all payments were appropriate and accurate, and that all initiatives were completed.

Conclusion: Effectiveness

Overall, the program achieved the outcomes articulated in its logic model to varying extents. However, the lack of performance measures and the inconsistent use of project evaluation grids created difficulty in readily demonstrating evidence of the program’s impacts. The absence of a formal central administrator or program manager led to challenges for sponsors in understanding roles and responsibilities. Sponsors had an uneven understanding of the program, resulting in incomplete tracking of performance measures and inconsistent use of project evaluation grids. A low level of program awareness contributed to the lack of use by some sectors.

The program’s terms and conditions are based on the 2000 version of the Policy on Transfer Payments rather than the 2008 version. Consequently, there is a lack of alignment with some current policy requirements such as the need to implement performance measurement, service standards, and risk-based assessments for monitoring, as well as to integrate official languages requirements and to post program information on the Secretariat’s external website.

4.2.2 Economy and Efficiency

This core evaluation issue examines the program’s resource use in relation to the production of outputs and progress toward expected outcomes.

Evaluation Question 2.2 – Economy and Efficiency

  • To what extent is the RPIA Program economical and efficient?

Finding

The evaluation found that the RPIA Program used financial resources economically. Compared with other programs, it had the lowest overhead costs.

Some aspects of the RPIA Program detracted from its efficiency, chiefly the level of administrative involvement required of sponsors and beneficiaries and the requirement for the Sector Head, Chief Financial Officer and Deputy Head to approve the agreements.

The jurisdictional review showed that the RPIA Program was as economical as possible. The RPIA Program had minimal overhead because it had no dedicated employees (full-time equivalents) and operated with comparatively simple application and selection processes. Of the fully comparable programsFootnote 14 examined in the jurisdictional review, the RPIA Program had the lowest overhead.Footnote 15 The number of full-time equivalents involved in managing the RPIA Program was minimal, and it was noted by one group of managers that the current volume of requests was manageable. This finding was consistent with the original plan outlined in program’s background documents. The range of dedicated resources found among comparable programs ranged from zero, as evidenced in the Secretariat, to four full-time equivalents, as evidenced in the Social Sciences and Humanities Research Council of Canada.Footnote 16

According to a few interview groups, outcomes were achieved in the most economical manner possible and the program operated more efficiently than the process prior to the RPIA Program, which required a Treasury Board submission for each RPIA initiative. Nevertheless, the evaluation found that the RPIA Program was not administered as efficiently as possible.

The majority of sponsor groups stated that the program requires a substantial level of administrative involvement from sectors to understand how it works and to meet program requirements. For example, there are no written materials or guidance documents that explain the necessary steps. The program relies solely on sponsors to engage partners rather than having the program promoted on the Secretariat’s external website, enabling third party organizations to approach sponsors.

Although the program provides funds within a short time frame,Footnote 17 sponsors viewed the program as disproportionately burdensome and rigid for beneficiaries, notably international organizations, and for low-value initiatives. According to interview groups, low-funding amounts ($4,000 to $5,000) were not efficiently handled for sponsors, managers, or beneficiaries. In one case, an international partner opted not to go forward because the reporting burden was disproportionate to the value ($5,000) of the initiative.

Another inefficiency noted by one beneficiary was that the requirement under the program’s terms and conditions for the Crown to maintain the intellectual property of deliverables was a barrier for international organizations. “The RPIA program requirements differ from those adopted and agreed to by international organizations, requiring intensive back and forth between countries to resolve the issue.”

One group of sponsors pointed out that the program would be more efficient if delegated approvals were streamlined. Currently, the terms and conditions must be approved by the Sector Head, the Chief Financial Officer, and the Deputy Head. However, a review of the Policy on Transfer Payments found no requirement for these approvals. The jurisdictional review showed that there was little consistency between the approvals processes of the various programs. For example, the highest level of approval varies, from Deputy Head, in the Secretariat, Public Safety Canada, and the Department of Finance Canada, down to Director General, in the Atlantic Canada Opportunities Agency.

In their interviews, a majority of sponsor groups commented that the initiatives generated additional cash contributions from other partners. The document review also showed that the RPIA funds resulted in in-kind contributions, such as free meeting rooms and volunteer hours from the beneficiary organizations. In a few cases, the RPIA initiatives attracted new partners for the program. The key intangible benefits were strengthened partnerships with beneficiary organizations and new partnerships.

4.2.3 Design Alternatives

This evaluation issue presents program design aspects from key informants’ perspectives that have the potential to increase the RPIA Program’s efficiency.

Evaluation Question 2.3 – Design Alternatives

  • What might optimize the performance of the program?

Finding

To optimize the program’s performance and to establish more appropriate administrative requirements for low-value initiatives, the possibility of simplifying the terms and conditions and streamlining approval requirements should be examined.

Use of plain language in the terms and conditions would improve communications with international organizations and expedite the implementation of initiatives.

The program’s administrative processes are the same for initiatives valued at $500,000 and $5,000. Some beneficiary organizations indicated that the time and effort for low-cost initiatives did not offer an enticing value proposition when weighed against the program’s administrative burden. Sponsors recommended that the terms and conditions for low-value initiatives be streamlined.

To reduce the administrative burden on managers, sponsors, and beneficiaries and to increase the turnaround time for processing contribution agreements, there should be a review of the requirement for the Chief Financial Officer and the Deputy Head to approve low-risk, low‑materiality initiatives with long-standing reputable partners.Footnote 18

To facilitate access for international organizations, the program’s terms and conditions should be simplified and clarified. As reported to sponsors, beneficiaries characterized applications as difficult to understand. In particular, because applications contained Canadian legal terminology, international organizations reported delays, owing to concerns from their legal departments. Delays related to interpreting and negotiating the program’s forms were common.

Conclusion: Economy and Efficiency; Design Alternatives

Outcomes were achieved in the most economical manner possible. Although minimal resources were expended to administer the program, a more focused program management approach in line with the program’s low materiality and risk would promote even more efficient operations.

5. Conclusions by Evaluation Issue

5.1 Relevance

The RPIA Program is relevant. Through third party organizations, it responded to the government’s continuing need to develop and access ways of streamlining government operations and enhancing the effectiveness of public service management practices. The program addressed this need by supporting ad hoc initiatives to improve public policy research and enhance public service management practices.

i. Continuing Need for the Program

The RPIA Program met beneficiaries’ needs to a great extent. Beneficiaries expressed interest in participating in the program if it were extended. The program has also met sponsors’ needs, although only two sectors used the program in the last five years.

ii. Alignment with Government Priorities and the Secretariat’s Strategic Outcome

The program’s final outcomes were aligned with both government and the Secretariat’s priorities, given the common focus on sound management practices, increased accountability, and risk management. The RPIA Program’s objectives and initiatives were thematically aligned with the government’s priorities of returning to balanced budgets and making government operations more efficient.

iii. Alignment with Federal Roles and Responsibilities

There is a role for government to strengthen the effectiveness of public service practices, to improve service delivery to the public. The evidence demonstrated that the RPIA Program is aligned with this role by facilitating ad hoc research initiatives that contributed to the advancement of public service management.

5.2 Performance

The RPIA Program achieved the short- and medium-term outcomes articulated in its logic model. However, the lack of performance measures and the inconsistent use of project evaluation grids created difficulty in readily demonstrating evidence of the program’s impacts. The absence of a formal central administrator or program manager led to challenges for sponsors in understanding roles and responsibilities. Sponsors had an uneven understanding of the program, resulting in incomplete tracking of performance measures and inconsistent use of project evaluation grids. A low level of program awareness contributed to the lack of use by some sectors.

The program’s terms and conditions (renewed in 2009) were based on the 2000 version of the Policy on Transfer Payments rather than the 2008 version. This resulted in a lack of alignment with a number of current policy requirements, such as the need to implement performance measurement, service standards, and risk-based assessments for monitoring, as well as requirements to integrate official languages requirements and to post program information on the Secretariat’s external website.

i. Effectiveness

There was evidence that the RPIA Program’s immediate outcomes were achieved to some extent. A more systematic approach to reporting on impacts of initiatives would aid in a clearer demonstration of the program’s overall ability to achieve its stated outcomes.

The program achieved its intermediate outcomes. Through its initiatives, it expanded the Secretariat’s knowledge in areas of leadership, innovation, and risk management. The Secretariat benefited from rich partnerships with prominent national and international organizations.

ii. Economy and Efficiency

The outcomes were achieved in the most economical manner possible. Compared with similar programs operating in the public service, the RPIA Program had the lowest overhead costs. It also leveraged resources (financial and professional) from third party organizations, and donations from other entities.Footnote 19 Although minimal resources were expended to administer the program, a more focused program management approach in line with the program’s low materiality and risk would promote even more efficient operations.

Some aspects of the RPIA Program detract from its efficiency, namely, the terms and conditions, which impose heavy administrative requirements on sponsors and beneficiaries, as well as the requirements for the Sector Head, Chief Financial Officer and Deputy Head to approve all initiatives.

iii. Design Alternatives

The RPIA Program’s design is flexible; however, a risk-based approach to funding (e.g., removing the requirement for Chief Financial Officer and Deputy Head approval of low‑risk, low‑materiality initiatives with long-standing third party organizations) could reduce administrative burden on managers, sponsors, and beneficiaries, and result in faster implementation of initiatives.Footnote 20

6. Recommendations

  1. It is recommended that Corporate Services Sector, in consultation with the Office of the Comptroller General, update the terms and conditions of the RPIA Program and monitor the terms and conditions on an ongoing basis, to ensure conformance with any future updates to the Treasury Board Policy on Transfer Payments.
  2. It is recommended that Corporate Services Sector, in conformance with the Treasury Board Policy on Transfer Payments, post RPIA Program templates, guides, and information products on the Secretariat’s external website for third party organizations and sponsors.
  3. It is recommended that the Corporate Services Sector implement a performance measurement strategy calibrated to the size and risk of the RPIA Program.
  4. It is recommended that the Corporate Services Sector develop options for the designation of a program manager for the RPIA Program, to ensure that materials are up to date, to make the program more visible, and to monitor program outcomes by implementing the performance measurement strategy.
  5. It is recommended that the Corporate Services Sector use a risk-based approach for approvals, to increase the efficiency of the RPIA Program and to reduce administrative burden.

Appendix A: Evaluation Issues and Questions

1.0 Relevance

1.1 Continuing Need for the Program
  • To what extent does the Research and Policy Initiatives Assistance (RPIA) Program address a continuing need?
  • Is the program responsive to departmental and beneficiaries' needs?
1.2 Alignment with Government Priorities and the Secretariat’s Strategic Outcome
  • Do RPIA program objectives align with government and the Secretariat’s priorities?
  • To what extent are RPIA initiatives consistent with the Secretariat’s priorities?
1.3 Alignment with Federal Roles and Responsibilities
  • Is there a role for the federal government in improving, advancing, and promoting public service management practices that contribute to the advancement of public policy research and public service management?

2.0 Performance

2.1 Effectiveness

a. Program Design

  • Are the program design and eligibility criteria appropriate?
  • Are the roles and responsibilities of the program components clear? Appropriate?
  • Is the program’s communications strategy adequate?
  • To what extent does the RPIA Program’s performance measurement system meet management’s information needs?
  • To what extent is the RPIA Program aligned with the Policy on Transfer Payments?

b. Achievement of Immediate Outcomes

  • To what extent is the RPIA Program achieving its immediate outcomes?

c. Achievement of Intermediate Outcomes

  • To what extent is the RPIA Program achieving its intermediate outcomes?

d. Achievement of Final Outcomes

  • To what extent is the RPIA Program achieving its final outcomes?
2.2 Economy and Efficiency
  • To what extent is the RPIA program economical and efficient?
2.3 Design Alternatives
  • What might optimize the performance of the program?

Appendix B: Percentage of Total Cost of Initiatives Funded Through the RPIA Program

Of the 11 initiatives supported over the five-year evaluation period (2008–09 to 2012–13), the Research and Policy Initiatives Assistance (RPIA) Program supported single and multi-year initiatives from two sectors for a total of $442,209. Beneficiaries included national, international, and non-profit, non-governmental organizations.Footnote 21 These initiatives funded a lecture series, a strategic meeting, workshops, and research papers on topics such as the advancement of public sector management, public service reform, leadership, innovation, and the management of emerging risks. Initiatives included delivery of a series of outputs such as workshops sessions, final research papers, final narrative reports, discussion forums, and summary session reports. The profiles of these projects were consistent with those in the previous five-year period, in which $423,674 was awarded.

A review of the total costs of the initiatives identified in contribution agreements and the amounts awarded shows that the program financed between 7 per cent and 48 per cent of the total cost of initiatives. In addition to using their own resources, beneficiaries used donations, funds from registration fees, in-kind contributions from other federal departments (e.g., the Department of Finance Canada, Industry Canada, Natural Resources Canada, the Privy Council Office, and the Bank of Canada) as well as donations from other governments and non-profit organizations to fund the balance.

Table 4. Percentage of Total Cost of Initiatives Funded Through the RPIA Program
Project Year RPIA Initiative ($) Total Costs of Initiativetable 4 note * ($) % of Total Cost

Table 4 Notes

Table 4 Note 1

Information taken from the contribution agreements for the initiatives.

Return to table 4 note * referrer

Table 4 Note 2

The Secretariat’s contribution accounted for 36.5 per cent of the overall total initiatives ($1,211,008). Beneficiary organizations paid the balance.

Return to table 4 note ** referrer

Public Policy Forum, “Good Government is Good Politics” 2008–09 28,366 79,750 35.6
Public Policy Forum, “Adapting Public Policy to a Changing World” 2009–10 20,000 82,500 24.2
Public Policy Forum, “A Vision Too Far?” 2010–11 20,000 50,640 39.5
Public Policy Forum, “On Authenticity – How the Truth Can Restore Faith in Politics and Government” 2011–12 20,000 56,190 35.6
Institute on Governance, “Agents of Parliament Project” 2011–12 33,443 98,450 34.0
Organisation for Economic Co-operation and Development (OECD), "Communicating and Measuring Progress in Regulatory Reform” 2011–12 150,000 310,000 48.4
Centre for the Study of Living Standards, Festschrift Volume, “New Directions for Intelligent Government in Canada” 2011–12 5,000 67,615 7.4
International Risk Governance Council, “Public Sector Governance of Emerging Risks” 2011–12 50,000 150,400 33.2
OECD, “The 2011 Meeting of the Network of Senior Officials from the Centers of Government” 2011–12 63,000 148,400 42.5
OECD, “Development of the Observatory for Public Sector Innovation” 2012–13 32,400 91,063 35.6
Public Policy Forum, “Insights on the Relevance of the Cooperative Model for Public Service in Canada” 2012–13 20,000 76,000 26.3
Total 442,209 table 4 note ** 1,211,008 36.5

Management Response and Action Plan

Program management has reviewed the evaluation and agrees with the recommendations of the report.

Recommendations Proposed Action Start Date Targeted Completion Date Office of Primary Interest
Recommendation 1
It is recommended that Corporate Services Sector, in consultation with the Office of the Comptroller General, update the terms and conditions of the RPIA Program and monitor the terms and conditions on an ongoing basis to ensure conformance with any future updates to the Treasury Board Policy on Transfer Payments.
We agree with the recommendation. Corporate Services Sector will update the terms and conditions of the RPIA Program, to ensure conformance with the Policy on Transfer Payments, and will also consider opportunities to modernize the RPIA Program. Director, Financial Strategies and Reporting
Recommendation 2
It is recommended that Corporate Services Sector, in compliance with the Treasury Board Policy on Transfer Payments, post RPIA Program templates, guides, and information products on the Secretariat’s website for third party organizations and sponsors.
We agree with the recommendation. Pending the results of Recommendation 4, Corporate Services Sector will work with other sectors (for example, Strategic Communications and Ministerial Affairs) to develop a communications strategy and to leverage appropriate communication vehicles. Director, Financial Strategies and Reporting
Recommendation 3
It is recommended that the Corporate Services Sector implement a performance measurement strategy calibrated to the size and risk of the RPIA Program.
We agree with the recommendation. Corporate Services Sector, supported by the Internal Audit and Evaluation Bureau, will review the performance measurement strategies of comparable programs, with a view to implementing a strategy calibrated to the size and risk of the RPIA Program. Director, Financial Strategies and Reporting
Recommendation 4
It is recommended that the Corporate Services Sector develop options for the designation of a program manager for the RPIA Program, to ensure that materials are up to date, to make the program more visible, and to monitor program outcomes by implementing the performance measurement strategy.
We agree with the recommendation. Corporate Services Sector agrees that the RPIA Program will benefit from a central administrator. Corporate Services Sector will work with other sectors to select a program manager. Director, Financial Strategies and Reporting
Recommendation 5
It is recommended that the Corporate Services Sector use a risk-based approach for approvals, to increase the efficiency of the RPIA Program and to reduce administrative burden.
We agree with the recommendation. Given the size and cost of the program, Corporates Services Sector agrees that implementing a risk-based approach for approvals is sensible. Corporates Services Sector will submit new delegation of authority levels for the RPIA Program to the President of the Treasury Board for approval. Director, Financial Strategies and Reporting
Report a problem or mistake on this page
Please select all that apply:

Privacy statement

Thank you for your help!

You will not receive a reply. For enquiries, contact us.

Date modified: