Regional Development Agencies Internal Audit of Recipient Selection – Phase II

What the purpose was

The objective of this audit was to assess the design and implementation of the management control frameworks over the process for selecting recipients in Regional Development Agencies (RDAs).

To accomplish this objective, the audit determined whether the following were in place in the RDAs during the period under review:

• governance
• application assessments
• funding agreements

What was audited

The audit scope included management control frameworks for the process of selecting recipients for the following RDAs:

• Atlantic Canada Opportunities Agency (ACOA)
• Federal Economic Development Agency for Southern Ontario (FedDev Ontario)

Government agenda and delivering results for Canadians

Transfer payments are a large part of the Government of Canada’s spending and are one of the key instruments it uses to achieve policy objectives and priorities. RDAs use these instruments to contribute to successful, diversified and participative local economies by providing regionally tailored programs, services, knowledge and expertise for the benefit of all Canadians.

Public trust and confidence

Transfer payment programs must be designed, delivered and managed in a way that is fair, accessible and effective for all involved, including departments, applicants and recipients.

Program management

Management control frameworks over programs:

• help ensure that funds are spent prudently, equitably and according to policies and legislation
• help avoid concerns about program performance and accountability

RDA resources

Each RDA is mandated to promote economic development for its region. Such development is primarily done through transfer payment programs. As such, the majority of RDAs’ human and financial resources are dedicated to funding, delivering and managing transfer payment programs.

RDA mandate and program objectives

Once a project is funded, RDAs have limited control over the realization of a project’s results or ultimate outcomes. As such, a robust process for selecting recipients is one of the most effective ways through which RDAs can increase the likelihood that:

• project outcomes are realized
• program objectives are achieved
• the mandates of RDAs are fulfilled

As a result of these considerations, this audit was identified as an area of high inherent risk for RDAs during consultations for the Office of the Comptroller General’s 2016-2020 RDA Risk-Based Audit Plan.

Atlantic Canada Opportunities Agency (ACOA)

Figure 1: map of ACOA’s operating area - Text version

The image is a map of the provinces in which ACOA operates: New Brunswick, Nova Scotia, Prince Edward Island, and Newfoundland and Labrador.

ACOA’s core mandate is to create opportunities for economic growth in Atlantic Canada by:

• helping businesses become more competitive, innovative and productive
• working with diverse communities to develop and diversify local economies
• championing the strengths of Atlantic Canada

ACOA is headquartered in Moncton, New Brunswick, with regional headquarters in each Atlantic provincial capital city.

For more information regarding the Agency, please visit ACOA’s website.

Figure 2: ACOA’s total budget for the 2016-2017 fiscal year - Text version

This image is a pie chart that represents ACOA’s budget separated into two groups: Total Agency Transfer Payments $262 Million (79% of the Agency’s budget); and Other $69 Million (21% of the Agency’s budget).

For more information regarding ACOA’s total budget for 2016-2017 fiscal year, please refer to the Public Accounts of Canada 2016-2017: Expenditures.

Federal Economic Development Agency for Southern Ontario (FedDev Ontario)

Figure 3: map of FedDev Ontario’s operating area - Text version

The image is a map of the province of Ontario with a circle around southern Ontario depicting the area in which FedDev Ontario operates.

FedDev Ontario’s core mandate is to:

• strengthen southern Ontario’s economic capacity for innovation, entrepreneurship and collaboration
• promote the development of a strong and diversified southern Ontario economy

FedDev Ontario is headquartered in Waterloo, Ontario, with additional offices in Ottawa, Toronto and Peterborough.

For more information regarding the Agency, please visit FedDev Ontario’s website.

Figure 4: FedDev Ontario’s total budget for the 2016-2017 fiscal year - Text version

This image is a pie chart that represents FedDev Ontario’s budget separated into two groups: Total Agency Transfer Payments $193 Million (87% of the Agency’s budget); and Other $28 Million (13% of the Agency’s budget).

For more information regarding FedDev Ontario’s total budget for 2016-2017 fiscal year, please refer to the Public Accounts of Canada 2016-2017: Expenditures.

Transfer payment programs included

Transfer payment programs were selected in consultation with each RDA. The selection criteria included the value to the RDAs of including each program, programs managed in regional offices, and programs that were ongoing and were on a direct delivery basis. For details on the programs audited, refer to Appendix F.

ACOA

• Atlantic Innovation Fund
• Business Development Program
• Innovative Communities Fund

FedDev Ontario

• Advanced Manufacturing Fund
• Economic Development Initiative
• Investing in Business Innovation
• Investing in Commercialization Partnerships
• Investing in Business Growth and Productivity
• Investing in Regional Diversification

Governance

• The RDAs implemented processes that generally support citizen- and recipient-focused selection processes.
• The transfer payment programs, policies, roles and responsibilities generally support alignment with the Policy and the Directive.
• Although the RDAs have strategies to measure program-level performance, strategies to measure performance of the process to select recipients varied between the RDAs.
• Both RDAs developed and implemented tools to assess applications.
• Opportunities for improvements in governance were noted for both RDAs.

Application assessments

• Assessing a funding application generally considers whether projects support the achievement of program objectives and outcomes. The extent of such consideration varied between the RDAs and between programs.
• Funding applications were generally assessed consistently against:
  • program Ts&Cs
  • established guidance and expectations
• Some opportunities for improvement were identified for both RDAs.

Funding agreements

• Both RDAs used standard funding agreement templates that aligned with the Ts&Cs of the applicable program.
• The extent to which standard funding agreements aligned with the Directive varied between the RDAs.
• The funding agreements used for the projects sampled aligned with their respective templates. The impact of risk considerations on funding agreements varied between the RDAs.

Although the design and implementation of the management control frameworks varied between the two RDAs, they are generally aligned with and support the achievement of the expectations and objectives outlined in the Policy on Transfer Payments.

Conformity statement

This audit engagement was conducted in conformance with the International Standards for the Professional Practice of Internal Auditing.

Mike Milito, MBA, CIA, CRMA
Assistant Comptroller General and Chief Audit Executive
Office of the Comptroller General

Governance findings 1: citizen- and recipient-focused processes

Governance findings 2: program design, guidance and roles and responsibilities

Governance findings 3: performance measurement

What was expected

• Program performance measurement strategies exist.
• Performance measurement strategies include process-related activities to select recipients.
• Performance of recipient selection activities is monitored and reported on.
• Application assessments consider project contributions to program objectives.

Key takeaways

• Both RDAs have strategies to measure program performance that, to varying degrees, include process-related activities to select recipients. The comprehensiveness of monitoring and reporting on the performance of the process to select recipients varied between the RDAs.
• Application assessments generally included consideration of projects’ support of program objectives.

Why this is important

Performance measurement strategies:

• help measure the effectiveness of the recipient selection process
• strengthen accountabilities for all involved

Ensuring that all applications are assessed for project contributions to program objectives:

• harmonizes and standardizes the assessment process across all programs
• helps ensure that projects selected will increase the likelihood of achieving program objectives

What was found

Recipient selection process performance

• ACOA identified, tracked, monitored and reported to senior management on performance targets for processes to select recipients.
• FedDev Ontario tracked certain statistics related to the process to select recipients.

Figure 5: Percentage of applications assessed that identified project results, program objectives, how projects contribute to achieving program objectives, and how project results align with program objectives - Text version

These bar graphs identify the percentage of applications assessed that:

1. Identified expected project results:
   • Both Agencies had 100% in this area.
2. Identified program objectives to which the project aims to contribute:
   • ACOA had 85% in this area, while FedDev Ontario had 100% in the same area.
3. Analyzed how projects contribute to the achievement of program objectives:
   • ACOA had 70% in this area, while FedDev Ontario had 100% in the same area.
4. Expected project results are aligned with program objectives:
   • Both Agencies had 100% in this area.

Table 6: impacts and recommendations of performance measurement

Impacts of findings	Recommendations
Selecting, monitoring and reporting on recipient selection performance targets: assists with continuous process improvements provides decision-makers with information that signals a need to adjust promotional/outreach activities to ensure that applications received are for projects that contribute to program objectives	FedDev Ontario management should formally identify performance targets for the recipient selection process. Once adopted, management should: communicate the targets track, monitor, and report on related performance (See Recommendation 4)
Assessing applications for their contribution to program objectives helps contribute to balanced assessments that result in approving projects that: further the RDAs’ goals help RDAs fulfill their mandates	Findings for individual application assessments regarding program objectives are linked to those identified in this audit’s governance-related section on guidance and tools. As such, no additional recommendations have been included.

Application assessment findings

What was expected

• Applications were consistently assessed against program Ts&Cs in key areas.
• Application assessments consistently considered key areas in accordance with RDA guidance and expectations.
• Application assessments followed RDA guidance and expectations in key areas.

Key takeaway

The assessment of applications against program Ts&Cs, established guidance and expectations was generally completed.

Why this is important

Consistent assessments provide management and Canadians with assurances that transfer payment programs are designed and delivered in a manner that is prudent, equitable and in accordance with applicable guidance, policies, directives and expectations.

What was found

Figure 6 below depicts the overall findings for the application assessments sampled and examined. The examination covered the areas below in terms of consistency of assessments and alignment with guidance and expectations (that is, program Ts&Cs, the Policy and Directive, and RDA guidance and expectations):

Figure 6: overall findings for assessments of applications sampled and examined

RDA	Eligibilityfigure 6 note *	Stacking Limits	Contribution Type	Program Objectives	Risksfigure 6 note *	Project Feasibility and Viability	Ability to Repay	Service Standards	Separation of Dutiesfigure 6 note *	Communication	Delegated Authority – Section 32	Proactive Disclosure
High Risk Area Medium-High Risk Area Medium Risk Area Low Risk Area
Figure 6 Notes Figure 6 Note 1 Through additional audit work, in collaboration with the RDAs, the audit team confirmed that there were no residual risks within the sampled and examined application assessments in the areas noted as “Medium-High Risk Area” or higher. For example, although eligibility was assessed inconsistently between projects, all sampled projects assessed by the audit team were found to have been appropriately deemed to be eligible or ineligible based on the information available. Return to figure 6 note * referrer
ACOA	High Risk Area	Low Risk Area	Low Risk Area	Medium Risk Area	Low Risk Area	Low Risk Area	Low Risk Area	Low Risk Area	Medium-High Risk Area	Low Risk Area	Medium Risk Area	Low Risk Area
FedDev Ontario	Medium-High Risk Area	Low Risk Area	Low Risk Area	Low Risk Area	Medium-High Risk Area	Medium Risk Area	Low Risk Area	Low Risk Area	Low Risk Area	Medium Risk Area	Low Risk Area	Low Risk Area

Table 8: impacts and recommendation based on application assessment findings

Impacts of findings	Recommendation
Separation of duties: helps ensure that: challenge functions over assessments are completed individuals are not assessing and approving projects that are in their own interest contributes to consistent levels of due diligence	Findings for the individual application assessments are directly linked to those identified in the governance-related sections on Guidance, Program Objectives, and Tools (refer to Findings – Governance). As such, no additional recommendations have been included.
Ensuring that all eligibility requirements are consistently assessed and guidance is adhered to helps ensure that only those applicants and projects that are aligned with program Ts&Cs are the ones that receive funding.
Assessing applications for their contribution to program objectives helps contribute to balanced assessments that: result in projects that further the RDAs’ goals help them fulfill their mandates
Completed risk and viability assessments ensure that: key considerations are made sufficient information and analysis is available for due diligence that supports a rejection or approval decision processes are harmonized and followed

Recommendations for ACOA

Recommendations for FedDev Ontario

Priority ratings

Priority ratings were established for the purpose of ranking recommendations relative to one another and are not meant to reflect overall risks for the RDAs.

Recommendations for both RDAs

Priority ratings

Priority ratings were established for the purpose of ranking recommendations relative to one another and are not meant to reflect overall risks for the RDAs.

Acknowledgement

The audit team gratefully acknowledges ACOA and FedDev Ontario staff at their head offices and at the regional offices. Their contributions were essential to the completion of this audit.

The findings and recommendations of this audit were presented to ACOA and FedDev Ontario management.

Management has agreed with the findings in this report and will take action to address the recommendations that apply to their RDA.

ACOA’s best practices

• ACOA presents dashboard updates to its Executive Committee on progress made against targets for its process to select recipients.
• ACOA’s Newfoundland and Labrador region has presented programming information to local accounting firms to help them support their clients in managing ACOA funding agreement expectations.
• ACOA uses one document to support an assessment decision to:
  • fund or reject a project
  • approve funding decisions

FedDev Ontario’s best practices

• FedDev Ontario uses program-specific eligibility checklists to ensure that all eligibility requirements are assessed to justify further project assessment or statutory rejection.
• FedDev Ontario’s reporting requirements are based on the results of the risk assessment for the client and project.
• Across programs, FedDev Ontario’s websites:
  • contain similar key information
  • are constructed in a similar fashion to facilitate accessibility and understanding of program information

Financial Administration Act

The Financial Administration Act provides for:

• the financial administration of the Government of Canada
• the establishment and maintenance of the accounts of Canada
• the control of Crown corporations

Policy on Transfer Payments

The Policy on Transfer Payments sets out roles, responsibilities and mandatory requirements to ensure that transfer payment programs are managed with integrity, transparency and accountability in a manner that is:

• sensitive to risks
• citizen- and recipient-focused
• designed and delivered to address government priorities in achieving results for Canadians

Directive on Transfer Payments

The Directive on Transfer Payments supports the objectives of the Policy on Transfer Payments by providing operational requirements for departmental managers who have been assigned responsibilities for the management of transfer payments. It also provides for the tailoring of certain general requirements to reflect relationships with specific categories of recipients.

Governance

1.1. The Terms and Conditions of the programs are aligned with the Treasury Board Submissions, the requirements of the Treasury Board Policy and Directive on Transfer Payments and applicable legislation.

1.2. There are appropriate policies, procedures and practices, and clear roles and responsibilities including proper separation of duties that govern the recipient selection process.

1.3. There is a promotional strategy that clearly communicates program information to potential applicants.

1.4. There is a strategy to engage applicants and recipients in support of innovation, continuous improvement and the establishment of positive relations.

1.5. There is a performance measurement strategy in place for the oversight of the recipient selection process and reporting of performance of program objective.

Program applications

2.1. Eligibility of applications is consistently assessed against established criteria, which are based on the program’s Terms and Conditions.

2.2. Funding decisions are made in accordance with approved Terms and Conditions and are prioritized based on the recipient’s ability to contribute to program objectives.

Program funding agreements

3.1. Standard funding agreements are used, and they are aligned with the approved Terms and Conditions and comply with the Treasury Board Policy and Directive on Transfer Payments.

3.2. Funding agreements address recipient risk in accordance with the program’s risk management strategy.

3.3. Funding agreements are approved under section 32 of the Financial Administration Act prior to signing.

Transfer payments

Transfer payments are monetary payments, or transfers of goods, services or assets to third parties. They represent a large part of the Government of Canada’s spending. Their tangible results touch the lives of Canadians and others every day, and cover all sectors of society. Transfer payments include grants, contributions, and other transfer payments.

The transfer payment life cycle

The transfer payment life cycle consists of:

• program design
• promotion and communication
• assessment and selection
• funding agreements
• payment
• monitoring
• evaluation and reporting

The process for selecting recipients

The process for selecting recipients consists of:

• promoting, communicating and raising awareness of program objectives, criteria and administrative requirements
• assessing applications received and selecting recipients
• drafting, negotiating and signing funding agreements

Objective of the Policy

The objective of the Policy on Transfer Payments is to ensure that transfer payment programs are:

• managed with integrity, transparency and accountability
• sensitive to risks
• citizen- and recipient-focused
• designed and delivered to address government priorities in achieving results for Canadians

Expected results of the Policy

The expected results of transfer payment programs are that they:

• are supported by clearly defined and understood roles, responsibilities and accountabilities
• are designed, delivered and managed in a manner that takes account of risk and demonstrates value for money
• are supported by cost-effective oversight and control systems at both departmental and government-wide levels
• are accessible, understandable and useable by applicants and recipients to ensure effective control, transparency and accountability