Integrated Risk-Based Audit and Evaluation Plan 2016 to 2017: chapter 4

3. Planning approach

3.1 Key requirements

There are a number of requirements and considerations stemming from TB policies, directives and guidelines which drive audit and evaluation planning in the federal government. The more significant of these are summarized in the following sections.

However, at the time of the preparation of this plan, revisions to the Internal Audit and Evaluation policies were underway as part of TBS’s broad Policy Suite Reset initiative. For example, the new TB Policy on Results was released in July 2016, together with supporting instruments. As well, consultations were underway on the proposed revised Internal Audit Policy and Directive. Accordingly, where possible and practical, the expected requirements of the new policies have been considered. Nonetheless, this year’s RBAEP will largely remain a transitional document, and the new policies and directives will be more fully reflected as part of the next planning exercise and next year’s RBAEP.

A. Internal audit

Internal Audit provides independent and objective appraisals that use a disciplined, evidence-based approach to assess and improve the effectiveness of risk management, control and governance processes. It is intended to contribute to the basis by which decision-makers exercise oversight and control over the organization and apply sound risk management^Footnote1.

The DM must approve and provide annually to the Comptroller General a departmental multi-year risk-based internal audit plan that considers:

• departmental areas of higher risk and significance;
• predominantly focused on the provision of assurance services;
• government-wide horizontal audits led by the Comptroller General; and
• the recommendations of the EAAC.

There are other planning considerations that stem from previous OCG audit guidance and professional standards. For instance, the draft guide on Internal Audit Planning recommends the periodic conduct of specific audits related to risk management and governance^Footnote2.As well, OCG guidance and internal audit professional standards recommend the conduct of an overall fraud risk assessment on a periodic basis^Footnote3.

B. Evaluation

In the Government of Canada, evaluation is a systematic and neutral collection and analysis of evidence to inform decision-making, improvements and accountability, based primarily on considerations of program relevance, effectiveness and efficiency. The DM is required to submit to TBS annually a departmental evaluation plan. The plan's coverage must consider:

1. The FAA (section 42.1) requires the review of each ongoing program of grants and contributions every five years^Footnote4. The new Policy on Results exempts programs of grants and contributions that have average expenditures of less than $5M per year, and assessed contributions to international organizations;
2. Evaluations are also normally required for renewals of programs, and specific evaluation requirements can also be specified under TB submissions or decisions.

The new TB Policy on Results and its related instruments indicate that the Deputy Head is responsible for maintaining a robust and neutral evaluation function, and approving annually in the manner prescribed by the Secretariat a five-year rolling departmental evaluation plan that:

• is informed by a planning exercise that includes consultation with TBS;
• presents planned evaluation coverage (including spending coverage and programs in the Program Inventory);
• identifies a rationale for spending and programs not scheduled to be evaluated;
• includes all ongoing grant and contribution programs greater than $5M (average);
• includes all evaluations required by legislation, requested by TBS, or evaluation activities in support of centrally-led evaluations.

These planning requirements essentially become effective April 1, 2017, and in the interim the Policy stipulates that departments should continue to plan as required under the previous policy instruments being replaced.

Upon the advice of the DEC, AEB is presenting this evaluation plan as a transitional plan and has incorporated new policy requirements where feasible. As such, this evaluation plan is also presented on an exception basis as a 2 year plan only (2016-17 and 2017-18), until the requirements and flexibilities of the new Policy on Results are better assessed and ECCC’s Departmental Results Framework and Program Inventory are further developed. As well, any current plan beyond 2017-18 would be tentative and subject to considerable changes as a result of the new Policy.

3.2 Approach and considerations

The starting point for AEB's annual planning exercise is always the previous year's risk assessment and plans. The approach to planning is also grounded on the following key elements and principles:

1. Planning and Reporting Cycle. The RBAEP must be considered as part of AEB’s annual planning and reporting cycle, which helps ensure the plans evolve and remain current (“evergreen”). This cycle incorporates the following key components, which integrate both the internal audit and evaluation functions:

• A planning exercise is conducted before the start of each fiscal year, in consultation with senior executives, EAAC and DEC, to review AEB's plans and projects. The result is the Integrated RBAEP approved by the DM;
• At mid-year, the progress against the RBAEP is reviewed in consultation with EAAC and DEC, the RBAEP is updated as required and any revisions approved. In addition, progress against the RBAEP is regularly monitored at both EAAC and DEC.
• In the subsequent fiscal year, the Chief Audit and Evaluation Executive (CAEE) prepares an Annual Report^Footnote5which he submits to EAAC, DEC and the DMs.

2. Planning "Universe". The planning universe serves as the "roadmap" for assessing risk and planning, and helps structure the potential audit and evaluation areas and projects:

• For evaluation, the planned projects are defined according to ECCC’s 2016-17 Program Alignment Architecture (PAA), which is consistent with current Policy requirements;
• For internal audit, the key elements of TBS's Management Accountability Framework (MAF) are utilized to identify and organize the planned audit projects.

3. Risks and Considerations. Both internal audit and evaluation rely to some extent on risks, to either identify or help scope planned projects, in addition to other considerations. AEB’s independent planning assessment is based on the following steps:

• An initial review based on both AEB's knowledge of ECCC's programs and priorities, and consideration of a number of key sources and documents;
• Consideration of other related risk assessments (e.g. OCG’s three-year risk-based internal audit plan, AEB’s previous Fraud Risk Assessment);
• Consideration of similar risks and projects identified by key partner departments (e.g. science-based departments and agencies); and
• Consultations with senior executives, as well as with EAAC, DEC and the Deputies.

The results of the risk assessment are summarized in part in the detailed audit plan (appendix A. "Risks and Rationale" column).

4. Balanced Approach. The current RBAEP is developed by taking into account a number of competing requirements and factors, mainly:

• Evaluation and Internal Audit requirements, and key risks and considerations;
• Audits or reviews to be conducted by external assurance providers (e.g. CESD, OAG or the Public Service Commission (PSC)), and horizontal audits planned by the OCG;
• Opportunities for collaboration with other science-based departments and agencies;
• Ability or capacity of ECCC branches to accommodate multiple projects; and
• AEB resources, capacity and expertise.

5. Process and Approval. The development of the RBAEP is founded on significant consultations with senior executives, and the exercise is largely iterative. The process included the following major milestones:

• Consultations with Executive Management Committee members including the DMs;
• Initial Draft RBAEP provided to EAAC for review and recommendation (audit plan components);
• Initial Draft RBAEP provided to DEC for comments (evaluation plan components);
• Final RBAEP approved by the DM, based on the recommendation of EAAC and comments by DEC (evaluation plan component); and provided to TBS, OCG, OAG and posted on ECCC’s website.