Archived: Integrated Risk-Based Audit and Evaluation Plan 2014-2015: chapter 7

3. Planning Approach

3.1 Key Requirements

There are a number of requirements and obligations stemming from TB policies, directives and guidelines which drive audit and evaluation planning in the federal government.  The more significant of these are summarized in the following sections.

A. Internal Audit

• Internal audit provides independent and objective appraisals that use a disciplined, evidence-based approach to assess and improve the effectiveness of risk management, control and governance processes. It is intended to contribute to the basis by which decision-makers exercise oversight and control over the organization and apply sound risk management^Footnote1. 
• The DM must approve and provide annually to the Comptroller General a departmental multi-year risk-based internal audit plan that considers:
  • departmental areas of higher risk and significance;
  • predominantly focussed on the provision of assurance services;
  • government-wide audits led by the Comptroller General; and
  • the recommendations of the EAAC.
• There are other important audit planning considerations that stem from OCG guidance and professional standards. For instance, the draft guide on Internal Audit Planning^Footnote2 recommends the periodic conduct of specific audits of risk management and of governance. As well, OCG guidance and internal audit professional standards recommend the conduct of an overall fraud risk assessment on a periodic basis.

B. Evaluation

• According to TB Policy, evaluations must evaluate the relevance and performance of Departmental programs^Footnote3. The DM is required to submit to TBS annually a rolling five-year departmental evaluation plan.  The plan's required coverage is twofold:
  1. The Financial Administration Act (section 42.1) requires evaluations be performed of all ongoing grant and contribution (G&C) programs on a five-year cycle.  These are to be performed before the corresponding program comes up for TB renewal^Footnote4;
  2. The TB Policy and Directive on Evaluation require that all direct program spending (excluding G&C covered above) be evaluated on a five-year cycle.
• Mandatory program evaluations are normally required for renewals of programs (not limited to G&C programs).  As well, specific evaluation requirements or commitments can be specified under program TB submissions and related TB decisions. 
• Evaluation plans must also:
  • Align with the Management, Resources and Results Structure (MRRS);
  • Support the Expenditure Management System, including strategic reviews;
  • Include other program specific evaluations or applicable elements of the Government's Evaluation Plan if requested by TBS; and
  • Consider as part of individual evaluations any requirements stemming from
    Performance Measurement and Evaluation Plans for regulatory programs^Footnote5.
• During 2013-14, a new draft Guide^Footnote6 on the preparation of a Departmental Evaluation Plan was circulated by TBS for review, but was not yet finalized or formally issued at the time this RBAEP was initially being developed.  While anticipated elements^Footnote7 of the new draft guidance have been considered where appropriate, AEB has ensured that the RBAEP is based on existing requirements and current guidance.  Also, some components on the new guidance will be addressed through the AEB's Annual Report.  The RBAEP and Annual Report combined are expected to be consistent with the new TBS Guide once it takes effect.

Accordingly, the evaluation plan and projects are determined mainly by mandatory requirements as described above. The risk assessment is used to help scope ("calibrate") the evaluation projects as well as determine priorities for non-mandatory projects.  The required evaluation project effort is mainly driven by the nature (e.g. complexity, evaluability^Footnote8, horizontality) and scope (materiality) of the program. 

3.2 Approach And Considerations   

The starting point for AEB's annual planning exercise is always based, to some extent, on the previous year's risk assessment and plans. The approach to the planning exercise is also grounded on the following key elements and principles:

1. Planning and Reporting Cycle. The RBAEP must be properly considered as part of the AEB annual planning and reporting cycle. This cycle incorporates the following key components, which integrate both the internal audit and evaluation functions:

• A planning exercise is conducted before the start of each fiscal year, in consultation with senior executives, EAAC and DEC, to review AEB's plans and projects.  The result is the Integrated RBAEP approved by the DM.
• At mid-year, AEB's progress against the RBAEP is reviewed in consultation with EAAC and DEC, and the RBAEP is updated as required and any revisions approved.
• At the end of each fiscal year, the Chief Audit and Evaluation Executive prepares his Annual Report^Footnote9 which he submits to EAAC, DEC and the Deputy Minister.

The planning and reporting cycle reflects that the AEB plans and priorities are evolving documents and must remain evergreen.  In addition to the mid-year RBAEP review and update, progress against the RBAEP is regularly monitored at both EAAC and DEC.

2. Planning "Universe".  The planning universe serves as the "roadmap" or "backdrop" for risk assessment and planning, and helps define the potential audit and evaluation areas and projects:

• For evaluation, the planned projects are defined according to EC’s 2014-15 Program Alignment Architecture (PAA), which aligns and supports EC's Management, Resources and Results Structure and is consistent with TB Evaluation Policy requirements (e.g., Direct Program Spending coverage).  Hence, the evaluation plan also supports the Expenditure Management System, including strategic reviews;
• For internal audit, the key elements of TBS's Management Accountability Framework (MAF) are utilized to identify and organize (map) the planned audit projects. 

3. Risks and Considerations. As reflected above (section 3.1), both internal audit and evaluation rely in part on risks, to either identify or scope projects, in addition to other requirements and considerations.  Accordingly, as part of its planning exercise AEB conducts an independent risk assessment, based on the following steps:

• A preliminary risk assessment based on both AEB's knowledge of EC's programs and priorities, and on consideration of a number of key sources (e.g., prior year's assessment; past audits and evaluations; CRP exercise; latest MAF assessment; and key corporate documents); and,
• The assessment is further developed primarily through consultations with senior executives, the EAAC, the DEC and the DM.

The results of AEB's risk assessment are reflected in part in the detailed audit plan (appendix A. "Risks and Rationale" column), while a more complete summary of the risk assessment is provided to EAAC and DEC as a separate document.

4. Balanced Approach. The current RBAEP is revised and updated from the previous year, by taking into account a number of competing requirements and factors, mainly:

• Internal audit requirements and coverage of key risks and considerations;
• Mandatory evaluation coverage and risks for non-mandatory evaluations;
• Audits or reviews to be conducted by external assurance providers (e.g. CESD) and horizontal audits planned by the OCG;
• Ability or capacity of EC branches to accommodate multiple projects;
• Comments and advice received from senior executives; and,
• AEB resources, capacity and expertise.

5. Process and Approval. The development of the RBAEP is founded foremost on significant phased consultations with senior executives, and accordingly the exercise is largely iterative.  This year, the overall process included the following major milestones:

• Consultations with EMC members including the DM (January-February);
• Initial consultation with EAAC members (February);
• Initial Draft RBAEP provided to EAAC for review and recommendation (March);
• Initial Draft RBAEP provided to DEC for comments (April);
• RBAEP Summary provided to EMC for information (May);
• Evaluation plan revised based on Government's schedule for expected renewal of sunsetter programs (June)
• Final RBAEP approved by the DM; and
• Approved RBAEP provided to TBS, OCG and OAG
  (and subsequently translated and posted on the EC web-site).

This final RBAEP is approved by the DM, based on the recommendation of the EAAC (audit plan component) and comments by the DEC (evaluation plan component).