Archived: Integrated Risk-Based Audit and Evaluation Plan 2014-2015: chapter 8

4. Integrated Plan

As reflected in the previous section, the proposed audit and evaluation plan strives to seek a balance between a number of different competing requirements and considerations, and does so in the context of the AEB's resources (sub-section 4.5 and appendix C.).  In addition to the Policy requirements and the Departmental priorities outlined previously, the audit and evaluation plans cover the key aspects of the EC's programs and operations, as detailed in this section. 

Please note that an integrated summary of the multi-year audit and evaluation plans is presented in section 4.6, and the more detailed audit and evaluation plans are presented in appendices A and B. 

4.1 Internal Audit Plan

A.  Overview

The AEB's Internal Audit Directorate (IA) provides assurance and advice to the DMs and senior management through objective and independent evidence-based information and assessment on the adequacy of management practices, controls and accountability mechanisms.  Other activities can include providing management advisory services or conducting special assignments and investigations.  IA also monitors the implementation of past audit recommendations and related management action plans to provide assurance that previous audit findings and issues are effectively addressed. 

B. Plan Summary

The internal audit plan includes a total of 22 projects to be completed over the next three years, of which 10 projects were in progress at the start of 2014-15.  The plan is based on available resources consistent with the approved Branch budget for 2014-15 (appendix C) and reflects a total staff complement of 9 Full Time Equivalents (FTEs), excluding 2 FTEs for external audit liaison, and relies partially on external professional services. 

• Of the 22 projects planned or in progress‚ the majority are carried-forward from last year’s audit plan, while 3 projects were added and are aligned to key risks or senior management’s recommendations.
• 2 projects included in the plan are based in part on central agency guidance or requirements (i.e.‚ Fraud Risk Assessment; Governance Review). Another project is intended to be conducted in collaboration with the OCG as per its horizontal audit plan (IT Security).

The detailed internal audit plan (appendix A) briefly identifies all assurance and other engagements planned for the next three years, and provides initial project descriptions and rationale (i.e. link to departmental priorities, MAF and risks). 

C.  Internal Audit Core Areas

Internal Audit is intended to provide independent assessment of the effectiveness of the Department's risk management, control and governance processes (core internal audit areas). The AEB has considered in its multi-year audit plan the necessary coverage of these topics:

• All internal audit projects will consider key management and internal controls processes related to the specific areas they cover.  Also MAF core controls^Footnote10 are always considered as part of the planning phase of each audit.
• An internal audit of risk management was completed in 2011-12. As well, most audit projects included in the plan cover risk management to some extent, while some more clearly focus on risk management, in particular the following: Fraud Risk Assessment, Information Technology Security, and Emergency Management and Planning.
• In regard to governance, the plan includes a Governance Framework Review (starting in 2015-16).

D. Departmental Risks, Priorities and MAF

The three-year internal audit plan takes into consideration alignment with key risks and considerations, departmental priorities and the key elements of the Government's Management  Accountability Framework (MAF).  Specifically, the detailed audit plan (appendix A) indicates, for each audit project, the link with the key risks identified through AEB's risk assessment exercise, the link to EC's key priorities and the link to key elements of the MAF.

4.2 Evaluation Plan

A. Overview

The Evaluation Directorate ensures that departmental senior management receives timely, strategically focused, objective and evidence-based information on the relevance and performance of EC programs and initiatives. Other activities include providing expert advice and services in the areas of evaluation and performance measurement, and reviewing TB submissions and Memoranda to Cabinet to ensure existing evaluation evidence is considered and future evaluation commitments identified.

B. Plan Summary 

The detailed evaluation plan presented in appendix B includes a total of 39 funded evaluation projects over the next five years, and is based on an expected staff complement of 10 FTEs.  The plan is for the most part consistent with the anticipated resources identified in the AEB initial budget for 2014-15 ( refer to appendix C). The evaluation plan was developed using the following general planning approach to identify projects to be evaluated during each of the next five years:

• The overriding planning priority has been to ensure the statutory requirements of the Financial Administration Act (FAA, section 42.1), and all evaluation commitments specified in TB submissions, are fulfilled.  Accordingly, the plan provides for full (100%) coverage of all transfer payment programs (G&C) and existing TB evaluation commitmentswithin its 5 year planning period.
• The current plan includes a total of 39 distinct evaluation projects. Of these, there are:
  • 8 evaluations of programs with grants and contributions (as per the FAA),
  • 11 evaluation commitments linked to TB submissions,
  • 13 evaluations of programs with both G&C and TB evaluation commitments, and
  • 7 risk-based evaluations ^Footnote11 (driven by neither a specific TB commitment nor a G&C).
• Significant changes have been made to the Evaluation Plan to address program requirements to accelerate the timelines for the renewal of the of Clean Air Agenda and Great Lakes program. Additional resources will be required from the programs affected.
• The proposed evaluation plan provides for an estimated coverage of 98% of direct program spending over the next five years.  As such, the plan does not fully meet the TB Evaluation Policy requirement of 100% coverage, but is however consistent with the coverage strategy presented and approved by the Departmental Evaluation Committee. The plan relies primarily on the use of PAA-level evaluations to maximize evaluation coverage by expanding the scope of planned evaluations to cover entire elements of the PAA. The use of the PAA-based evaluations to maximize coverage is an on-going effort and continues to be developed to maximize departmental coverage to the extent possible.  AEB’s strategy and approach to maximize evaluation coverage has recently been presented and endorsed at DEC.

The resources required to evaluate a program depend primarily on the program’s nature (e.g., complexity, "evaluability", horizontality), rather than its materiality. As such, there is a great deal of variability in annual average DPS coverage rates, despite similar annual resource investments.  Appendix B presents the estimated annual planned evaluation coverage for each of the plan’s 5 years, as well as estimated resource requirements in person days and dollars, and estimated G&C and DPS coverage of each evaluation.

Notwithstanding the coverage limitations presented above, the evaluation component of this RBAEP essentially meets the requirements of the recent TBS "Guide to Developing a Departmental Evaluation Plan" as outlined under previous paragraphs.  In addition, the AEB is providing separately to the TBS's Centre of Excellence for Evaluation, the requested evaluation templates or information specified in the Guide ^Footnote12

4.3 External Audit Liaison

The AEB is responsible for liaison with external assurance providers or auditors such as the Office of the Auditor General (OAG), the Commissioner of the Environment and Sustainable Development (CESD), the Public Service Commission (PSC) and others.  As such, the Branch coordinates any external audits or assessments conducted by these organizations, and fosters ongoing professional relationship, liaison and communication with them.  The coordination also includes the provision of advice and assistance to the DM, senior managers and EAAC throughout the external audit process.  The external audit liaison function also assists with coordinating and responding to environmental petitions received from the CESD.

A number of external audits and studies are underway or planned by external assurance providers. These external audits and studies were taken into account in the plan to ensure proper coverage and minimize possible duplication of efforts, based on external assurance organizations’ work plans provided to date.  The Audit and Evaluation Plan summary (section 4.6) includes known external audits and studies underway or to be carried out during the period covered by this plan. 

Also included in the plan is one horizontal audit of Information Technology (IT) Security to be performed in collaboration with the Office of the Comptroller General (OCG).  Under the Policy on Internal Audit, the OCG can direct departments to conduct sectorial and horizontal audits (i.e. government-wide), and can also perform department-specific audits.  The AEB has indicated to the OCG its preference to participate in the horizontal audit of IT security, planned for 2013-14, which would be combined with AEB's similar internal audit of the same topic.   

4.4 Branch Planning and Coordination

The AEB's Strategic Planning and Coordination (SPC) Directorate is responsible for a variety of functions and activities that contribute to the AEB's and the Department's priorities. SPC is primarily responsible to:

• Coordinate AEB’s risk-based audit and evaluation plans and annual report;
• Coordinate regular monitoring follow-up exercises on status of outstanding management actions in response to prior audit and evaluation report recommendations; 
• Maintain AEB’s overall quality assurance and project management frameworks;
• Provide support and coordination for the EAAC and the DEC;
• Coordinate Branch-wide management and administrative functions such as pertaining to procurement, financial and human resources; and
• Coordinate AEB's participation in EC corporate exercises (e.g. MAF, DPR/RPP, CRP).

4.5 Planned Resources

This RBAEP is fully costed and is based on the AEB's available capacity, professional staff and estimated financial resources for the upcoming year (2014-15), as further detailed under appendix C. The Branch resources are comprised of a total of 29 Full Time Equivalent Employees (FTE) and a total combined Salary and O&M budget allocated of $3,650K.

For multi-year planning purposes, the audit and evaluation plans are based on the assumption that financial and project resources available in future years (i.e. after 2014-15) would be similar to the resources planned for 2014-15.

Based on the planned resources presented herein, the AEB's management team confirms it will be able to deliver its 2014-15 audit plan and projects, and essentially all of its evaluation plan and projects, as reflected in this RBAEP document.  The qualification for the evaluation plan and projects is derived from the very recent revisions made to the evaluation plan to accommodate the advancement of the Government's schedule for the renewal of sunsetter programs. However, the AEB's shortfall was identified at DEC and is expected to be fully addressed through EC's first quarter financial review exercise.

Should the feasibility of delivering on the RBAEP be constrained by available resources or other factors, the Chief Audit and Evaluation Executive will bring this to the attention of the EAAC and/or DEC for consideration, and will recommend any necessary changes to the DM.

4.6 Integrated Plan Summary

The table on the subsequent pages summarizes in a comparative format all internal audit and evaluation projects planned over the next 3 years and 5 years respectively, and accordingly presents an integrated view of all the AEB planned coverage.  The table also includes known external audit engagements such as those pertaining to the CESD. The table is organized according to key MAF elements.

The table presents projects by year of their planned completion.  The presentation, therefore, is somewhat of an oversimplification, as projects may begin and end in different months or periods throughout the fiscal year. 

The external audits presented in the table are those known to the AEB with reasonable certainty at the time of the preparation of this RBAEP.  The document, therefore, may be subject to change and does not necessarily represent all the audits that will be undertaken by external assurance providers in the periods identified.  

The detailed audit and evaluation plans attached (appendices A and B) provide more comprehensive information on the planned projects.

Table 1 - Audit and Evaluation Plan Summary (Refer to legend on last page)

1. Policy And Programs

Strategic Outcome 1: Canada’s natural environment is conserved and restored from present and future generations

Outcome 2: Canadians are equipped to make informed decision on changing weather, water and climate conditions

Strategic Outcome 3: Threats to Canadians and their environment from pollution are minimized

2. Citizen-Focussed Service

3. Governance And Strategic Directions

4. Risk Management

5. Results And Performance

6. Stewardship

7. People

8. Public Service Values

9. Accountability

10. Learning, innovation and change management

Legend and Notes

Blue: Evaluations

Red: Internal Audits

Green:External Audits