Archived: Integrated Risk-Based Audit and Evaluation Plan 2015-2016: chapter 5
4. Integrated Plan
As reflected in the previous section, the proposed audit and evaluation plan strives to seek a balance between a number of different competing requirements and considerations, and does so in the context of AEB's resources (sub-section 4.5 and appendix C.). In addition to the requirements and priorities outlined previously, the audit and evaluation plans cover important aspects of EC's programs and operations, as detailed in this section.
4.1 Internal Audit Plan
AEB's Internal Audit Directorate provides assurance and advice to the DMs and senior management, through objective and independent evidence-based information and assessment of management practices, controls and accountability mechanisms. Other activities can include providing management advisory services or conducting special assignments and investigations.
B. Plan Summary
The internal audit plan includes a total of 24 projects to be completed over the next three years, of which 8 projects were in progress at the start of 2015-16. The plan is based on expected available resources consistent with the Branch’s planned budget for 2015-16 (appendix C) and reflects a total staff complement of 9 Full Time Equivalents (FTEs; excluding 2 FTEs assigned to external audit liaison), and relies partially on external professional services. Highlights include:
- Of the 24 projects planned or in progress, the majority are carried-forward from last year's audit plan, while 5 projects were added and are aligned to key risks or senior management's recommendations;
- Two projects are based in part on central agency guidance or requirements (i.e., Fraud Risk Assessment and Management Review underway; Governance Framework Review planned). One project is currently being conducted in collaboration with the OCG (IT Security) and another project is planned to begin in 2015-16 (Emergency and Business Continuity Planning);
- Discussions are underway to collaborate with the Public Service Commission (PSC) to conduct a joint audit of Staffing and Classification in 2015-16. This innovative collaborative initiative would allow PSC to follow-up on its previous audit as part of a broader AEB assignment, and also serve as a developmental exchange opportunity for both organizations.
The detailed internal audit plan (appendix A) briefly identifies all assurance and other engagements planned for the next three years, and provides initial project descriptions and rationales (i.e. link to departmental priorities, MAF and risks).
C. Internal Audit Core Areas
Internal Audit is intended to provide an independent assessment of the effectiveness of the Department's risk management, control and governance processes (core internal audit areas). AEB has considered in its multi-year plan the necessary coverage of these topics:
- All internal audit projects, as part of their planning phase, will consider key management and internal control processes related to the specific areas they cover, including MAF core controlsFootnote 9 where applicable;
- An internal audit of risk management was completed in 2011-12. As well, audit projects included in the plan generally take into consideration risk management to some extent, while some more clearly focus on key aspects of risk management (e.g. Fraud Risk Assessment and Management Review, Information Technology Security, and Emergency and Business Continuity Planning);
- The plan includes a Governance Framework Review (starting in 2015-16).
D. Departmental Risks, Priorities and MAF
The three-year internal audit plan takes into consideration the alignment with key risks and considerations, departmental priorities and the key elements of the Government's Management Accountability Framework (MAF). Specifically, the detailed audit plan (appendix A) indicates, for each audit project, the link with the key risks identified through AEB's risk assessment exercise, the link to EC's key priorities and the link to key elements of the MAF.
4.2 Evaluation Plan
The Evaluation Directorate ensures that departmental senior management receives timely, strategically focused, objective and evidence-based information on the relevance and performance of EC programs and initiatives. Other activities include providing expert advice and services in the areas of evaluation and performance measurement, and reviewing TB submissions and Memoranda to Cabinet to ensure existing evaluation evidence is considered and future evaluation commitments identified.
B. Plan Summary
The detailed evaluation plan presented in appendix B includes a total of 44 funded evaluation projects over the next five years, and is based on an expected staff complement of 9 FTEs. The plan is consistent with the anticipated resources identified in AEB’s planned budget for 2015-16 (appendix C). The evaluation plan was developed using the following general planning approach to identify projects to be evaluated during each of the next five years:
- The overriding planning priority has been to ensure the statutory requirements of the Financial Administration Act (FAA, section 42.1), and all evaluation commitments specified in TB submissions, are fulfilled. Accordingly, the plan provides for full (100%) coverage of all transfer payment programs (G&C) and existing TB evaluation commitments within its 5 year planning period.
- The current plan includes a total of 44 distinct evaluation projects. Of these, there are:
- 8 evaluations of programs with grants and contributions (as per the FAA),
- 13 evaluation commitments linked to TB submissions,
- 16 evaluations of programs with both G&C and TB evaluation commitments, and
- 7 risk-based evaluationsFootnote 10 (not driven by a specific TB commitment or a G&C).
- The proposed evaluation plan provides for an estimated 98.5% coverage of DPS over the next five years. As such, the plan does not fully meet the TB Evaluation Policy requirement of 100% coverage, but is consistent with the coverage strategy previously approved by the DEC. The plan relies on PAA-level evaluations to help maximize coverage by expanding the scope of evaluations to cover entire PAA elements. The use of the PAA-level evaluations is an on-going effort, and continues to be developed to maximize departmental coverage to the extent feasible.
The resources required to evaluate a program depend primarily on the program’s nature (e.g., complexity, "evaluability", horizontality), rather than its materiality. As such, there is a great deal of variability in annual average DPS coverage, despite similar annual resource investments. Appendix B presents the estimated annual planned evaluation coverage for each of the plan’s 5 years, as well as estimated resource requirements in person days and dollars, and estimated G&C and DPS coverage of each evaluation.
Notwithstanding the limitations presented above, the evaluation component of this RBAEP essentially meets the requirements of the recent TBS "Guide to Developing a Departmental Evaluation Plan" as presented previously. In addition, AEB is providing separately to TBS's Centre of Excellence for Evaluation, the Guide’s requested evaluation templates or informationFootnote 11
4.3 External Audit Liaison
AEB is responsible for liaison with external assurance providers or auditors such as the Office of the Auditor General (OAG), the Commissioner of the Environment and Sustainable Development (CESD), the Public Service Commission (PSC), and others. As such, the Branch coordinates any external audits or assessments by these organizations, and fosters ongoing professional relationship, liaison and communication. The coordination also includes providing advice and assistance to the DMs and senior managers throughout the external audit process. The external audit liaison function also assists with coordinating and responding to environmental petitions received from the CESD.
A number of external audits and studies are underway or planned by external assurance providers. These external audits and studies were considered to ensure proper coverage and minimize possible duplication of efforts, based on external assurance organizations’ work plans available to date. The integrated plan summary (section 4.6) includes known external audits and studies to be carried out during the period covered by this plan.
The plan also includes two horizontal audits performed in collaboration with the OCG (Information Technology (IT) Security; Emergency and Business Continuity Planning). Under the Policy on Internal Audit, the OCG can direct departments to conduct sectorial and horizontal audits, and can perform department-specific audits. AEB has indicated its preference to participate in the OCG’s audit of IT Security currently being completed, as well as participate in the OCG audit of Emergency and Business Continuity Planning, starting in 2015-16.
4.4 Branch Planning and Coordination
AEB's Strategic Planning and Coordination (SPC) Directorate is responsible for a variety of functions and activities that contribute to AEB's priorities, including:
- Coordinate AEB’s risk-based audit and evaluation plans and annual report;
- Coordinate regular monitoring exercises (follow-ups) on the status of outstanding management actions resulting from prior audit and evaluation report recommendations;
- Maintain AEB’s overall quality assurance and project management frameworks;
- Provide support and coordination for the EAAC and the DEC;
- Coordinate Branch management and administrative functions such as pertaining to procurement, financial and human resources; and
- Coordinate AEB's participation in EC’s planning and reporting exercises.
4.5 Planned Resources
This RBAEP is fully costed and is based on AEB's available capacity, professional staff and planned financial resources for 2015-16. The Branch resources are comprised of an estimated total of 28 FTEs and a total combined Salary and Operations and Maintenance planned budget of $3,328K (appendix C). For multi-year planning purposes, the audit and evaluation plans are based on the assumption that financial and project resources available in future years (i.e. after 2015-16) would be similar to the resources planned for 2015-16.
Based on the planned resources presented herein, AEB's management team expects to be able to deliver its 2015-16 audit and evaluation plans and projects, as reflected in this RBAEP document. Should the feasibility of delivering on the RBAEP be constrained by available resources or other factors, the Chief Audit and Evaluation Executive will bring this to the attention of the EAAC and/or DEC for consideration, and will recommend any necessary changes to the DM.
4.6 Integrated Plan Summary
The table on the subsequent pages summarizes in a comparative format all internal audit and evaluation projects planned over the next 3 years and 5 years respectively, and presents an integrated view of all planned projects. The summary also includes known external audits such as those pertaining to the CESD.
The table is organized according to key MAF elements, and presents projects by year of their planned completion. This presentation is an oversimplification, as projects may begin and end in different periods and fiscal years. The external audits presented in the table are those known with reasonable certainty at the time of the preparation of this RBAEP, and therefore does not necessarily represent all the audits that will be undertaken by external assurance providers. The detailed audit and evaluation plans attached (appendices A and B) provide more comprehensive information on the planned projects.
The following table summarizes planned projects according to MAF elements and presents projects by year of their planned completion.
Table 1 - Audit and Evaluation Plan Summary
(Refer to legend on last page)
|Evaluation of Habitat Conservation Partnerships Program*
Evaluation of Water Quality and Aquatic Ecosystems Health*
Evaluation of Sustainability and Reporting Indicators Program*
Evaluation of Ecosystem and Environmental Assessment*
Evaluation of Great Lakes*
Evaluation of Lake Simcoe Ecosystem Initiative
OAG Audit of Labrador Inuit Land Claims Agreement
|Evaluation of Species at Risk Program
Evaluation of Migratory Birds Program
Evaluation of St. Lawrence Ecosystem Initiative
Evaluation of Lake Winnipeg Ecosystem Initiative
Evaluation of Compliance Promotion and Enforcement - Wildlife
|Evaluation of Biodiversity Policy and Priorities Program
Evaluation of the Major Projects Management Office Initiative
|Evaluation of the National Wetlands Conservation Fund
Evaluation of Protected Areas
Evaluation of the World-Class Tanker Safety System South of the 60 North Latitude
Evaluation of Hydrological Service and Water Survey
Evaluation of Eyford Response
Evaluation of EcoAction Community Funding Program
Evaluation of Environmental Damages Fund Program
|Evaluation of Water Resource and Management Use Program
Evaluation of Environmental Youth Employment Program
Evaluation of the Ecosystem Partnerships Program
|Evaluation of Weather Observations, Forecasts and Warnings Program*
Evaluation of Federal Services to the Toronto 2015 Pan American and Parapan American Games*
Evaluation of Clean Air Adaptation*
|Evaluation of Meteorological and ice services in support of marine navigation
CESD Audit on Science to Mitigate Effects of Severe Weather
|Evaluation of Weather Warning and Forecast System
Evaluation of Arctic Meteorological and Navigational Warning Services (METAREAS)
|Evaluation of Radarsat Constellation Mission|
|Evaluation of Chemicals Management Plan*
Evaluation of Climate Change and Clean Air Regulatory (CARA IV)*
Evaluation of International Climate Change and Clean Air Partnerships*
Evaluation of EC Umbrella Terms* and Conditions*
Internal Audit of the Regulatory Process (World Class Regulator)*
|Evaluation of Genomic R&D
CESD Audit on Federal Support for Sustainable Communities
|Evaluation of Effluent Management
Evaluation of Environmental Emergencies
Evaluation of Grant to Sustainable Development Technology Canada (SDTC)
Evaluation of Complementary Measures (World Class Regime)
|Evaluation of Contaminated Sites Program
Evaluation of Compliance Promotion and Enforcement - Pollution Program
|Evaluation of Chemicals Management Plan
Evaluation of Other Clean Air Transportation
Evaluation of Environmental Technology Program
|No project identified.||-||-||-||-|
|Internal Review and Benchmarking of Privacy Processes*
Internal Audit of Information Technology Security*
|Internal Audit Review of Governance Framework||Internal Audit of Integrated Planning/Operational Planning
Internal Audit of Implementation of S&T Strategy and Plans
|Internal Audit of Regional Roles and Responsibilities, and Management
Internal Audit of Communications
|-||-||Internal Audit of Information Management||-||-|
|System Under Development of SAP*
Internal Audit of Management and Delivery of Procurement*
Internal Audit of Vote Netted Revenue (VNR)*
|Internal Audit of Expenditure Management and Controls
Internal Audit of Emergency and Business Continuity Planning
|Internal Audit of Financial Management Framework||Internal Audit of Real Property and Accommodation
Inventory Management and Control
|No project identified.||-||-||-||-|
|CESD Audit on Departmental Progress in implementing SDSs
Internal Audit of External Reporting on Performance*
|Internal Audit of Aboriginal Land Claim Agreements
Internal Audit of International Environmental Agreements
|Internal Audit of Staffing and Classification (PSC Collaboration)||Internal Audit of Pay and Benefits||Internal Audit of HR Planning and Management||-|
|Internal Fraud Risk Assessment and Review*||-||-||-||-|
|No project identified.||-||-||-||-|
Legend and Notes
Red: Internal Audits
Green: External Audits
*: Projects in progress (started prior to 2015-16)
- Date modified: