Archived: Integrated Risk-Based Audit and Evaluation Plan 2014-2015: chapter 8
4. Integrated Plan
As reflected in the previous section, the proposed audit and evaluation plan strives to seek a balance between a number of different competing requirements and considerations, and does so in the context of the AEB's resources (sub-section 4.5 and appendix C.). In addition to the Policy requirements and the Departmental priorities outlined previously, the audit and evaluation plans cover the key aspects of the EC's programs and operations, as detailed in this section.
4.1 Internal Audit Plan
The AEB's Internal Audit Directorate (IA) provides assurance and advice to the DMs and senior management through objective and independent evidence-based information and assessment on the adequacy of management practices, controls and accountability mechanisms. Other activities can include providing management advisory services or conducting special assignments and investigations. IA also monitors the implementation of past audit recommendations and related management action plans to provide assurance that previous audit findings and issues are effectively addressed.
B. Plan Summary
The internal audit plan includes a total of 22 projects to be completed over the next three years, of which 10 projects were in progress at the start of 2014-15. The plan is based on available resources consistent with the approved Branch budget for 2014-15 (appendix C) and reflects a total staff complement of 9 Full Time Equivalents (FTEs), excluding 2 FTEs for external audit liaison, and relies partially on external professional services.
- Of the 22 projects planned or in progress‚ the majority are carried-forward from last year’s audit plan, while 3 projects were added and are aligned to key risks or senior management’s recommendations.
- 2 projects included in the plan are based in part on central agency guidance or requirements (i.e.‚ Fraud Risk Assessment; Governance Review). Another project is intended to be conducted in collaboration with the OCG as per its horizontal audit plan (IT Security).
The detailed internal audit plan (appendix A) briefly identifies all assurance and other engagements planned for the next three years, and provides initial project descriptions and rationale (i.e. link to departmental priorities, MAF and risks).
C. Internal Audit Core Areas
Internal Audit is intended to provide independent assessment of the effectiveness of the Department's risk management, control and governance processes (core internal audit areas). The AEB has considered in its multi-year audit plan the necessary coverage of these topics:
- All internal audit projects will consider key management and internal controls processes related to the specific areas they cover. Also MAF core controlsFootnote10 are always considered as part of the planning phase of each audit.
- An internal audit of risk management was completed in 2011-12. As well, most audit projects included in the plan cover risk management to some extent, while some more clearly focus on risk management, in particular the following: Fraud Risk Assessment, Information Technology Security, and Emergency Management and Planning.
- In regard to governance, the plan includes a Governance Framework Review (starting in 2015-16).
D. Departmental Risks, Priorities and MAF
The three-year internal audit plan takes into consideration alignment with key risks and considerations, departmental priorities and the key elements of the Government's Management Accountability Framework (MAF). Specifically, the detailed audit plan (appendix A) indicates, for each audit project, the link with the key risks identified through AEB's risk assessment exercise, the link to EC's key priorities and the link to key elements of the MAF.
4.2 Evaluation Plan
The Evaluation Directorate ensures that departmental senior management receives timely, strategically focused, objective and evidence-based information on the relevance and performance of EC programs and initiatives. Other activities include providing expert advice and services in the areas of evaluation and performance measurement, and reviewing TB submissions and Memoranda to Cabinet to ensure existing evaluation evidence is considered and future evaluation commitments identified.
B. Plan Summary
The detailed evaluation plan presented in appendix B includes a total of 39 funded evaluation projects over the next five years, and is based on an expected staff complement of 10 FTEs. The plan is for the most part consistent with the anticipated resources identified in the AEB initial budget for 2014-15 ( refer to appendix C). The evaluation plan was developed using the following general planning approach to identify projects to be evaluated during each of the next five years:
- The overriding planning priority has been to ensure the statutory requirements of the Financial Administration Act (FAA, section 42.1), and all evaluation commitments specified in TB submissions, are fulfilled. Accordingly, the plan provides for full (100%) coverage of all transfer payment programs (G&C) and existing TB evaluation commitmentswithin its 5 year planning period.
- The current plan includes a total of 39 distinct evaluation projects. Of these, there are:
- 8 evaluations of programs with grants and contributions (as per the FAA),
- 11 evaluation commitments linked to TB submissions,
- 13 evaluations of programs with both G&C and TB evaluation commitments, and
- 7 risk-based evaluations Footnote11 (driven by neither a specific TB commitment nor a G&C).
- Significant changes have been made to the Evaluation Plan to address program requirements to accelerate the timelines for the renewal of the of Clean Air Agenda and Great Lakes program. Additional resources will be required from the programs affected.
- The proposed evaluation plan provides for an estimated coverage of 98% of direct program spending over the next five years. As such, the plan does not fully meet the TB Evaluation Policy requirement of 100% coverage, but is however consistent with the coverage strategy presented and approved by the Departmental Evaluation Committee. The plan relies primarily on the use of PAA-level evaluations to maximize evaluation coverage by expanding the scope of planned evaluations to cover entire elements of the PAA. The use of the PAA-based evaluations to maximize coverage is an on-going effort and continues to be developed to maximize departmental coverage to the extent possible. AEB’s strategy and approach to maximize evaluation coverage has recently been presented and endorsed at DEC.
The resources required to evaluate a program depend primarily on the program’s nature (e.g., complexity, "evaluability", horizontality), rather than its materiality. As such, there is a great deal of variability in annual average DPS coverage rates, despite similar annual resource investments. Appendix B presents the estimated annual planned evaluation coverage for each of the plan’s 5 years, as well as estimated resource requirements in person days and dollars, and estimated G&C and DPS coverage of each evaluation.
Notwithstanding the coverage limitations presented above, the evaluation component of this RBAEP essentially meets the requirements of the recent TBS "Guide to Developing a Departmental Evaluation Plan" as outlined under previous paragraphs. In addition, the AEB is providing separately to the TBS's Centre of Excellence for Evaluation, the requested evaluation templates or information specified in the Guide Footnote12
4.3 External Audit Liaison
The AEB is responsible for liaison with external assurance providers or auditors such as the Office of the Auditor General (OAG), the Commissioner of the Environment and Sustainable Development (CESD), the Public Service Commission (PSC) and others. As such, the Branch coordinates any external audits or assessments conducted by these organizations, and fosters ongoing professional relationship, liaison and communication with them. The coordination also includes the provision of advice and assistance to the DM, senior managers and EAAC throughout the external audit process. The external audit liaison function also assists with coordinating and responding to environmental petitions received from the CESD.
A number of external audits and studies are underway or planned by external assurance providers. These external audits and studies were taken into account in the plan to ensure proper coverage and minimize possible duplication of efforts, based on external assurance organizations’ work plans provided to date. The Audit and Evaluation Plan summary (section 4.6) includes known external audits and studies underway or to be carried out during the period covered by this plan.
Also included in the plan is one horizontal audit of Information Technology (IT) Security to be performed in collaboration with the Office of the Comptroller General (OCG). Under the Policy on Internal Audit, the OCG can direct departments to conduct sectorial and horizontal audits (i.e. government-wide), and can also perform department-specific audits. The AEB has indicated to the OCG its preference to participate in the horizontal audit of IT security, planned for 2013-14, which would be combined with AEB's similar internal audit of the same topic.
4.4 Branch Planning and Coordination
The AEB's Strategic Planning and Coordination (SPC) Directorate is responsible for a variety of functions and activities that contribute to the AEB's and the Department's priorities. SPC is primarily responsible to:
- Coordinate AEB’s risk-based audit and evaluation plans and annual report;
- Coordinate regular monitoring follow-up exercises on status of outstanding management actions in response to prior audit and evaluation report recommendations;
- Maintain AEB’s overall quality assurance and project management frameworks;
- Provide support and coordination for the EAAC and the DEC;
- Coordinate Branch-wide management and administrative functions such as pertaining to procurement, financial and human resources; and
- Coordinate AEB's participation in EC corporate exercises (e.g. MAF, DPR/RPP, CRP).
4.5 Planned Resources
This RBAEP is fully costed and is based on the AEB's available capacity, professional staff and estimated financial resources for the upcoming year (2014-15), as further detailed under appendix C. The Branch resources are comprised of a total of 29 Full Time Equivalent Employees (FTE) and a total combined Salary and O&M budget allocated of $3,650K.
For multi-year planning purposes, the audit and evaluation plans are based on the assumption that financial and project resources available in future years (i.e. after 2014-15) would be similar to the resources planned for 2014-15.
Based on the planned resources presented herein, the AEB's management team confirms it will be able to deliver its 2014-15 audit plan and projects, and essentially all of its evaluation plan and projects, as reflected in this RBAEP document. The qualification for the evaluation plan and projects is derived from the very recent revisions made to the evaluation plan to accommodate the advancement of the Government's schedule for the renewal of sunsetter programs. However, the AEB's shortfall was identified at DEC and is expected to be fully addressed through EC's first quarter financial review exercise.
Should the feasibility of delivering on the RBAEP be constrained by available resources or other factors, the Chief Audit and Evaluation Executive will bring this to the attention of the EAAC and/or DEC for consideration, and will recommend any necessary changes to the DM.
4.6 Integrated Plan Summary
The table on the subsequent pages summarizes in a comparative format all internal audit and evaluation projects planned over the next 3 years and 5 years respectively, and accordingly presents an integrated view of all the AEB planned coverage. The table also includes known external audit engagements such as those pertaining to the CESD. The table is organized according to key MAF elements.
The table presents projects by year of their planned completion. The presentation, therefore, is somewhat of an oversimplification, as projects may begin and end in different months or periods throughout the fiscal year.
The external audits presented in the table are those known to the AEB with reasonable certainty at the time of the preparation of this RBAEP. The document, therefore, may be subject to change and does not necessarily represent all the audits that will be undertaken by external assurance providers in the periods identified.
The detailed audit and evaluation plans attached (appendices A and B) provide more comprehensive information on the planned projects.
Table 1 - Audit and Evaluation Plan Summary (Refer to legend on last page)
1. Policy And Programs
Strategic Outcome 1: Canada’s natural environment is conserved and restored from present and future generations
Evaluation of Invasive Alien Species Strategy for CanadaFootnotea
Evaluation of Water Resource Management and Use ProgramFootnotea
Evaluation of Environmental Youth Employment ProgramFootnotea
Evaluation of Great Lakes Action Plan
Evaluation of Habitat Conservation Partnerships Program
Evaluation of Ecosystem Assessment and Approaches
Evaluation of Water Quality and Aquatic Ecosystems Health ProgramFootnotea
Evaluation of Lake Simcoe Ecosystem Initiative
Evaluation of Sustainability and Reporting Indicators Program
Evaluation of Species at Risk Program
Evaluation of Migratory Birds Program
Evaluation of Lake Winnipeg Ecosystem Initiative
Evaluation of St. Lawrence Ecosystem Initiative
|Evaluation of Biodiversity Policy and Priorities Program||
Evaluation of Protected Areas
Evaluation of Hydrological Service and Water Survey
Evaluation of EcoAction Community Funding Program
Evaluation of Environmental Damages Fund Program
Evaluation of Community Ecosystem Partnerships Program
Evaluation of Compliance Promotion and Enforcement - Wildlife
Outcome 2: Canadians are equipped to make informed decision on changing weather, water and climate conditions
|CESD Audit of Environmental Impacts of Marine Transportation in Canada’s ArticFootnotea||
Evaluation of Weather Observations, Forecasts and Warnings Program
Evaluation of Federal Services to the Toronto 2015 Pan American and Parapan American Games
Evaluation of Clean Air Adaptation
|Evaluation of Service to Marine Transportation||Evaluation of Weather Observations, Forecasts and Warnings Program||Evaluation of Radarsat Constellation Mission|
Strategic Outcome 3: Threats to Canadians and their environment from pollution are minimized
Evaluation of Environmental Technology ProgramFootnotea
Evaluation of EC Umbrella Terms and ConditionsFootnotea
CESD Audit on Climate Change MitigationFootnotea
CESD Audit on Oil Sands Environmental MonitoringFootnotea
Internal Audit of World Class Regulator*
Evaluation of Chemicals Management Plan*
Evaluation of Climate Change and Clean Air Regulatory Program (CARA III)Footnotea
Evaluation of Other Clean Air Transportation
Evaluation of International Climate Change and Clean Air Partnerships Program
Evaluation of Genomic R&D
CESD Follow up Audit on Enforcement of CEPA 1999
CESD Audit on Transboundary Waste
|Evaluation of Compliance Promotion and Enforcement - Pollution Program||Evaluation of Effluent Management
Evaluation of Environmental Emergencies
Evaluation of Grant to Sustainable Development Technology Canada (SDTC
Evaluation of Complementary Measures (World Class Regime)
|Evaluation of Contaminated Sites Program|
2. Citizen-Focussed Service
|No project identified.||-||-||-||-|
3. Governance And Strategic Directions
Internal Audit Review of Governance Framework
Internal Audit of Integrated Planning/Operational Planning
|Internal Audit of the Implementation of S&T Strategy and Plans||-|
4. Risk Management
|No project identified.||-||-||-||-|
5. Results And Performance
|Internal Audit of External Reporting on PerformanceFootnotea||CESD Audit on SDS Assessing Progress||-||-||-|
Internal Audit of Capital Asset ManagementFootnotea
Internal Audit of IT Service DeliveryFootnotea
Internal Audit of Management of LaboratoriesFootnotea
Internal Review and Benchmarking of Privacy ProcessesFootnotea
Internal Audit of Information Technology Security
Internal Audit of Systems, Applications and Products (SAP) ImplementationFootnotea
Internal Audit of Management and Delivery of Procurement
Internal Audit of Vote Netted Revenue (VNR)
Internal Audit of Emergency Management and Planning
Internal Audit of Expenditure Management Framework
Internal Audit of Records Management including Implementation of GCDocs
Internal Audit of Pay and Benefits
Internal Audit of Real Property and Accommodation
|-||Internal Audit of Staffing and Classification||Internal Audit of HR Planning and Management||-||-|
8. Public Service Values
|Internal Fraud Risk Assessment and ReviewFootnotea||-||-||-||-|
|-||Internal Audit of Management and Compliance of International Environmental Agreements||-||-|
10. Learning, innovation and change management
|No project identified.||-||-||-||-|
Legend and Notes
Red: Internal Audits
Report a problem or mistake on this page
- Date modified: