Audit of the Process to Support Decision Making at Strategic Management Committees

May 2016

7050-8-38 (ADM(RS))

Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

Acronyms and Abbreviations

Results in Brief

Assistant Deputy Minister (Review Services) (ADM(RS)) conducted an audit of the process to support decision making at strategic management committees with a focus on the effectiveness of governance, internal controls, and risk management. This audit was included in the departmental Risk-Based Internal Audit Plan for fiscal years 2013/14 to 2015/16. To be effective and efficient, strategic management committees rely on having accurate and relevant information presented in a clear and timely manner. An effective information management process is therefore fundamental to the success of these committees.

The focus of the audit included the review of the process to support decision making for the Department's six key strategic management committees. Specifically, the audit looked at the process for preparation, review, and validation of information from initial preparation to presentation at a committee. Having adequate controls and processes increases the likelihood of consistently providing accurate, complete, and timely information to the committees. At the time of this audit, concurrent third-party governance reviews were taking place. The audit team ensured that there was limited overlap between the scope of this audit and the ongoing reviews. In addition, the governance structure in the Department of National Defence (DND) is being reviewed and updated through the Defence Renewal Initiative 7.2.

Findings and Recommendations

Governance. Each examined committee has formally documented terms of reference. These terms of reference contain important information for committee operations, as well as guidance for the preparation, review, and validation of information that is presented at the committees and used for decision making. Two of the sampled committees' terms of reference did not include all of the key guidance that can be found in the terms of reference for the other committees, such as roles and responsibilities and various committee processes.

Additionally, terms of reference generally include the deadline for information to be submitted to the committee secretary prior to each committee meeting. Committee secretaries indicated that this deadline was sometimes missed, which would not allow sufficient time for committee secretary analysts to properly review and validate the information or for committee members to review the information sufficiently in advance of the meetings. This increases the risk that decisions may be made based on inaccurate or incomplete information.

Committee meetings were regularly conducted for each of the committees during the period examined. Attendance requirements for these committee meetings are documented and available in the terms of reference for each of the committees. Acceptable proxies are also documented for all but one of the committees examined as part of the audit.

However, in multiple instances, the replacements attending meetings for members were not at the pre-approved positional level. Although committee secretary analysts validate and challenge information during their review, committee members or acceptable proxies must also perform the challenge function at a more strategic level. Therefore, without the expected office holders in attendance, there is a risk that the information presented at these meetings may be challenged with insufficient rigour or by individuals who are not at the pre-approved level.

ADM(RS) recommends that the committees' terms of reference be updated to consistently include key guidance requirements for the preparation, review, and validation of information, including acceptable replacements should the primary office holders need to be absent. In addition, the terms of reference requirements should be consistently adhered to, specifically as they relate to information submission deadlines and attendance at committee meetings.

Internal Controls. Each committee secretary employs analysts to review presentations and related information. The analysts assess these using a template that has been developed for their respective committee. However, these templates do not include sufficient instructions on how they should be completed or the level of analysis or challenge that is required. Additionally, while some important direction and guidance were available to information preparers, reviewers, and presenters, these elements were not sufficiently detailed or consolidated for easy reference.

ADM(RS) recommends that a guidance document with more detailed criteria for review and validation of information be completed by each of the committee secretaries. Additionally, it is recommended that a consolidated guidance document be created to act as standard operating procedures for the preparation and presentation of information to all committees.

Risk Management. The reliability of information refers to the degree to which the committee can rely on the information being presented. Guidance for presenting information reliability at committee meetings was not in the terms of reference of the committees or in any other guidance document examined. As such, the reliability of the information was inconsistently presented to committees, and in some instances, not at all.

ADM(RS) recommends that a standard approach be developed for assessing and communicating the reliability of information that is presented to the committees.

Note: Please refer to Annex A—Management Action Plan for the management response to the ADM(RS) recommendations.

1.0 Introduction

1.1 Background

1.1.1 DND Strategic Management Committees

There are six strategic management committees that make key decisions within the Department. They are identified in Table 1 and formed the basis of the audit. These committees were originally identified by the Deputy Minister (DM) and the Chief of the Defence Staff (CDS) at a Defence Strategic Executive Committee meeting in July 2009. In 2013, additional strategic management committees, the Defence Executive Policy Committee (Policy Committee) and the Investment and Resource Management Committee (IRMC) were each created, with the latter replacing the Defence Finance Committee. The strategic management committees are listed in Table 1 along with a description of their roles. Although not specified in the terms of reference of every committee, it was observed that committees were generally meeting on a monthly basis, missing a month only on occasion.

Stakeholders have notified ADM(RS) that significant changes have been made to some committees' governance structures as a result of the recent governance reviews conducted by third parties. This will be discussed further in Section 1.4 of this report.

Table 1. Primary Role of the Strategic Management Committees. This table describes the primary role of each of the six main strategic management committees.

Some examples of decisions made at these committees include the following:

• approval of the Corporate Risk Profile at DSX;
• management of the Investment Plan at DSX and IRMC;
• approval of financial policy decisions at the Policy Committee;
• endorsement of Defence priorities document¹ at DMC;
• review of trimestral financial and in-year resource allocation decisions at IRMC;
• review of the affordability of force development initiatives at PMB; and
• approval of investment priorities at DCB.

1.1.2 Key Stakeholders

The Corporate Secretary (Corp Sec) is responsible for, among other things, the coordination of Defence governance executive meeting planning. Reporting to the Corp Sec are both the National Defence Headquarters Secretariat (NDHQ Sec) and the Defence Governance Directorate.

The core of the NDHQ Sec business is to manage the strategic committees chaired by the DM, CDS, VCDS, or the CFO. Committees administered and fully supported by NDHQ Sec include the Policy Committee, DSX, and DMC. Support activities include preparing agendas, scheduling, and recording and tracking minutes and decisions. IRMC, PMB, and DCB have their own secretaries; however, they receive services from the NDHQ Sec in accordance with negotiated service level agreements.² As such, NDHQ Sec plays a significant role in supporting governance within the Department.

The Defence Governance Directorate develops, administers, and maintains the Defence governance framework to enable decision making by the Senior Executive Team. This directorate is also responsible for the strategic analysis of items raised at the strategic management committees and boards.³ Lastly, it has been tasked with reviewing and updating the governance structure in the Department through the Defence Renewal Initiative 7.2. The purpose of this initiative is to review how the Department is governed in order to better align resources and efforts to enable better decision making. Three elements that this initiative will examine are departmental accountabilities and authorities, the structure of the governance committees, and the tools and standard operating procedures in support of the governance committees.

As executive secretaries for IRMC, PMB, and DCB respectively, Director Budget, Chief of Programme, and Chief of Force Development provide many of the same support services as those provided by the NDHQ Sec.

1.1.3 Strategic Management Committee Meeting Process

A general description of the process for a strategic management committee meeting is described in Figure 1. This Figure starts at the beginning of the process before a meeting takes place and describes the major steps up to the official conclusion of a meeting.

The typical flow of information to the committees follows the process outlined in Figure 1.

Figure 1. Flow of Information. This chart illustrates the flow of information that leads to decision making.

1.2 Rationale for Audit

Having complete, reliable, and timely information plays a critical role in the decision-making process by enabling decision makers to make better informed decisions. Key strategic management committees require accurate and timely information to support decision making. If this information is presented in a succinct and uniform format, it increases the efficiency and effectiveness of the committee meetings. Given the importance of the decisions made by the Department's strategic management committees, the process to support decision making at these committees was identified as a high audit priority. Therefore, this audit was included in the Risk-Based Internal Audit Plan for fiscal years 2013/14 to 2015/16.

1.3 Objective

The objective of the audit was to determine whether appropriate control processes, governance, and risk management practices are in place to promote the preparation and presentation of complete, accurate, and timely information to Defence strategic management committees to support decision making.

1.4 Scope

The scope of this audit included a review of the process for presenting information at the Department's six strategic management committees. These committees included DSX, Policy Committee, DMC, IRMC, PMB, and DCB.⁴ A sample of decisions was analyzed from the six strategic management committees within the timeframe of May 2013 and July 2014.

The audit team was aware of third-party governance reviews that were ongoing at the time of the audit, and it ensured that there was limited overlap with the scope of the audit and that the messaging for similar findings between the audit and the reviews was consistent. These reviews looked at identifying opportunities for improving effectiveness of decision making with available resources, ensuring committees supported the DM's responsibilities, harmonizing the roles amongst governance committees, and identifying areas where the challenge function could be enhanced.

The scope of this audit did not include testing of the accuracy of information presented to the committees or the timeliness and clarity of decisions taken, but rather the process in place to ensure the quality of this information. Additionally, the audit did not assess the interrelationships of the committees or the appropriateness of committee structures. Testing the completeness and accuracy would entail a separate audit and will be considered as part of future risk-based audit planning.

1.5 Methodology

The Audit of the Process to Support Decision Making at Strategic Management Committees focused on the process to deliver information to the key strategic management committees, including the preparation, review, validation, and presentation of the information. The reason this approach was chosen is that a framework that includes appropriate controls and processes would increase the likelihood of consistently providing accurate, complete, and timely information to the key strategic management committees.

The audit approach included the following:

• interviews with key stakeholders involved in various aspects of strategic management committee meetings, including preparers, reviewers, and presenters of information;
• review of documents such as committee terms of reference and relevant directives and review of documents posted to the strategic calendar located on the NDHQ Executive Meeting Coordination (NEMC) SharePoint⁵ site and documents received from those interviewed; and
• walkthroughs of key information preparation processes using a sample of seven committee decisions. Committee decisions were chosen based on a review of committee presentations and their supporting documents.

1.6 Audit Criteria

The audit criteria can be found at Annex B.

1.7 Statement of Conformance

The audit findings and conclusions contained in this report are based on sufficient and appropriate audit evidence gathered in accordance with procedures that meet the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing. The audit thus conforms to the Internal Auditing Standards for the Government of Canada as supported by the results of the quality assurance and improvement program. The opinions expressed in this report are based on conditions as they existed at the time of the audit and apply only to the entity examined.

2.0 Findings and Recommendations

2.1 Governance

The committees' terms of reference provide some guidance and expectations. However, they are not consistently documented or applied to promote the timely submission and comprehensive validation of information.

2.1.1 Terms of Reference for Committees

Each of the strategic management committees has produced terms of reference to facilitate the administration and operation of their respective committee. The terms of reference for two of the committees did not contain the same level of guidance as the terms of reference of the other committees examined. The terms of reference for DSX, DMC, PMB, and DCB generally contained a similar level of detail and instruction. The terms of reference for the IRMC and the Policy Committee differed in that they did not contain the following:

• responsibilities broken down by position, such as the committee chair, executive secretary, and committee members;
• information for making an application to present to the committee (e.g., where to apply and what material to include with the application);
• information for presenting/briefing to the committee;
• information for managing committee operations (e.g., information on NEMC SharePoint and standard committee agendas);
• information on the expected level of designated representatives (missing for Policy Committee only); and
• information on meeting frequency (missing for IRMC only).

Due to urgent requirements, the Policy Committee and IRMC were not formed in the same fashion as most committees. This may have affected the development of comprehensive terms of reference. As a result, there is a risk that information presented at these two committees may not meet the committee requirements, which could impact the completeness and quality of information being presented for decision-making purposes.

2.1.2 Analytical Support

For each committee presentation, the required information is prepared within the responsible organization and then submitted to the committee secretary analysts to be validated and summarized. This step allows for additional review of the information and provides additional confidence in the accuracy and completeness of the information. Upon completion, the presentation, summary, and any accompanying information are distributed to committee members over the NEMC SharePoint. This process is generally explained in the terms of reference for each committee. In addition, most terms of reference include a deadline for submitting the presentation and related information to the respective committee secretary analysts for review. However, interviews with committee secretaries and their analysts indicated that information and presentations are often not received by the required deadline. This results in less time to complete a comprehensive review of the information before submitting it to the committee secretary and committee members. If organizations submit their information late, and the information has not been appropriately reviewed and validated, there is an increased risk that committee decisions will be based on incomplete or inaccurate information.

2.1.3 Challenge of Information

While committee secretary analysts review and challenge information, in order to have effective decision making, committee members must also perform a challenge function at a more strategic level. The members who are required to attend committee meetings are listed in the terms of reference for each committee while suitable proxies are identified in the terms of reference for all but one of the committees. Although attendance requirements are documented, they are not being adhered to consistently.

Attendance at six of the Department's key strategic management committees' meetings was assessed and compared to the requirements outlined in the respective committees' terms of reference. Meetings were held regularly for each committee and were generally well attended. However, there were 19 specific instances during the 36 meetings analyzed where a person in attendance did not meet the positional level specified in the terms of reference.⁶ This is an issue regarding the seniority level of the representatives who may attend a committee meeting on behalf of a designated office holder. One of the committees' terms of reference permits representatives; however, it does not specify a minimum required level for the representative. Without the expected office holders at the meetings, there is a risk that information may not be challenged with sufficient rigour or by someone at a level required by the terms of reference.

2.1.4 Summary

Direction and guidance is documented within each committee's terms of reference, including the deadline for the submission of information to the committee, the persons required to attend committee meetings, and in most instances, the approved replacements when an expected office holder cannot attend the meetings. Although the terms of reference for four of the committees were generally consistent in documented direction and guidance, the terms of reference for two of the committees could include additional guidance to improve the effectiveness of preparers and presenters prior to and during committee meetings. Additionally, although these requirements were documented, they were not being consistently followed. When information is submitted late, committee secretary analysts may not have sufficient time to complete a comprehensive review.

Finally, although the attendance expectations and suitable replacements are generally documented in the committees' terms of reference, these requirements were not consistently followed. This could lead to information that is challenged without sufficient rigour or by someone not at the pre-approved level.

2.2 Internal Controls

While there is some guidance and documentation of procedures for preparers, presenters, and reviewers of information, it is not sufficiently documented or communicated to promote complete, accurate, and timely information for decision making.

2.2.1 Validation of Information by Committee Secretary Analysts

Before a presentation reaches a committee and its members, it undergoes multiple levels of review. Presentations are first reviewed within the presenter's own organization. Information is then submitted to the committee where it is assigned to a committee secretary analyst for review and validation. It is the role of these committee secretary analysts to verify the information for accuracy and completeness before summarizing it for the committee members. This provides additional confidence to the committee members that information being presented is accurate.

To assist with their review and validation and with the presentation of their results, analysts are provided a blank template to complete with information they have summarized from both the presentation and supporting documentation. Templates vary from committee to committee; however, there are some common required categories, including the following:

• decision requested;
• key issues and considerations; and
• strategic alignment or link to corporate risks and priorities.

Although the templates provide the headings for these categories and a space to enter comments, analysts are not provided with any further details or instructions on what analysis, review, and challenge is expected and required in these sections. This can make it difficult for analysts to determine how much rigour should be put into the challenge function. For example, it may be difficult to determine when they are required to consult subject matter experts or perform a more in-depth analysis in a certain area. In addition, the lack of guidance does not promote a uniform level of analysis, which could lead to lengthier than required, or less than adequate, review and validation. For example, the review template of one committee has a section titled "Issues," but there is no guidance as to the type of items to include in it. This could lead to different interpretations from different analysts and therefore different advice provided to the committee members. Documented guidance could also prove useful for new employees who are required to complete these reviews. Ultimately, insufficient guidance on the level of analysis can lead to an inadequate level of review and validation and to committee decisions being made without the necessary information.

2.2.2 Documentation of Standard Operating Procedures for Preparers and Presenters of Information

Documented guidance is available to assist preparers and presenters with the development of presentations for the committees; although, to access all of this information requires preparers and presenters to consult multiple sources. These sources include the committee terms of reference, the VCDS Directive on Governance Boards and Committees, the Directive for Distribution of Information in Support of National Defence Headquarters L0/L1 Executive Meetings, and the Direction – Governance Meeting Briefing Guidelines, among others. Since these directives were never consolidated in one all-encompassing document as standard operating procedures, it can be difficult for inexperienced preparers and presenters to know where to locate the various pieces of information and make appropriate use of them.

Each of these directives contains different guidance for preparers and presenters of information, such as the expected length of presentations and an acceptable number of slides to include. However, the guidance from these documents does not include key elements, such as roles and responsibilities, specific content to include in presentations and level of analysis, and accuracy required of information to be presented. This issue was confirmed by some preparers and presenters during interviews. Without adequate guidance for the preparers and presenters, the information that the committees receive may not be what is required for effective decision making or to make best use of the committees' time.

2.2.3 Tracking of Decisions and Action Items

For each of the six strategic management committees, a sample of between five and seven open action items were examined in order to determine if a process existed to track and follow up on actions items in a timely manner. It was determined that there were processes in place to track open action items for each committee. This process and the associated tools varied depending on the executive secretary of the committee and the person responsible for tracking the items.

The majority of the committee action items were actioned and tracked. However, for 5 out of 36 action items sampled, there was no documentation that any actions had been taken. Action items could have been discussed and decisions made outside the committee meeting without notifying the person responsible for tracking the action items. Also, it was not possible to determine if action items were being followed up on in a timely manner. In most instances, when direction was given during committee meetings, no timeline was assigned, which made it very difficult to determine whether the elapsed time to close an action item was reasonable.

Defence Renewal Initiative 7.2, described in Section 1.1.2 of this report, plans to review and formalize the tracking of decisions in order to improve the ability of committees and committee members to track open action items. Therefore, given this initiative, it is not necessary to make any recommendations regarding the tracking of decisions at this time.

2.2.4 Summary

Some directives and guidance are available for preparers, presenters, and reviewers of the information that is presented to the committees. However, this guidance is not sufficiently documented to promote complete, accurate, and timely information for decision making. Also, the guidance that is available is not easily referenced since it is located in multiple locations and has not been consolidated into one document. All of this could lead to inefficiencies or even decisions being made on inaccurate or incomplete information. Additionally, there are processes in place to ensure that decisions and open items are being tracked by committees; however, it is difficult to determine how timely these are actioned.

2.3 Risk Management

The reliability of information is not clearly or consistently communicated in the presentations or in the validations prepared by the committee secretary analysts.

2.3.1 Information Reliability

The reliability of information can have an effect on the decisions that are made at strategic management committees. Some examples of details that could impact the reliability of information include the following:

• age or timeliness of the information;
• completeness of the information;
• complexity of the information and difficulty of validating it, as well as whether or not it was validated by another party;
• dependability of the information system from which it was extracted;
• assumptions used; and
• likelihood that future events might materially affect the information as provided.

The review of seven presentations determined that information reliability was inconsistently presented to the committees and, in some cases, not at all. Furthermore, guidance documentation did not include direction to presenters or preparers of information on presenting information reliability to the committees. When information reliability was presented, it was not done in a clear and consistent manner.

Additionally, the reviews completed by committee secretary analysts were examined to determine if they identified any information reliability issues. Similar to the presentations, it was determined that reference to information reliability was sometimes made; however, it was inconsistently documented. Within the template provided to committee secretary analysts, there was no specific section for reporting on information reliability or any instructions on how to do so.

This inconsistency in the presentation of information reliability is due to the absence of guidance and communicated requirements to present it in a standardized format in presentations or for committee secretary analysts to include it in their reviews. This could lead to committees making decisions without knowledge of the risk related to information reliability. For example, if committee members were not made aware of the potential variance of future costs such as in-service support for a new platform, this could lead to a decision that has unexpected budgetary implications in the future.

2.3.2. Summary

Specific guidance on the requirement to present information reliability was not in place. When information reliability was addressed, it had not been done in a consistent manner and was not clearly stated. Additional guidance for identifying the reliability of information could help improve the quality of information used for decision making. Without additional guidance for preparers or reviewers, it is possible that committees could make decisions unaware of the assumptions used, the timeliness of information, or other factors that could affect the reliability of information.

3.0 General Conclusion

The decisions made by strategic management committees within DND have a significant impact on the direction of the Department. There are guidelines in place to manage the information process and promote the preparation of complete, accurate, and timely information for decision making. Additional guidance and improved diligence on the enforcement of the terms of reference for the committees could increase the effectiveness of the Department's committees.

Committee terms of reference should be consistent and include all required guidance requirements to help make the committees more effective. In addition to the terms of reference, many other useful guidance documents have already been created. However, these should be consolidated to enable more efficient preparation of presentations that better support the needs of committee members. Additionally, increased adherence to already established committee rules, such as requirements for committee attendance and deadlines for submitting information to committees, is required.

Lastly, risks related to information reliability are currently not being assessed and reported in a consistent manner that would help decision makers understand the assumptions and limitations of the information being presented to them.

Annex A—Management Action Plan

ADM(RS) uses recommendation significance criteria as follows:

Very High—Controls are not in place. Important issues have been identified and will have a significant negative impact on operations.

High—Controls are inadequate. Important issues are identified that could negatively impact the achievement of program/operational objectives.

Moderate—Controls are in place but are not being sufficiently complied with. Issues are identified that could negatively impact the efficiency and effectiveness of operations.

Low—Controls are in place but the level of compliance varies.

Very Low—Controls are in place with no level of variance.

Governance

Management Action

1.1 Corp Sec accepts ADM(RS)'s recommendation. As part of Defence Renewal Initiative 7.2 – Governance Renewal, Corp Sec is renewing the terms of reference for L0 committees and L1 functional committees. This renewal effort contains two elements: Part 1 will bring the outdated terms of references up to date with status quo governance processes in place. Part 2 goes above and beyond the recommendations of this audit and will include a complete review of the mandate, focus area, and membership list of core governance committees to enable L0 decision making. Furthermore, additional concepts such as sunset clauses and quorum will be introduced in the terms of references.

1.2 To ensure the terms of references for all the committees consistently include key guidance such as "committee stakeholder responsibilities, attendance at committee meetings, and meeting frequency," Corp Sec is establishing a standardized format for the terms of references.

1.3 To ensure the terms of references for all the committees consistently include key guidance such as "the process required to make a presentation to the committee, information on presenting to a committee, and management of committee operations," Corp Sec is creating a stand-alone standard operating procedures document that maps out committee procedures, separate from terms of references (but hyperlinked), and that breaks down the process in detail. Standard operating procedures, as appropriate, will be customized to meet specific committees' procedural requirements. The standard operating procedure document is being developed in collaboration with other L0 executive secretaries and by pulling together necessary information from a series of existing directives and sources (e.g., the VCDS Direction – Governance Meeting Briefing Guidelines; Directive for Distribution of Information in Support of National Defence Headquarters L0/L1 Executive Meetings; and Implementation of National Defence Headquarters L0/L1 Executive Meeting Management Improvements). They are also being updated to reflect current DM/CDS direction.

OPI: Corp Sec

Target Date: June 2016

Management Action

2.1 Corp Sec accepts ADM(RS)'s recommendation. The Governance Renewal includes a review of core governance committee business processes.

2.1 One of the key activities will consist of developing key messaging and communication materials to support the terms of references. Hyperlinking to the integrated standard operating procedure document, located on the Intranet page, will provide further support in this respect.

2.3 Additionally, Corp Sec is mentoring L1 executive teams on best practices that support the terms of references for L0 committees and L1 functional committees, assisting the executive secretaries and topic sponsors in ensuring briefings, presentations, and committee meetings are more timely, complete, consistent, and conducive to effective decision making. Corp Sec will continue to broaden its mentoring role, as the Defence Renewal 7.2 activities move forward.

Furthermore, a section has been added in the stand-alone standard operating procedures document to target late submission of committee materials. It clearly identifies that table drops will not be accepted as they may prevent the membership from conducting a comprehensive analysis of an agenda item and providing advice to the Chair.

2.4 Finally, regarding the "required attendance at committee meetings," the concepts of quorum and of the lowest acceptable representatives will be introduced through the terms of reference review.

OPI: Corp Sec

Target Date: June 2016

Internal Control

Management Action

3.1 Corp Sec accepts ADM(RS)'s recommendation, consistent with the management action plan described at paragraph 1.3. Working with the ADM(Fin)/CFO for IRMC, VCDS / Chief of Force Development for DCB, and VCDS / Chief of Programme for PMB, the committees for which the processes differ from standards used by other committees will be treated separately within the same stand-alone standard operating procedures document. The process will reside in one centralized location on the Intranet and will make it easier for Corp Sec to update and communicate procedural changes to the Defence Team.

OPI: Corp Sec

Target Date: June 2016

Management Action

4.1 Corp Sec accepts ADM(RS)'s recommendation and is working with the ADM(Fin)/CFO for IRMC, VCDS / Chief of Force Development for DCB, and VCDS / Chief of Programme for PMB to create standard operating procedures that will assist review analysts in the development of topic analysis to ensure completeness, clarity, and relevance of the products they develop to support the decision making of the committee chairs. This activity has been initiated and will be completed as part of the review of core governance committee business processes.

OPI: Corp Sec

Target Date: June 2016

4.2 The secretary of the IRMC will develop a template and guidelines for the completion of staff analysis of issues forwarded to IRMC for consideration.

OPI: ADM(Fin)/CFOTarget Date: June 2016

4.3 Chief of Force Development accepts ADM(RS)'s recommendation and has developed an aide memoire designed to assist Chief of Force Development staff in the analysis of the Statement of Requirement and Business Case documents, as well as the staffing of briefing documentation in support of the DCB Chair and Executive Secretary. Chief of Force Development will continue to work with Corp Sec in its review of core governance committee business processes.

OPI: VCDS

Target Date: Completed

Risk Management

Management Action

5.1 Corp Sec accepts ADM(RS)'s recommendation. Formal metrics are currently being developed through a Defence Renewal Team-led KPMG task, to rate the accuracy and completeness of submissions to L0 core governance committees. This work is in progress, and the implementation timelines are yet to be determined. This activity will be conducted with support from the ADM(Fin)/CFO for IRMC, VCDS / Chief of Force Development for DCB, and VCDS / Chief of Programme for PMB.

5.2 Corp Sec intends to review and normalize, when appropriate, the products developed by L0 analyst teams. Likewise, for a given committee or type of topic, consideration will be given for making the submission requirements more prescriptive. The degree of reliability of the information is currently communicated by analysts through Yellows (for PMB, DCB, and IRMC) or L0 topic analysis (for DSX, DMC, and Defence Executive Policy Committee).⁷ Those products are provided to the chairs in advance of the meeting. The review will also validate current expectation for submissions to include risks, impacts, assumptions, other stakeholder concerns/differing recommendations, and resource considerations. Trends that will be identified through the review of current and future performance metrics will be reported to the appropriate committee.

OPI: Corp Sec

Target Date: June 2016

Annex B—Audit Criteria

Criteria Assessment

The audit criteria were assessed using the following levels:

Assessment Level and Description

Level 1: Satisfactory

Level 2: Needs Minor Improvement

Level 3: Needs Moderate Improvement

Level 4: Needs Significant Improvement

Level 5: Unsatisfactory

Governance

Roles and responsibilities are established and communicated to govern the information management process to promote complete, accurate, and timely information.

Assessment Level 3. Guidance on roles and responsibilities is available in the committees' terms of reference; however, not all terms of reference were consistently detailed with sufficient information. In addition, improved adherence to current guidance is required to promote more complete, accurate, and timely information.

Internal Controls

Adequate processes and procedures are in place to promote the preparation and presentation of complete, accurate, and timely information.

Assessment Level 3. While processes and procedures are in place, additional documented guidance and direction is required to promote the preparation and presentation of more complete, accurate, and timely information.

Risk Management

Risks contributing to inaccurate, incomplete, and delayed information have been identified, reported, and managed.

Assessment Level 2. Risks contributing to inaccurate, incomplete, and delayed information are inconsistently assessed and reported, and there is currently no formal requirement to do so.

Source of Criteria

Treasury Board Secretariat, Audit Criteria related to the Management Accountability Framework: A Tool for Internal Auditors, March 2011.Governance:

• Reference to: G-1, G-2, AC-1, AC-2

Internal Controls:

• Reference to: G-3, G-6, ST-12, ST-18, ST-19, ST-20, ST-29

Risk Management:

• Reference to: RM-1, RM-2, RM-3, RM-4, RM-5, RM-6

___________________________________________________________________________________________________________________________

Footnote 1 The Defence priorities include the key areas of focus for the Department to achieve its mission and expected results. They also represent the areas where the Department will direct additional effort in order to address gaps in capability or capacity and where broader government direction dictates that greater action be taken.

Footnote 2 Information was obtained from the Governance and Committees Intranet page.

Footnote 3 Information was obtained from the Governance and Committees Intranet page.

Footnote 4 The audit relied on the VCDS Executive Coordination Division's strategic governance Intranet page to identify the committees that form the core strategic governance structure in the Department. (Note: Although on the Intranet page the Armed Forces Council and Commanders' Council were not identified as being part of the committees that form the core strategic governance structure in Defence, they appear to be included as strategic management committees in more recent publications.)

Footnote 5 NEMC SharePoint is used to manage committee agendas. The Web site is also used as a location to post presentations, accompanying material, meeting minutes, and any other relevant documents for committee members to access.

Footnote 6 It was not always possible to determine from the committee meeting minutes which committee meetings were attended by a representative who was acting in the position of a required office holder.

Footnote 7 Yellows and L0 topic analysis are the summaries that the analysts complete for their respective committees.