Internal Audit of Project Risk Management Practices

September 2013

7050-11-44 (CRS)

Reviewed by CRS in accordance with the Access to Information Act (AIA). Information UNCLASSIFIED.

Acronyms and Abbreviations

Results in Brief

Chief Review Services (CRS) conducted several capital acquisition audits that highlighted issues related to project risk management practices. This has led to the inclusion of an audit of project risk management practices in the fiscal year (FY) 2012/13 to 2014/15 CRS Risk-Based Audit Plan. The objective of the audit was to identify and assess risk management practices used in projects to ensure strategic and operational risks are being identified and managed proactively. Risk management is an area that continues to mature and grow in importance as updated frameworks and policies have been recently promulgated by Treasury Board of Canada Secretariat (TBS) and the Department of National Defence (DND).

Findings and Recommendations

Risk Management Policy. There is room for improvement with regard to risk tolerance levels, early risk management planning and the promulgation of key industry practices. Despite the requirement in the January 2007 DND Integrated Risk Management policy, no risk tolerance levels have been set by the Department. With regard to risk management planning, the Project Approval Directive (PAD) only requires a risk management plan (RMP) by the end of the project definition phase, usually the fourth year of a project. However, to identify project risk as early as possible, an RMP is needed to provide the necessary methodology to manage risk. The audit has identified some fundamental industry practices that could also be included in the PAD.

With input from the Defence Capability Board and the Program Management Board (PMB), the Vice Chief of the Defence Staff (VCDS) should recommend to the Chief of the Defence Staff (CDS) and the Deputy Minister (DM) that risk tolerance levels for cost, schedule, and requirements for the capital program be set at each project phase. In addition, the VCDS should update the PAD to require that an RMP be developed earlier in a project’s life and include risk ranking techniques.

Risk Management Oversight. Currently, there is no requirement to include risk information in project briefs to the PMB. While risk information is briefed to the Senior Review Board (SRB), no standardized template has been developed except for the Canadian Army projects. Reporting risks to senior management in a consistent manner and obtaining SRB approval of RMPs will help project managers make more informed decisions and will give them better oversight over their projects. Although project-specific risks are reported to the PMB, there are no mechanisms in place to monitor capital program risks. A corporate dashboard (showing key performance indicators rolled up at a departmental level) could be used to report these risks to senior management. Currently, SRBs do not have visibility of project RMPs that are approved by project managers. For complex projects, greater oversight of RMPs is needed to help ensure that due process is in place to manage risk.

It is recommended that the VCDS revise the PAD to include standard risk information to be presented at PMBs/SRBs and require Level One quarterly program briefs at PMB to include a capital program risk dashboard for complex projects as a minimum. Furthermore, the VCDS should revise the PAD to require SRBs to approve RMPs for complex projects.

Risk Management Practices. Although some projects employ good risk management practices that are taught in DND project management training, there were shortfalls in the use of industry techniques and tools to identify, assess, respond to and monitor risks for complex projects. Additional techniques could be used to identify risks and compile sufficient information to assess and mitigate the risks. For most projects, risk assessments did not quantify the potential impact on the project cost or schedule to determine project contingency funding. There was no monitoring of the risk for capital projects once mitigation plans had been implemented. Not all of the industry practices, tools and techniques are accessible to project staff on the Defence Wide Area Network.

It is recommended that Assistant Deputy Minister (Materiel) (ADM(Mat)), Assistant Deputy Minister (Information Management) (ADM(IM)) and Assistant Deputy Minister (Infrastructure and Environment) ADM(IE)) have a complete set of risk management tools and techniques available to projects on their respective websites. In addition, the VCDS should amend the PAD to include a reference to the risk management tools available.

Note: For a more detailed list of CRS recommendations and management response, please refer to Annex A—Management Action Plan.

Introduction

Rationale for Audit

In line with the Treasury Board of Canada Policy on Internal Audit, risk management is considered in every CRS audit. Eight previous CRS capital acquisition audits¹ included at least one risk management issue. Given that risk management is a key component of the Department’s capital program, these audit observations led to the inclusion of a project risk management audit in the CRS Risk Based Audit Plan for FY 2012/13 to FY 2014/15.

Background

Benefits of Risk Management. Risk is defined as the possibility that an event will occur and adversely affect the achievement of objectives.² Risk management is defined as a systematic approach to setting the best course of action under uncertainty by identifying, assessing, understanding, making decisions on and communicating risk issues.³ Risk management serves two main purposes: taking advantage of opportunities and minimizing negative outcomes in the future. Effective risk management should lead to positive consequences, including more effective decision making.⁴ If risk is not managed effectively in the capital program, the operational capability of the CAF could be delayed or project cost could increase.

Policies. The Treasury Board (TB) Framework for the Management of Risk was released in 2010 and replaced the 2001 Integrated Risk Management Framework. The new framework was intentionally not written as a policy to allow departments the flexibility to write their own policies that could be tailored to departmental objectives. The DND Integrated Risk Management policy was approved in 2007 to develop and strengthen the Department’s awareness of risk management. A new DND Integrated Risk Management policy has been in draft since November 2011. The PAD is another key document that provides risk management policy specific to projects. It replaced the Project Approval Guide in October 2011. Risk management guidance is also found in the Project Management Body of Knowledge and the ISO 31000 standard, two authoritative documents stemming from outside the Department.

Project Risk Management in DND. As portrayed in Figure 1, project risk management occurs throughout the life cycle of a project. The PAD requires that risk management be documented in project briefs, risk registers and RMPs. A Project Complexity and Risk Assessment (PCRA) must also be submitted to TBS for each project before it can proceed to the definition phase and implementation phase. The PCRA determines whether a project can be approved by the Minister of National Defence (MND) or by TB. Each year, the Department also produces a Corporate Risk Profile that documents the major risks facing DND.

Figure 1. Project Risk Management Documents and Approval Authorities. This flowchart demonstrates the various risk management documents and approval authorities required during each phase of the project life cycle. TB approval may be required, depending on the Organizational Project Management Capacity Assessment level and PCRA scores.

Objective

The objective of this audit was to identify and assess risk management practices used in projects to ensure strategic and operational risks are identified and managed proactively.

Scope

The scope of the audit included the following:

• major capital equipment, infrastructure and information management projects only—minor projects less than $5 million were excluded; and
• risk management practices in all phases of a project’s life cycle: Identification, Options Analysis, Definition and Implementation.

The audit excluded the following from the scope:

• risk management related to project contingency estimates because a separate CRS audit of capital project cost estimation was under way at the time of the audit; and
• risk management by defence contractors.

Methodology

The audit results are based on evidence from the following sources:

• interviews with key DND and TB staff;
• an examination of policies, guides, frameworks, industry standards, project documentation and the Capability Investment Database (CID);
• a sample of 15 current information management, infrastructure and equipment projects in different phases with a value of $38.8 billion representing 36 percent of the major capital program; and
• focus group discussions with individuals from each project office in the audit sample.

Criteria

The audit criteria are outlined in Annex C.

Statement of Conformance

The audit findings and conclusions contained in this report are based on sufficient and appropriate audit evidence gathered in accordance with procedures that meet the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing. The audit thus conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program. The opinions expressed in this report are based on conditions as they existed at the time of the audit and apply only to the entity examined.

Findings and Recommendations

Risk Management Policy

There is room for improvement in the departmental risk management policy with regard to risk tolerance levels, early risk management planning, and industry practices.

Risk Tolerance Levels. Although the DND Integrated Risk Management policy requires that risk tolerance levels be set by the Department, they have not yet been established. The policy reflects the TBS Framework for the Management of Risk that requires the establishment of risk tolerance levels as part of effective risk management. With guidance on acceptable risk levels at each phase of a project, project staff could manage risks related to cost, schedule and capability requirements more effectively. Tolerance levels may be different depending on the project type. For example, information management projects are usually higher risk than construction projects. Although the DND costing handbook provides some guidance on cost confidence levels at each phase of a project, similar tolerance levels do not exist for schedule or capability requirement risks.

Risk Management Plan Timing. The current project approval process requires that a project creates an RMP by the end of the definition phase, as an annex to the Project Management Plan. Industry practices and ISO 31000 suggest creating a risk management plan in the early stages of a project. The RMP generally includes the methodology for developing the identification phase risk assessment in the project brief, the Project Opportunity and Risk Assessment and the risk register. Creating the RMP earlier in the project life cycle process would make it easier to develop the risk register and project brief. The RMP could then become a living document that would be updated as necessary.

Risk Assessment Practices. The PAD could be improved by including fundamental industry practices for risk management. Specifically, the risk map lacks numerical impact and probability scores as well as positive outcome thresholds. Including numerical scores for impact and probability makes risk severity assessments more accurate so that risk mitigation strategies can be ranked in order of priority.

In addition to threats to a project’s future outcomes, opportunities should also be considered during the risk management process. Evaluating impact/probability for positive outcomes will allow project teams to consider the beneficial opportunities available to the project.

Project Opportunity and Risk Assessment Content. The Project Opportunity and Risk Assessment that was developed to provide a summary of the main risks of a project does not include a complete list of common risk categories. Unless a project brief already includes a risk information summary, an up-to-date Project Opportunity and Risk Assessment is required especially for complex projects. The information required in the Project Opportunity and Risk Assessment is a summary of the scope, cost, schedule and program risks for a project. Common risk areas that are not included in the Project Opportunity and Risk Assessment are human resources and procurement that need to be assessed early in a project life cycle to avoid significant schedule delay.⁵

Summary. Despite requirements in the DND Integrated Risk Management policy, no risk tolerance levels have been set for the Department. Without such guidance, inappropriate mitigation strategies could be adopted by projects that increase costs, delay schedule, or reduce capability requirements. Risks that are normally higher in the early phases will not be mitigated early enough or ranked in order of severity under the current PAD policy on risk planning and assessment. The policy also does not include the identification or assessment of procurement and human resource risks.

Risk Management Oversight

Better oversight of risk management at both the program and project level is needed.

Corporate Dashboard. Although it is the responsibility of the PMB to assess the risk of the capital program,⁶ currently, there is no requirement for a corporate dashboard to identify program risks, which would be used to identify the level and type of risk anticipated by projects. ⁷ Although there is a performance dashboard of current issues presented quarterly at PMB for ADM(Mat), there is no risk reporting of the future outcome of the capital program for each environment to senior management. From the analysis of the available CID⁸ project risk data, it was found that schedule risks were more prevalent than cost or technical risks. A corporate view of risk information could be useful to senior management as it would indicate which categories of program risk or individual projects need attention to help minimize loss or delay of future operational capability.

Risk Reporting at Oversight Committees. As it currently stands, the PMB briefing format for projects does not include standard risk information and is limited to seven slides. Moreover, with the exception of the Canadian Army, there is also no standard format for briefings at the SRB. The reporting of risks to both the PMB and SRB is important as it helps senior management make decisions and provides assurance that the mitigation plans address significant risks.

For the audit sample of 15 projects, the audit team reviewed 84 PMB and SRB briefings held over 10 years and found that 21 of the briefings had no evidence of project risk being discussed. The inclusion of risk information in a standard briefing format would ensure that risks are consistently reported to senior management.

Project Risk Management Planning. The RMP is approved by the project manager and outlines a project’s methodology. It is used to conduct continuous risk analysis in each project phase. Although this key document does not require approval by the project SRB, it should detail the project risk management process as well as the tools and approaches used to mitigate risks. However, an analysis of the 15 projects in the audit sample showed that the projects did not all have RMPs and that most projects did not distinguish between inherent and residual risk.

Summary. To foster program risk reporting to senior management, a corporate dashboard could be an effective tool to gather and summarize project risk information at PMB. Without a view of the capital program risk, a strategic approach for safeguarding future operational capabilities is more difficult to achieve. From the audit sample of projects, it was found that risk is not always reported to the oversight bodies (SRB, PMB) due in part to the lack of a standard requirement for risk management briefings. As a result, management boards do not always have complete information regarding the identification of and mitigation of significant risks. As well, not all projects detailed their risk management methodologies in an RMP and some risk management practices were inconsistent with those stipulated in the PAD and Project Management Body of Knowledge. Currently approved by the project manager, the RMP for complex projects requires greater oversight as it acts as the risk management framework for the project. Without a sound RMP, the thorough identification, assessment, response and monitoring of risk is less likely.

Risk Management Practices

Industry practices in project risk management are not being implemented in all projects.

Industry Practices Formal DND risk management training courses include industry practices in accordance with the Project Management Body of Knowledge.⁹ Some of the industry risk management practices are available on the Materiel Knowledge Network (MatKNet).¹⁰Although most project management staff have received some level of risk management training, once a project is in the definition phase, we have found that more industry practices could have been used in the audit sample of 15 projects. These industry practices are necessary to identify, assess, respond to and monitor project risks. The detailed results of the audit sample analysis of industry practices may be found at Annex D.

Risk Identification. Proper risk identification will allow project staff to mitigate risks early in the project before they become more serious issues. For example, one common risk identification standard in ISO 31000 is to conduct an analysis of the interest and influence of all stakeholders on a project. However, from the audit sample, the following was observed:

• Most projects did not include the industry practices related to risk identification in their RMPs. Therefore, some risks may not be identified.
• Some projects did not use risk information sheets or the Risk Radar detailed report prescribed in the Materiel Knowledge Network. These tools are used to provide complete information for each risk in order to assess and respond to and monitor the risk.
• Most project RMPs did not contain a methodology for identifying potential positive outcomes in order to take advantage of opportunities.

Risk Assessment. A risk assessment should estimate the likelihood and impact of each risk, classify the risks, and rank the risks in order of severity. By doing so, projects can focus resources on higher-priority risks. There were several shortfalls in the assessment of risks in the audit sample of projects:

• Half of the projects did not have enough risk information in the project risk documentation to assess the severity level of each risk.
• Some projects did not use a risk register tool from the PAD or the Materiel Knowledge Network to assess and rank the risks.
• Most projects with risk registers did not use risk quantification techniques to calculate the required contingency funding necessary to mitigate risk.
• The project risk management practices were not always in accordance with the practices described in the projects’ RMP. Some projects did not use the risk assessment tools that were described in their RMPs.

Risk Quantification Tool. Although the need for a risk quantification tool has been previously recommended by CRS,¹¹ the Department does not have a tool available to facilitate this aspect of risk management. There are several applications available that help in quantifying contingency funds necessary to mitigate cost and schedule risks. Assistant Deputy Minister (Finance and Corporate Services) is in the process of acquiring a project costing tool. This costing tool will enable projects to quantify the costs associated with risk.

Risk Response and Monitoring. The risk response includes options and actions to minimize outcome threats and enhance opportunities. The response should be timely and cost-effective. Risk monitoring requires tracking of identified risks, implementing mitigation plans, monitoring residual risks, identifying new risks, and evaluating the effectiveness of the risk management process. Several projects in the audit sample did not employ good risk response and monitoring practices:

• Most projects did not distinguish between the inherent and residual risk in their risk register.
• Some projects with a risk register did not include the necessary information to monitor each risk.
• A review of the project meeting minutes found that one third of the projects had no discussion of risk management in their meetings.

Summary. Not all projects are using industry risk management practices to identify, assess, respond to and monitor risks. Only three projects in the audit sample had a risk trend that indicated a decrease in project risk, and most projects did not quantify the risk impact and mitigation costs to determine the total contingency funding. As portrayed in Annex E, many of the tools and techniques that are included in the DND risk management courses are not made available on any DND website and there is no reference in the PAD to risk management tools or techniques. With a 23 percent project office vacancy rate, there are simply not enough resources to implement all risk management processes. Therefore, the appropriate risk management tools need to be readily available for project staff that do not manage risk as a full-time responsibility.

Risk Management Training

Most projects’ risk management plans have not formally identified risk management training requirements.

Risk Management Training. Most RMPs did not identify the required level of risk management training. Certain project staff should have a formal four-day risk management course while other staff may only require a three-hour course available on line. The RMP should identify the key project staff who need formal training.

Project Director Training. The initial stages of projects are managed by the project director staff before the transition to the project manager at the definition phase. Representing the project sponsors’ operational environments from the Royal Canadian Navy, Canadian Army and Royal Canadian Air Force, project directors often have less risk management experience than the project implementation staff. Usually, project risks are much higher in the early phases of a project. Therefore, it is critical that project director staff receive formal risk management training.

Risk Management in Smaller Projects. Projects with smaller budgets have fewer resources and less staff to dedicate to risk management. Unlike larger complex projects, the risks in smaller projects may not be as significant. While some risk management tools are more appropriate for larger projects, the repository of tools also includes user-friendly applications such as Risk Radar that could be used by smaller projects.

Summary. There are currently some shortfalls in the identification of risk management training requirements that are needed to effectively mitigate project risk. If risk management training needs are not identified early in a project when risks tend to be more significant, projects are not well positioned to proactively identify and manage risks.

Conclusion

Improvements in project risk management policies, practices, and oversight are needed in order to ensure that strategic and operational risks are being identified and managed proactively.

The audit found room for improvement in the risk management policy. In particular, risk tolerance levels have not been established, risk management planning was not expected early enough and fundamental industry practices were missing. As well, the internal reporting necessary for the effective oversight of project risk could be strengthened. To identify capital program risk, a corporate risk dashboard could be presented to senior management by each environment. SRBs and PMBs are two forums where project risk oversight can occur but there is no standard for briefing on risk. Greater oversight of risk management will create an impetus for project staff to ensure that more industry practices are in place.

Although good practices were in place for some projects, the audit team observed that recognized industry practices were not followed by all projects. Without these industry practices, project staff have not been able to reduce the project risk over time and are unable to quantify the risk mitigation costs to calculate contingency funding. There are many industry techniques and tools that are taught on DND risk management courses that are not readily available to project staff, but could facilitate easier and more effective risk management. Improved access to risk management tools and techniques adapted to the complexity of a project should address the shortfalls observed in risk identification, assessment, response and monitoring phases. Training requirements also need to be identified early in a project to ensure key project sponsor and implementer staff have the appropriate risk management training.

Annex A—Management Action Plan

CRS uses recommendation significance criteria as follows:

High—Controls are not in place or are inadequate. Important issues are identified that could negatively impact the achievement of program/operational objectives.

Moderate—Controls are in place but are not being sufficiently complied with. Issues are identified that could negatively impact the efficiency and effectiveness of operations.

Low—Controls are in place but the level of compliance varies.

Risk Management Policy

Management Action

Departmental and Government of Canada risk tolerance for cost is very low and defined. Latitude exists in scope and schedule risk tolerance levels, the analysis of which will be included in the ongoing Project Approval Process Redesign Business Process Review. A communications plan will also be developed to determine an effective way to communicate these changes to all stakeholders.

OPI: VCDS

Target Date: December 2014

Management Action

The PAD will require that the RMP be completed prior to a project entering the definition stage, verified by the Chief of Programme analyst prior to the PMB. Project leaders will be encouraged to include the RMP during the options analysis stage, to be verified at SRBs.

OPI: VCDS

Target Date: November 2013

Risk Management Oversight

Management Action

The VCDS will ensure that the bi-annual briefs by the functional Level One and the environmental command staffs will include a program risk dashboard. National Defence Headquarters Secretariat will amend the PMB templates to reflect this change and Chief of Programme analysts will ensure that this information will be provided in future briefs.

OPI: VCDS

Target Date: November 2013

Management Action

A standard risk briefing slide will be developed by National Defence Headquarters Secretariat as a requirement for PMB and Defence Capability Board briefing packages. Project leaders will be encouraged to present their information in a similar format for their SRBs.

OPI: VCDS

Target Date: January 2014

Risk Management Practices

Management Action

ADM(IE) will provide a direct link on its website to the ADM(Mat) Materiel Knowledge Network website. The Project Risk Management site is a key source for tools, procedures, and guidelines for ADM(IE) project managers.

OPI: ADM(IE)

Target Date: 31 October 2013

ADM(IM) will coordinate with ADM(Mat) to provide a single source of tools and techniques that are adaptable to Information Management/Information Technology project complexity utilizing the Materiel Knowledge Network website managed by ADM(Mat). Furthermore, ADM(IM) will ensure ADM(IM) project staff are aware of the availability of additional tools and techniques by providing a link from the ADM(IM) Group website to the Materiel Knowledge Network website.

OPI: ADM(IM)

Target Date: 31 December 2013

ADM(Mat) will update the Materiel Knowledge Network website to include, or link to, risk management tools/techniques, including industry practices according to the Project Management Body of Knowledge, 5th Edition. The Materiel Knowledge Network website will also contain improved procedures, advice, and guidance as to how project staff should develop and manage their risk management strategy.

OPI: ADM(Mat)

Target Date: Completed

Management Action

The PAD will be amended to provide references to risk management tools and techniques used within the Department. Each Level One is requested to provide a list of, and links to, all the risk management tools they believe to be of value to the Chief of Programme/Director Defence Programme Coordination 6 so that it may be added to the PAD. They shall thereafter conduct a review of the links they provided on an annual basis to ensure that the references are still currently in use and valid.

OPI: VCDS

Target Date: January 2014

Risk Management Training

Management Action

Chief of Programme staff will verify that risk management training has been completed by project staff as a pre-condition for the progress of complex projects. It remains the responsibility of project staffs to ensure that their members have the proper level of risk management training.

OPI: VCDS

Target Date: Ongoing

Annex B—Project Sample

Table 1. Project Sample. All risk management documentation was examined in detail for these projects—a mixture of equipment, information management and construction projects in different phases. Focus groups were also conducted with individuals from each of the related project offices.

Annex C—Audit Criteria

Objective

The objective of this audit was to identify and assess risk management practices used in projects to ensure strategic and operational risks are identified and managed proactively.

Criteria Assessment

The audit criteria were assessed using the following levels:

Governance

1. A documented governance framework is in place to effectively develop and implement integrated risk management in projects. (Risk Management 1)

Assessment Level 3 – Improvements are needed in the draft Integrated Risk Management policy. The risk management chapter in the PAD needs to include some key industry practices from the Project Management Body of Knowledge. Risk tolerance levels have not been set in the Department.

Internal Controls

2. An effective internal control system is in place to ensure risk management procedures are followed. (Risk Management 8)

Assessment Level 4 – There is no internal reporting system for the capital program risk. Improvements are needed in the oversight of risk management planning and the reporting of risks.

3. Project risks and management strategies are embedded in the organization’s planning. (Risk Management 6)

Assessment Level 2 – Risk management is properly embedded in most project planning.

Risk Management

4. A formal and effective project risk management process is in place to manage the project risks proactively – including all phases of risk identification, assessment, response, communication, and monitoring. (Risk Management 2, Risk Management 3, Risk Management 4, Risk Management 5 and Risk Management 7)

Assessment Level 4 – Many common industry practices were not implemented by several projects to identify, assess, respond to and monitor project risks and should be adopted to improve their risk management practices.

5. Staff are provided with the necessary training, resources and information to support their risk management responsibilities. (People 4)

Assessment Level 3 – Some risk management training is available; however, minimum training levels are not specified for most projects.

Source

Audit Criteria Related to the Management Accountability Framework: A Tool for Auditors, March 2011 (see reference after each criterion above).

Annex D—Industry Practices Analysis

Table 2. Audit Sample Results. The risk management practices in the audit sample of 15 projects were compared to 18 industry practices outlined in the Project Management Body of Knowledge.

Annex E—Risk Management Tools

Table 3. Risk Management Tools Available. The risk management tools available on the Materiel Knowledge Network and in the DND Integrated Risk Management guidelines are compared to additional tools available through sources such as the Project Management Body of Knowledge.

*Project Management Body of Knowledge

___________________________________________________________________________________________________________________________

Footnote 1 CRS capital acquisition audits from 2005 to 2011: Chemical, Biological, Radiological, Nuclear Omnibus project; Fixed Wing Search and Rescue; Halifax-Class Modernization/Frigate Life Extension; Joint Support Ship; Tactical Armoured Patrol Vehicle; Fleet Maintenance Facility Cape Breton; Materiel Acquisition and Support Information System; and Data Management System Contract.

Footnote 2 Committee of Sponsoring Organizations of the Treadway Commission; Internal Control – Integrated Framework 2013.

Footnote 3 TBS Framework for the Management of Risk 2010.

Footnote 4 Ibid.

Footnote 5 These two risks are required to be reported at the monthly interdepartmental oversight committee for major crown project meetings as they play a large part in determining the health of a project.

Footnote 6 PAD Chapter 15 paragraph 15.1.8.

Footnote 7 Such a dashboard exists at the monthly Interdepartmental Oversight Committee for Major Crown Projects. However, only 16 of 912 projects in the DND capital program are reported, representing 50 percent of the capital program value.

Footnote 8 The CID does not distinguish between inherent and residual risk. As well there is little guidance or information on the threshold for each risk severity level. CID risk reporting is being replaced with a Defence Resource Management Information System business intelligence tool.

Footnote 9 DND Materiel Acquisition and Support policy DAOD 3000 includes Project Management Body of Knowledge as the departmental project management standards.

Footnote 10 The Materiel Knowledge Network is an ADM(Mat) website that acts as a forum for sharing knowledge of materiel acquisition and support processes.

Footnote 11 CRS CP140 Aurora Data Management System Contract audit, August 2007.