Independent Review Panel for Defence Acquisition 2015-2016 Annual Report

The Independent Review Panel for Defence Acquisition (IRPDA) was established by the Governor in Council on May 29, 2015 to provide a credible, independent challenge function for the requirements of major military procurement projects as part of broader efforts to improve defence procurement. In accordance with its Terms of Reference, the Panel challenges military requirements, including the assessment of a project’s alignment with government policy and the nature of the capability gap a project is intended to fulfill. In so doing, the IRPDA provides independent advice to the Minister of National Defence via the Deputy Minister of National Defence before these projects seek Ministerial or Treasury Board approval.

This is the Panel’s first annual report and it contains highlights of the past year’s accomplishments, as well as early observations on key issues. In addition, Appendix A outlines the Panel’s membership and Appendix B provides details on the IRPDA’s budget. To support preparation of this report, the Panel conducted a self-assessment exercise, met with senior officials (e.g., the Deputy Minister and Associate Deputy Minister of National Defence; Vice Chief of the Defence Staff), and engaged the Department of National Defence (DND)/Canadian Armed Forces (CAF) force development community to gather observations and lessons learned. In future years, the Panel’s annual report will incorporate results from its recently established performance measurement processes.

Due to the sudden passing of Mr. David Caddey in September 2015, the Panel currently has a vacancy. Over the coming year, the Panel will support efforts to fill the vacancy through the Governor in Council appointment process and integrate the new member, while also giving due consideration to succession planning.

Maintaining Independence

The Panel’s unique contribution to the defence procurement system is based on its ability to provide an independent challenge function that enhances the confidence and trust of decision-makers in military requirements.

As part of the process of establishing the Panel over the last year, an IRPDA Governance Policy was developed and implemented with the support of a contracted third-party. This Policy incorporates the Panel’s Terms of Reference and outlines the practices, procedures and risk mitigation strategies designed to maintain the independence of the IRPDA both in reality and appearance. The Panel will regularly monitor its independence through its performance measurement framework. In addition, the Panel will continue to seek advice from the Office of the Conflict of Interest and Ethics Commissioner, when needed, to ensure the ongoing integrity of the Panel.

Accomplishments

The Panel commenced its work in June 2015, immediately following the announcement of its creation. Since then, the Panel has engaged with all relevant stakeholders within DND/CAF as well as senior officials from other departments, and has met monthly to review projects and conduct other related business. Between June 2015 and June 2016 (inclusive), the Panel met 14 times for two or three days.

During this period, the Panel reviewed a total of 17 projects at various stages in the procurement process, and submitted advice to the Minister on 4 of these projects. In accordance with the Panel’s mandate, a variety of issues were examined for each project (e.g., the capability gap; proposed requirements; options analysis; broader procurement context such as in-service support and anticipated level of competition) to enable the Panel to draw clear, independent conclusions about the proposed projects and their requirements.

Early Observations

DND/CAF’s Engagement with the Panel

Over the past year, new processes required to support DND/CAF engagement with the Panel have been effectively implemented. DND/CAF leadership has embraced the concept of an independent challenge function for military requirements, prioritized effective and collaborative engagement with the Panel, and issued supporting departmental directives and guidance. In addition, the Panel has been able to access departmental expertise directly, not via a central hub or solely through project sponsors, allowing it to probe issues and obtain perspectives directly from subject matter experts and senior officials. As a result, the Panel is satisfied that it had access to the information and personnel required to fulfill its mandate.

Linking Strategic Context and Policy Direction to Project Requirements

The Panel believes that establishing clear links between government policy, an identified capability gap, and a project’s detailed operational requirements is critical for conveying a project’s importance and rationale to decision-makers. However, the Panel has noted that DND/CAF has faced challenges in clearly articulating these linkages in certain instances. In some of these cases, the documentation has not clearly connected existing policy or direction to requirements, while in others government policy has not been clear or does not yet exist. The Panel looks forward to the results of the ongoing Defence Policy Review as it will provide the foundational strategic policy for future projects.

In addition, the Panel would like to highlight the work undertaken through DND/CAF’s Capability Based Planning process. The analysis and conclusions produced as a result of this process, as well as other related force development activities, should be integral to the development of requirements and project scopes. While some projects have incorporated Capability Based Planning results in their documentation, the Panel believes that a wider and more consistent application would greatly strengthen the analyses presented in project documentation, and complement professional military judgement.

The Panel also believes that it is important to list the activities and other projects that contribute to a capability and to demonstrate the relationship between individual projects and activities. For example, Command, Control, Communications, Computers, and Intelligence, Surveillance and Reconnaissance (C4ISR) projects can now be tied to an overarching DND/CAF framework to show how individual projects contribute to the broader CAF vision and C4ISR-related capability gaps.

Overall Quality of Requirements

Over the years, major military procurements in Canada have been subject to concerns that the military’s requirements exceeded what is necessary, or that they were directed towards a specific platform. The Panel did not see evidence of these concerns in the 17 projects reviewed in the past year. It has, however, observed that certain requirements would benefit from additional clarity or refinement, as noted in the section on High Level Mandatory Requirements (HLMRs) below.

In addition, the Panel has consistently encouraged the inclusion of comprehensive and rigorous analyses within project documentation. It believes that projects which “explain” the CAF's requirements, and on what basis the requirements were derived, are more useful than those that simply “tell” the decision-maker what the requirements are. In the Panel’s experience, project documents that clearly outline the judgements, calculations, and considerations that informed the development of project requirements contribute to greater understanding by decision-makers. This approach will also lend additional credibility to the scrutiny applied by internal DND/CAF governance processes in validating the requirements.

High Level Mandatory Requirements (HLMRs)

The concept of HLMRs was developed to provide high level, non-specialist expressions of defence requirements to support decision-making. HLMRs define the expected outcomes or effects to be delivered by a project, and failure to achieve or deliver any one of these HLMRs constitutes project failure from a DND/CAF perspective. The DND/CAF force development community has embraced the importance of this concept, and has devoted much effort to improve the substance of HLMRs. In particular, there is now a greater focus on the articulation of HLMRs at the front-end of the process, as well as on alignment and consistency between the capability gap, HLMRs and the preliminary Statements of Operational Requirements. This should pay dividends in the future as projects will be guided by requirements with clear links to HLMRs and overall project objectives.

However, this remains a work in progress. The Panel notes that a degree of uncertainty exists with respect to HLMRs, and greater refinement and clarity would be beneficial. Across the 17 projects reviewed, there have been occasions where greater specificity within the HLMRs would have enhanced the overall quality of the requirements by more precisely identifying what essential capability was required for project success. The Panel will provide input to DND/CAF’s upcoming revised direction on HLMRs, and will continue to focus its attention in this area going forward.

Analysis of Options

The Panel understands that templates and directives often require a particular approach be followed within the Options Analysis phase. However, the Panel recommends that DND/CAF consider focusing option analyses on the ability to meet essential operational requirements. In other words, options should first be screened against the HLMRs, with other select key criteria, such as business outcomes and cost, being applied after confirming that an option meets all HLMRs. This may help inform future decisions, particularly given the trade-offs that are often required between cost, capability and schedule.

Capacity in the Force Development Community

The Panel has been impressed with the dedication of DND/CAF staff responsible for the projects reviewed in the last year. The Panel has also noted the efforts by DND/CAF to discipline which projects move forward through departmental governance based on established priorities and the department’s investment portfolio to help ensure that limited resources are being focused on the most important projects.

However, in the Panel’s opinion, there is not enough overall capacity dedicated to force development, particularly in the Chief of Force Development (CFD) organization, but also in the environmental commands. The Panel believes the allocation of additional DND/CAF resources earlier in a projects life would contribute to the department’s ability to deliver the right equipment in a timely manner and support the implementation of the Defence Policy Review. To this end, the Panel encourages the initial efforts it understands to be underway to provide greater support (e.g., procurement-related, technical and financial expertise) to projects in the Options Analysis phase.

DND/CAF Internal Processes

To support the objective of acquiring the right equipment for the CAF in a timely manner, the Panel reviews pre-existing, approved documentation and makes best efforts to ensure that it does not delay projects by seeking changes that are not substantive or by requesting any non-essential additional work. Despite the Panel’s efforts in this area, it has observed that a more streamlined approach is needed for DND/CAF to manage both minor and major changes to requirements and documentation. Greater flexibility for projects to adjust requirements and expedite internal approvals would help projects make more timely and efficient progress through the procurement process, including any required returns to the Panel.

Performance Measurement

In future years, the Panel’s annual report will include results from its recently established performance measurement framework, which will focus on measuring the following strategic objectives:

• Appropriate Requirements - Requirements validated by the IRPDA support the delivery of the right equipment to the CAF in a timely manner.
• Useful Advice - Advice provided to the Minister of National Defence enhances decision-making on major military procurement projects.
• Enhanced Confidence - The defence procurement process is improved by enhanced confidence and trust with decision-makers and improved credibility with, and support from, industry.

Conclusion

In an area as complex as defence procurement, it will take time before broader, systemic results can be determined or measured. However, thus far, the Panel is generally satisfied with the approach being taken by DND/CAF to adapt procurement processes and incorporate an independent challenge function for military requirements. Over the coming year, the Panel will continue to provide a robust and independent challenge function to the requirements of major military procurement projects. The Panel will also continue to provide an independent perspective on issues related to the establishment of HLMRs and other force development activities as required.

The Panel recognizes that its work during this period has been greatly enabled by the dedication of those in the DND/CAF and broader defence procurement community. The Panel would also like to acknowledge the IRPDA Office, which serves as the daily presence of the Panel and provides Panel members with information, analysis and support. The Office will be prioritizing the completion of staffing activities in the coming year to ensure it is able to continue to provide effective support to the Panel. Finally, the Panel members were delighted that as a result of the Office’s significant contribution to the establishment and ongoing operations of the IRPDA, the staff were recently recognised by DND/CAF senior leadership with a Deputy Minister/Chief of the Defence Staff Innovation Award for “implementing transformational change to National Defence procurement.”

Appendix A: Panel Membership

Appendix B: IRPDA Budget