Language selection

Government of Canada / Gouvernement du Canada

Search

Independent Review Panel for Defence Acquisition – Progress Report 2019-2020

Progress report 2019-2020 [PDF - 2.38 MB]

ISSN 2564-159X

Table of Contents

Foreword from the Chair

The Independent Review Panel for Defence Acquisition (IRPDA) presents its fourth progress report, covering the 2019-2020 calendar years. It includes a summary of the Panel’s activities during this period, strategic observations based on the Panel’s work, and a statement of the Panel’s priorities going forward. The Panel is also pleased to present for the first time a complete list of the projects that it has reviewed (Annex A), feedback from the most recent survey undertaken with key IRPDA stakeholders (Annex B), and an illustration of its standard review methodology (Annex C).

The COVID-19 pandemic has presented unique challenges for the Panel, as both members and office staff have been dispersed in various regions of the country. The Panel is deeply grateful for the support that it has received from Maritime Forces Atlantic, 3 Wing Bagotville, 4 Division Headquarters Toronto, and CFB Edmonton, as well as from its own administrative staff, in ensuring that Panel members can continue their work remotely. Thanks to their assistance, the Panel was able to adapt to new virtual working methods, keep pace with DND/CAF processes and priorities, and continue to deliver timely feedback and advice to the Minister and the Department. The Panel also extends its gratitude to Departmental and office staff for facilitating the move of the Panel’s physical office space to the new National Defence Carling Campus in early 2020.

Since its establishment in 2015, the Panel has initiated the review of 70 major procurement projects, undertaken 146 engagements with Departmental stakeholders as part of those reviews, and delivered independent advice on 47 of these projects. Feedback collected from stakeholders indicates that the Panel’s work remains credible and relevant, and is positively impacting the defence procurement process. Going forward, the Panel will continue to support the Department in delivering on the objectives of Strong, Secure, Engaged (SSE), while responding to new capability priorities. Moreover, as the first projects on which the Panel provided advice begin to enter the final phases of the procurement process, we look forward to assessing the Panel’s impact on the wider defence procurement system and outcomes.

On behalf of the Panel, I would like to extend my sincere thanks to the Department for its ongoing support and meaningful engagement in the Panel’s review process. Enhancing the effectiveness of and confidence in the defence procurement process is a truly collective enterprise, and the Panel is proud and honoured to play a part in this vital effort.

[Original signed by]

Mr. Larry Murray
Chair of the Independent Review Panel for Defence Acquisition

Introduction

The IRPDA was created in 2015 as one of the core elements of the Defence Procurement Strategy. At that time, military procurement faced several challenges, including a perception that operational requirements were insufficiently justified and crafted in such a complex manner that they complicated the approval process. As a result, the Panel was established as an independent, third-party review body, reporting directly to the Minister of National Defence, to validate the requirements of major military procurement projects and ensure that they are appropriately stated prior to the expenditure of significant funds.

In accordance with the Panel’s Terms of Reference, every defence equipment and infrastructure project valued at over $100M is carefully reviewed by the Panel, in close consultation with relevant Departmental officials and in line with the Panel’s well-established and disciplined review methodology (see Annex C). From time to time, the Panel may also review additional projects that have been referred to it by the Minister or Deputy Minister of National Defence, or when Treasury Board approval is required.

The Panel reviews projects at two points early in the defence procurement process, at the beginning and at the end of the Options Analysis phase, before DND/CAF seeks approval from the Minister or Treasury Board for expenditure authority (Figure 1):

  • IRP1 engagement: The first Panel engagement (IRP1) focuses primarily on the strategic context and policy alignment of the project, the capability gap the project intends to address, and the High Level Mandatory Requirements (HLMRs). In this initial engagement, the Panel reviews the Strategic Context Document (SCD) after it has been endorsed by senior DND/CAF officials at the Defence Capabilities Board (DCB). This first engagement helps the Panel identify any potential issues with a project before the detailed work in Options Analysis begins. Following the first engagement, detailed feedback is provided to the Department, both to the Vice Chief of the Defence Staff (VCDS) and other senior-level stakeholders, as well as directly to the project team.
  • IRP2 engagement: The second Panel engagement (IRP2) focuses primarily on the results of Options Analysis, and on ensuring traceability from the High Level Mandatory Requirements to the more detailed and technical requirements captured in subsequent project documentation. As part of these engagements, the Panel reviews the Business Case Analysis (BCA) and the Preliminary Statement of Operational Requirements (PSOR), again following endorsement by senior DND/CAF officials.
  • Supplementary engagements: When necessary, the Panel will pursue supplementary engagements to ensure that any issues with the projects are addressed as early as possible, and that the Panel has sufficient information to produce credible advice for the Minister. For complex projects, the Panel may also request a ‘scene setter’ briefing at the outset of the review, to ensure that members have sufficient background information to review projects at IRP1.

Figure 1 – Panel Review Process

Upon completion of the review process, the Panel submits advice to the Minister, through the Deputy Minister, with a copy to the Chief of the Defence Staff and other relevant senior DND/CAF officials. The advice is submitted before the Department seeks approval from the Minister or Treasury Board to enter into Definition and to expend significant capital investment funds. The specific timeline for each project is unique, and the submission of advice will generally occur anywhere from one to three years after the initial IRP1 engagement. The Panel’s advice to the Minister is treated as Cabinet confidence.

In accordance with the Panel’s Terms of Reference, the Panel’s advice is focused on providing an independent evaluation of whether the Canadian Armed Forces is facing a credible capability need, whether the project requirements are clearly and appropriately stated, and whether there is a strong rationale for the recommended procurement options. In addition, the Panel may highlight any additional considerations that might be useful to the Minister or Deputy Minister to help them better understand the stated requirements for the project and to inform critical project decisions going forward. For example, the Panel may highlight potential cost or schedule risks, or unique integration challenges.

Panel advice can be shared with central agencies with the approval of the Minister.

Year in Review: Activities in 2019-2020

To give a sense of the scope and significance of the Panel’s work, the Panel is likely to review over 60% of the major projects within the currently funded DND/CAF capital equipment program.Footnote 1 In addition, the Panel reviews some unfunded initiatives and infrastructure projects.

Despite the difficult circumstances of the COVID-19 pandemic, the Panel’s review agenda remained busy throughout this reporting period.

The Panel continued to focus on projects associated with SSE implementation while also addressing emerging capability requirements. During this period, moreover, the Panel was asked to undertake a series of special reviews for the Canadian Coast Guard (CCG). Building on its review of the CCG Fleet Renewal Plan in late 2018, the Panel examined and provided advice on the Polar Icebreaker Project and initiated a review of the Multi-Purpose Vessel Project.

In 2019 and 2020, the Panel reviewed 42 projectsFootnote 2 and conducted 55 distinct engage ments related to these projects. As well, the Panel submitted 23 letters of advice. Advice on the remainder of the projects reviewed in 2019 and 2020 will be completed in the coming years. In the fall of 2019, the Panel began a new process of submitting IRP1 feedback letters directly to the Vice Chief of the Defence Staff, producing nine of these letters by the end of 2020.

Figure 2 presents the Panel’s 2019 and 2020 review activities compared with previous years. As the special reviews of CCG projects followed a modified review process, they are not reflected here. The Panel’s review agenda is scheduled in close consultation with DND/CAF, to ensure projects are reviewed in a timely manner. While Panel activity levels vary from year to year, the Panel consistently keeps pace with DND/CAF workflows.

While not captured in the formal engagement numbers below, the Panel also received at least one informational briefing per month from senior DND/CAF stakeholders. These sessions were vital in ensuring the Panel maintained overall strategic awareness, and that the Panel’s focus and approach remained aligned with Departmental priorities.

Acknowledging that project cost estimates and funding levels inevitably fluctuate over time, the approved funding value of projects upon which the Panel provided advice in this period totaled more than $25B. Moreover, Figure 3 illustrates that most projects reviewed by the Panel in 2019-2020 were in the $100-$250M range.

Figure 2 – Panel Activities Associated with DND/CAF Project Reviews since 2015

Figure 3 - Projects Reviewed by Funding Range for 2019-2020

In 2019-2020, the Panel continued to review major procurement projects from a variety of organizations within the Department – from transport aircraft for the Royal Canadian Air Force (RCAF) to secure radios for ADM (Information Management). The variety of project sponsors is reflected below in Figure 4, and illustrates the breadth of the Panel’s project review experience in 2019-2020.

The Panel has maintained a consistent pace of work since its inception, and occasionally managed notable increases, without significant growth in staff or budget. Financial information on the Panel and its supporting office (IRPDAO) is provided at Annex D.

In March 2020, the Panel’s physical office space was relocated from downtown Ottawa to the new National Defence Carling Campus. While the Panel, as an independent body, retains its own distinct office suite, this new location within the National Defence complex offers the Panel improved access to key departmental stakeholders.

Figure 4 – Projects Reviewed by Sponsor for 2019-2020

Panel Performance and Impact

The Panel is an independent third-party review body that provides advice directly to the Minister of National Defence. Nevertheless, Panel members recognize the importance of having productive relationships with DND/CAF stakeholders in carrying out their mandate. With that in mind, the Panel has conducted periodic surveys to obtain valuable feedback from DND/CAF stakeholders. The Panel conducted two stakeholder surveys during this reporting period, seeking feedback on its mandate, review methodology, engagement process, and advice. The Panel expanded the target audience for its most recent survey, undertaken in December 2020, to include relevant officials at the Treasury Board Secretariat and the Canadian Coast Guard.

In both surveys, a majority of respondents indicated that the Panel positively impacts project evolution, and that its advice is credible, relevant, and is being consulted by senior leadership, leading to increased confidence in requirements and the overall defence procurement process. Details on the 2020 survey results can be found at Annex B.

While the survey feedback offered renewed validation of its role, the Panel nevertheless identified areas for improvement in its review process and methodology. For example, following the 2019 survey, the Panel increased the clarity and focus of its review process, as reflected in the “Key Questions” document, which is now provided to all project teams in advance of project reviews (Annex C). This has proven to be a critical communication tool in the Panel’s efforts to ensure broad understanding of its engagement approach. The Panel has also clarified and elevated its feedback after IRP1 by providing a letter directly to the Vice Chief of the Defence Staff, as the Chair of the Defence Capabilities Board, with key strategic observations to inform DND/CAF’s work in Options Analysis.

In the year ahead, the Panel will improve its feedback to project teams after IRP2 reviews, by developing a new feedback mechanism that is separate and distinct from Ministerial advice, with the aim of offering additional supporting observations to officials as they prepare project submissions. In its review process, the Panel will also continue to encourage a more holistic perspective of project requirements, by enhancing its line of questioning around integration and interoperability requirements, as well as key enabling capabilities. Furthermore, the Panel will explore the possibility of developing a more tailored approach for infrastructure projects.

Strategic Observations

Drawing on the experiences of this reporting period, the Panel has identified a series of strategic observations for DND/CAF’s consideration. These observations build on those identified in previous progress reports, draw on the lessons learned from reviewing 70 projects over the past five years, and reflect the unique lens on defence procurement offered by the Panel’s review process and methodology. Overall, the Panel has witnessed continued improvement in DND/CAF’s approach to project development, and acknowledges ongoing efforts to ensure rigour and discipline in internal project governance.

In addition, the Panel offers the following observations:

Strategic Context

The Panel observes that, within the context of SSE, the policy foundation for most projects remains strong. The Panel also notes that the scope, focus, and relative priority of SSE projects are evolving, while new priorities and requirements are emerging based on rapid technological advancements and a changing global security environment. The Panel further notes the impact of horizontal procurement priorities on major defence procurement projects, including Gender-Based Analysis Plus (GBA+) and Greening Government.

In this dynamic environment, the Panel recognizes the need for a comprehensive and robust method for prioritizing projects. The Panel observes that an increasing number of new projects were not foreseen in SSE and therefore do not have identified funding. In that context, the Panel would emphasize the importance of anchoring project priorities in the Capability Based Planning process.

Capability Gap

The Panel observes that projects are effectively articulating the capability gaps and deficiencies facing the Canadian Armed Forces to substantiate major capital investments in equipment and infrastructure.

The Panel continues to encourage DND/CAF to expand its use of operational vignettes and historical data to illustrate the operational impact of capability deficiencies, in both qualitative and quantitative terms. In particular, the Panel would emphasize the need to further develop methodologies that are used to determine quantitative requirements, such as fleet sizes. Moreover, in light of the strategic context noted above, the Panel encourages DND/CAF to provide sufficient detail and greater fidelity about capability gaps to better inform critical discussions on project prioritization and trade-offs.

High Level Mandatory Requirements (HLMRs)

HLMRs define the scope and measures of success for a project, and reflect a binding agreement between DND/CAF approval authorities, such as the Defence Capabilities Board, and project teams. HLMRs are at the core of the IRPDA review mandate and the Panel remains seized with ensuring that they are used consistently and appropriately. The Panel welcomed the new guidance on HLMRs from the VCDS in 2019, and is satisfied that this guidance offers appropriate direction on the development of HLMRs.

Over the reporting period, however, the Panel continued to observe an inconsistent application of this guidance by project teams. Of note, the Panel identified HLMRs as a main issue of concern for nearly half of all the projects reviewed in 2019 and 2020. Therefore, the Panel will continue to work with internal stakeholders to help foster a common understanding of HLMRs as a critical decision-making and accountability mechanism and a foundational cornerstone for any project.

Options

The Panel welcomes the continuing trend towards the identification of options that reflect meaningful capability solutions rather than simply procurement approaches, to better inform decision-making.

In 2019 and 2020, the Panel reviewed numerous projects with insufficient costing information to effectively support the Options Analysis phase. The Panel requires a clear understanding of the capability requirements for projects that align with realistic cost estimates and funding levels. Recognizing that this is challenging given the level of uncertainty that exists early in the procurement process, the Panel would nevertheless encourage the Department to explore ways to improve project costing in the early phases of defence procurement. An earlier and more realistic costing effort would help avoid the impression that project costs have increased significantly when, in fact, the initial calculations underestimated the actual cost. This would also help ensure the Options Analysis phase is better-informed and avoid negative downstream impacts on the scope and schedule of a project.

The Panel applauds the growing trend to include ‘capability ladders’ in project documentation, as an important tool to support decision-makers in considering project flexibility and scalability in the case of funding constraints. Capability ladders represent an informed approach to prioritizing the most critical project elements and identifying those that could be reduced or removed if needed.

Agile Procurement

Given the incredible pace of technological change, the growing interconnectivity between military capabilities, the rapidly evolving threat environment, and the changing business practices under way in the private sector, the Panel observes that a more flexible and agile approach is needed in defence procurement. In the view of the Panel, the concept of ‘agility’ is about pursuing a more iterative approach to capability acquisition, such that CAF capabilities remain current, through continuous growth and adaptation. In practice, agility in defence procurement demands innovative approaches to many aspects of the procurement process: in programming and governance, project funding, and contract mechanisms.

Over the last two years, the Panel saw several projects struggle with the lack of agility in the procurement process, particularly in projects aimed at delivering information and communications technologies. The Panel also witnessed challenges associated with the need to synchronize and integrate complex capabilities across a variety of concurrent projects. Consequently, the Panel has witnessed growing interest within DND/CAF to explore more innovative and flexible approaches going forward. The Panel supports further exploration and experimentation in this area.

Looking Ahead: Priorities for 2021-2022

In line with its Mandate and Terms of Reference, the IRPDA will continue to uphold its fundamental and enduring priorities: maintaining Panel independence and credibility; supporting DND/CAF through the early phases of defence procurement; informing senior decision-makers, most notably, the Minister of National Defence; and ultimately, enhancing trust, transparency and confidence in defence procurement. Building on these broad priorities, in the coming years the Panel will continue to refine its own review process and methodology. Concurrently, the Panel will encourage improved outcomes for DND/CAF, by fostering a more consistent approach to the development of requirements, calling for a more disciplined analysis of cost-capability trade-offs early in the procurement process, and encouraging more flexible and agile procurement mechanisms.

The Panel will also continue to emphasize the importance of building trust, transparency and confidence in defence procurement: across government, with defence industry, and with the Canadian public. The Panel acknowledges that DND/CAF has undertaken significant efforts in recent years to publish more information on defence procurement, in particular with respect to the Defence Investment Plan and the Defence Capabilities Blueprint. The Panel also welcomes recent efforts to share its advice with the Treasury Board, and would continue to encourage exploring further opportunities to share Panel advice, where and when appropriate. For its part, the Panel is releasing the list of projects that it has reviewed (Annex A), as well as its methodology (Annex C), for the first time in this report. Moreover, in the year ahead, the Panel will pursue a deliberate outreach and communications strategy, recognizing the current limitations of the COVID environment, with the intent to promote greater trust and transparency in Canadian defence procurement.

Over the coming period, the Panel looks forward to undertaking a more in-depth assessment of its impact on defence procurement. The Panel has now been in place for over five years, and as stated at the outset, it has initiated the review of 70 major procurement projects and delivered independent advice on 47. While stakeholder feedback and anecdotal evidence indicate that the Panel’s work is positively influencing the projects that fall within its mandate and is informing senior decision-making, a more comprehensive analysis of the Panel’s impact on the procurement process has not yet been feasible. In the coming period, the first projects on which the Panel provided advice will begin to enter the final phases of the procurement process. This will present a timely opportunity to begin evaluating the practical impact the Panel has had on defence procurement outcomes.

In closing, the IRPDA is proud of the significant work that it has undertaken in 2019 and 2020, as captured in this progress report. The Panel remains deeply grateful for the meaningful collaboration that it continues to enjoy with stakeholders. The value of these relationships has never been more obvious than in the past year, as the Panel has relied heavily on DND/CAF, including key base personnel as well as its own office staff, to ensure business continuity despite the significant working restrictions during the pandemic. As we pivot to the future, the Panel looks forward to continuing its important review work, with the persistent aim of having a positive, tangible and lasting impact on defence procurement and ultimately on DND/CAF’s ability to deliver on the critical defence mandate.

Annex A: List of Projects Reviewed by the IRPDA

The list below shows all projects that have initiated a review process with the Panel, as of the end of 2020. Projects that have completed the review process and upon which the Panel has provided advice to the Minister are marked with an asterisk (*). 

Please note that the issuance of Panel advice does not necessarily indicate that a government decision on the project has been made.

Project Name Sponsor
2018 Canadian Coast Guard Fleet Renewal Plan* CCG
Accommodate 4 Engineer Support Regiment (4ESR)* ADM(IE)
Accommodate 436 Squadron (Hangar 5) ADM(IE)
Advanced Improvised Explosive Device Detection and Defeat CA
AFEC Construct Main Facility (2 Wing Bagotville)* ADM(IE)
Airlift Capability Project - Multi-Role Flight Services* RCAF
Armoured Combat Support Vehicle* CA
Automatic Inventory Technology* ADM(Mat)
Bridge & Gap Crossing Modernization* CA
C6 General Purpose Machine Gun Modernization* CA
Camp Sustain CA
Canadian Joint Incident Response Unit (CJIRU) Infrastructure* CANSOFCOM
Canadian Modular Assault Rifle CA
Canadian Multi-Mission Aircraft RCAF
Canadian Special Operations Training Centre (CSOTC)* CANSOFCOM
Canadian Surface Combatant* RCN
CC-130J Block 8.1 Upgrade RCAF
Combined Joint Intelligence Modernization CA
Common Heavy Equipment Replacement* CA
Cormorant Mid-Life Upgrade* RCAF
Cyber Defence - Decision Analysis and Response* ADM(IM)
Cyber Security Awareness ADM(IM)
Data-Centric Security Service ADM(IM)
Defence Enhanced Surveillance from Space RCAF
Defence Resource Management Information System Modernization ADM(DIA)
Defensive Cyber Operations - Decision Support ADM(IM)
Domestic and Arctic Mobility Enhancement Project CA
Electronic Health Record Platform CMP
Enhanced Recovery Capability* CA
Enhanced Satellite Communications Project - Polar RCAF
Future Aircrew Training RCAF
Future Fighter Capability* RCAF
Future Fighter Lead-In Training RCAF
Griffon Limited Life Extension* RCAF
Ground Based Air Defence* CA
Halifax Heating and Municipal Services Upgrade Project* ADM(IE)
Hornet Extension Project - Phase 1* RCAF
Hornet Extension Project - Phase 1 - Scope Change* RCAF
Hornet Extension Project - Phase 2* RCAF
Information Technology Infrastructure in Support of C2* ADM(IM)
Joint Deployable Headquarters and Signal Regiment Modernization Project CA
Joint Fires Modernization* CA
Junior Non-Commission Member Training Accommodation Facility, CFB Esquimalt ADM(IM)
Land Command Support System Intelligence, Surveillance and Reconnaissance Modernization CA
Land Command Support System Tactical Command and Control Information System Modernization CA
Land Command Support System Tactical Communications Modernization CA
Land Vehicle Crew Training System* CA
Light Armoured Vehicle - Specialist Variant Enhancement* CA
Light Utility Vehicle CA
Lightweight Torpedo Upgrade* RCN
Logistics Vehicle Modernization* CA
Manned Airborne ISR* CANSOFCOM
Multi Fleet Air Traffic Management Avionics* RCAF
Multi-Purpose Vessel CCG
National Defence Operational Headquarters ADM(IE)
Naval Large Tug* RCN
Night Vision System Modernization* CA
Polar Icebreaker* CCG
RCN Intelligence, Surveillance, Target Acquisition, and Reconnaissance* RCN
Remotely Piloted Aircraft System (formerly JUSTAS)* RCAF
Secure Radio Modernization ADM(IM)
SOFCOM Capabilities and Recapitalization Project - Next Generation Fighting Vehicle* CANSOFCOM
Strategic Tanker Transport Capability* RCAF
STRONGBOW* RCN
Surveillance of Space 2* RCAF
Tactical Integrated Command Control and Communications* RCAF
Tactical Narrowband Satellite Communications - Geosynchronous Coverage* RCAF
Tactical Power System CA
Victoria Class Modernization - Periscope and Flank Array* RCN
Weapon Effects Simulation Mid Life Upgrade CA

Annex B: Survey 2020 Results

The Independent Review Panel for Defence Acquisition (IRPDA) survey ran from the 30th of November through to the 18th of December, 2020. This survey was designed and implemented by the Independent Review Panel for Defence Acquisition Office (IRPDAO). The survey’s intended audience included Directors General and working-level staff who had participated in a Panel engagement as either a project sponsor or a project implementer. The IRPDAO received 42 completed and partially-completed surveys. The survey consisted of 28 multiple choice questions and four open-ended questions organized in four broad sections: mandate, engagement process (inclusive of engagement preparation and engagements), feedback and advice, and impact. 

In addition to the survey circulated to the Directors General and working-level staff, the IRPDAO circulated questionnaires to senior stakeholders and decision-makers at the Treasury Board Secretariat, the Canadian Coast Guard, and the Department of National Defence and the Canadian Armed Forces. These questionnaires consisted of between five and eight questions generally following the same broad sections as the survey. Participants were invited to provide long-form feedback to each of the questions, either verbally or in writing. Responses were collected directly by the IRPDAO Executive Director.

Below is a brief summary of the survey and senior stakeholder feedback organized by topic section. This is followed by the results of the survey’s multiple-choice questions.

Mandate

Responses from the multiple-choice questions indicate that the Panel’s mandate is clear, well understood, and that the Panel operates in an independent manner. Respondents were less certain (by about 30%) that the Panel continued to operate within its mandate.

Responses from the senior-level questionnaires also indicate that the Panel is mostly viewed as working within a clear mandate. Senior-level stakeholders had varying perspectives on whether the Panel mandate ought to be expanded or narrowed, and whether the Panel should engage on projects below the $100M limit or on issues beyond capital projects.

Engagement Process

Responses from the multiple-choice questions indicate that the Panel engagements are largely seen to be useful exercises that occur at the right moment in the project approval process. Additionally, project teams were generally confident that they knew what to expect from the engagement. 

Responses from the senior-level questionnaire also indicate that the Panel is predominantly viewed as having an effective engagement process that was seen as beneficial for the project and the project team. Senior-level stakeholders also agreed that IRP1 and IRP2 occurred at the right moment in the process, but noted the challenges of incorporating Panel advice following IRP2 due to the procurement process.

Feedback and Advice

Responses from the multiple-choice questions indicate that a majority of participants believe that both the IRP1 feedback email and the IRP1 VCDS letter were clear, useful, and consistent with project management guidance. Additionally, project staff assess that they incorporate over 75% of the Panel’s IRP1 feedback into subsequent project documentation. In contrast, the majority of project staff are uncertain whether the Panel’s IRP2 feedback is incorporated into subsequent decision documents, given their lack of visibility on advice. Notably, over 80% of participants indicated that it would not be helpful to receive additional feedback from the Panel. 

Responses from the senior-level questionnaire also indicate that the Panel is generally viewed as providing credible and effective advice. Senior-level stakeholders agreed that the Panel’s advice was a critical tool in communicating a project’s viability to central agencies and ministers. Additionally, senior-level stakeholders universally agreed that the Panel increases their confidence in project requirements. Senior-level stakeholders for the most part agreed that project teams incorporate Panel advice in subsequent project documents.

Impact

Responses from the multiple-choice questions indicate a more mixed understanding of the impact of the Panel on the project. For the most part, project staff agreed that the Panel helped improve the project documentation, clarify the project options, and had an overall positive impact on project evolution. However, project staff responded positively and negatively in equal measure when asked if the Panel helped clarify the policy cover, capability gap, and HLMRs of the project.

Responses from the senior-level questionnaire also indicate that the Panel is almost universally viewed as having a positive impact on projects. Senior-level stakeholders agreed that the Panel’s third-party challenge function significantly improves the quality of the project generally, and specifically improves the project requirements. In particular, both the Materiel Group and the Information Management Group use the advice to inform the development of the project in Identification and Implementation.

Multiple Choice Survey Results

- - 1 to 2 3 to 4 5 to 6 More N/A
Q1. How many Panel engagements have you participated in (including IRP1, IRP2 and supplementary engagements)? 32% 21% 11% 18% 18%
- - Strongly Agree Agree Disagree Strongly Disagree N/A
Q2. I am familiar with the mandate of the Independent Review Panel for Defence Acquisition (IRPDA). 50% 45% 3% 3% 0%
Q3. The IRPDA mandate is clear. 42% 50% 5% 0% 3%
Q4. The IRPDA operates within its mandate. 11% 53% 26% 3% 13%
Q5. The IRPDA acts in an independent manner. 39% 45% 8% 3% 5%
- - Yes No
Q6. Was the project team invited to attend a preparatory meeting with the IRPDA office staff in advance of the Panel engagement(s)? 85% 15%
- - Strongly Agree Agree Disagree Strongly Disagree N/A
Q7. The preparatory meeting with the IRPDA was helpful in preparing for the Panel engagement(s). 38% 52% 3% 3% 3%
Q8. The IRPDA office staff gave me sufficient information to prepare for the Panel engagement. 38% 52% 7% 0% 3%
Q9. I knew what to expect from the Panel engagement(s). 14% 58% 11% 3% 14%
Q10. The Panel engagement(s) were useful. 3% 61% 19% 3% 14%
Q11. The Panel engagement(s) occurred at the right time in the project approval process. 6% 50% 17% 8% 19%
Q12. The IRP1 Panel feedback email was clear. 21% 59% 0% 3% 18%
Q13. The IPR1 feedback email was useful. 18% 47% 15% 3% 18%
Q14. The IPR1 feedback email was consistent with internal project management guidance. 12% 47% 21% 3% 18%
- - Yes No N/A
Q15. Was the Panel’s IRP1 VCDS Letter shared with you after the engagement(s)? 41% 41% 18%
- - Strongly Agree Agree Disagree Strongly Disagree N/A
Q16. The Panel’s IRP1 VCDS Letter was clear. 50% 50% 0% 0% 0%
Q17. The Panel’s IRP1 VCDS Letter was useful. 29% 64% 7% 0% 0%
Q18. The Panel’s IRP1 VCDS Letter was consistent with internal project management guidance. 29% 64% 7% 0% 0%
- - Yes No
Q19. Would you find it useful to receive additional feedback from the Panel? 18% 82%
- - Less than 25% 25-50% 51-75% 76-100% N/A
Q20. In general terms, what proportion of the Panel's IRP1 feedback was incorporated in subsequent project documentation, such as the Business Case Analysis and the Preliminary Statement of Operational Requirements? 6% 15% 6% 53% 21%
Q21. In general terms, what proportion of the Panel's IRP2 feedback was incorporated in subsequent project documentation and decision documents, such as Ministerial Submissions or Treasury Board submissions? 9% 15% 18% 18% 41%
- - Strongly Agree Agree Disagree Strongly Disagree N/A
Q22. The IRPDA engagement(s) helped improve the project documentation. 19% 53% 16% 3% 9%
Q23. The IRPDA engagement(s) helped clarify the policy cover of the project. 6% 31% 34% 13% 16%
Q24. The IRPDA engagement(s) helped clarify the capability gap of the project. 9% 34% 44% 3% 9%
Q25. The IRPDA engagement(s) helped clarify the High-Level Mandatory Requirements (HLMRs) of the project. 0% 47% 34% 9% 9%
Q26. The IRPDA engagement(s) helped clarify the options of the project. 3% 53% 22% 6% 16%
Q27. The IRPDA engagement(s) positively impacted the evolution of the project. 3% 63% 19% 3% 13%
Q28. The IRPDA engaged in a timely and efficient way, such that the project was not unduly delayed. 9% 41% 22% 16% 13%

Annex C: IRPDA Review Methodology

The below grid reflects the Panel’s general approach to project reviews, in accordance with the areas of focus that were set out in the Panel’s Terms of Reference and with a clear distinction between IRP1 and IRP2 engagements. This is an evergreen document, and is updated periodically to reflect evolving project considerations and priorities. The below is an accurate reflection of the review methodology during the reporting period.

IRPDA Key Questions

- Strategic Context Capability Gap Requirements Options Procurement
IRP1

What are the relevant strategic drivers (or constraints) for this project?

  • The international and/or domestic security environment?
  • Adversary capabilities?
  • Technology?
  • Regulations?
  • Allied agreements and/or expectations?

What is the Government policy?

  • Is the project aligned with SSE? Is it funded?
  • Is it aligned with other relevant government policies?
  • Is additional policy guidance required?

What are the main risks identified for this project (schedule, funding, etc.)?

 What capability gap is being addressed?
  • Has this capability requirement been identified in capability-based planning?
  • What is the concept of operations for this capability? Is it well-illustrated (e.g. operational vignettes, etc.)?
  • How does it fit with other DND/CAF capabilities (i.e. linkages / dependencies)?
  • How is DND/CAF impeded without this capability (i.e. what is the risk of the status quo)?
  • What is the scope and scale of the capability gap?
  • How does this capability fit with partner capabilities (i.e. other government departments, allies)?
 What are the High Level Mandatory Requirements?
  • What is the evidence to support the HLMRs?
  • Do they flow logically from the strategic context and capability gap, and align with the business outcomes?
  • Do the HLMRs capture the full intent of the project?
  • Do they reflect the scope and scale ofthe required capability?
  • Do the HLMRs fulfill the six criteria (clear, essential, measurable, sufficient, comprehensive, results oriented)?
  • Have requirements regarding training, infrastructure and growth potential been captured? If not, why?

What are the potential options?

  • Are the options capability-based? Do they offer a range of potential capability levels (i.e. a capability ladder)?
  • Does the CAF need to own the capability? Are there other options? Partnership possibilities?
  • What are the criteria that will be used to assess the options? How were the criteria developed?

What are the procurement considerations?

  • How many potential suppliers are anticipated?
  • Is the project expected to leverage proven technology, or require developmental work?
IRP2
  • Has the strategic context, as noted above, changed since IRP1?
  • Has the capability gap evolved, become more/less acute, or been better defined since IRP1?
  • Have there been any recent changes to other DND/CAF and/or partner capabilities, thereby affecting the capability gap to be addressed in this project?
  • Have there been any changes to the HLMRs?
  • Has gender-based analysis informed the requirements?
  • Which HLMR(s) present the greatest challenge/risk?

What is in the Preliminary Statement of Requirements?

  • Is the PSOR traceable to the HLMRs?
  • Are the mandatory requirements sufficient to meet the HLMRs?
  • Were there any changes to the design of the options, or the selection criteria?
  • Do certain requirements constrain the options available (e.g. interoperability)?

What is the recommended option?

  • What methodology was used to select the preferred option?
  • Is it affordable? If not, what is the prioritization methodology?
  • What are the key risks associated with this option? Cost? Schedule? Residual gap?
  • How have potential suppliers been consulted on the HLMRs?
  • What concerns have potential suppliers expressed?
  • To what extent will the HLMRs limit competition?
  • Is there a potential role for Canadian industry?
  • Is the proposed schedule feasible? Risk of delays?
  • Which requirements are expected to be the key cost drivers?

Annex D: IRPDA and IRPDA Office Budget and Total Expenditures for 2018-20

- FY 2018-2019
Budget		 FY 2018-2019
Expenditures		 FY 2019-2020
Budget		 FY 2019-2020
Expenditures
Operating and Maintenance $200,000.00 $81,511.83Footnote * $200,000.00 $114,393.66
Salary (Panel members and IRPDAO staff) $1,257,000.00 $1,231,850.33 1,407,040.00 $1,320,044.15

Annex E: Panel Member Biographies

Chair

Mr. Larry Murray CM, CMM, CD

Mr. Larry Murray has held a number of senior positions in the Canadian Armed Forces and Public Service. During his career with the Canadian Armed Forces, he served at sea in a variety of ships and held several senior positions, including as Deputy Chief of the Defence Staff, Commander of Maritime Command, Vice Chief of the Defence Staff and, finally, Acting Chief of the Defence Staff from October 1996 until September 1997. Mr. Murray retired from the Canadian Armed Forces in 1997 and joined the Public Service as Associate Deputy Minister of Fisheries and Oceans. He was appointed Deputy Minister of Veterans Affairs Canada in 1999 and Deputy Minister of Fisheries and Oceans in 2003. He retired from the Public Service of Canada in 2007.

Since retiring from the Public Service, Mr. Murray has served on Task Forces, Advisory and Audit Committees. He is also a former Chair of the Board of the Public Policy Forum, a former President of the Nova Scotia Mainland Division of the Navy League of Canada and honourary Grand President of the Royal Canadian Legion.

Mr. Murray has received many prestigious awards in recognition of his leadership in the Canadian Armed Forces, Public Service of Canada, national voluntary commitments, as well as his support to Canadian Armed Forces personnel, Veterans and their families. He was appointed as a Member of the Order of Canada in 2013.

Members

Mr. Martin Gagné

Mr. Martin Gagné spent 17 years at CAE before retiring as Group President for Military Simulation and Training in 2012. During his career with CAE, he served in various roles such as: Vice-President of Visual Systems, Vice-President of Military Marketing and Sales, and Executive Vice-President of Civil Simulation and Training.

Prior to joining CAE, Mr. Gagné acquired extensive management and leadership experience during his 23 years as a member of the Canadian Armed Forces. As a senior aerospace engineering officer he was involved in the acquisition and maintenance activities of various aircraft fleets including the CF-18 and Maritime Helicopter Project. He holds degrees in electrical engineering and computer engineering as well as a certification in business administration from the McGill Executive Institute.

Mr. Gagné served several terms on the Board of Directors for the Canadian Association of Defence and Security Industries (CADSI) and is on the Canadian Commercial Corporation’s (CCC) Board of Directors as the Chair of the Operations Committee.

Mr. Philippe Lagassé

Mr. Philippe Lagassé is associate professor and the William and Jeanie Barton Chair at the Norman Paterson School of International Affairs, Carleton University. He holds degrees from McGill University, Royal Military College of Canada, and Carleton University. He taught previously at the Graduate School of Public and International Affairs, University of Ottawa.

Mr. Lagassé’s research focuses on defence policy and military procurement, as well as legislative oversight of armed forces and executive power in the Westminster tradition. His past and current research projects have been funded by the Social Science and Humanities Research Council of Canada, and his work has appeared in leading Canadian and international scholarly journals. He was also the co-editor of the 2020 book, Canadian Defence Policy in Theory and Practice.

His public sector experience includes consultancies with the Office of the Judge Advocate General and the Office of the Auditor General, and he was an independent reviewer of the 2012-2014 evaluation of options to replace Canada’s CF-18 fighter aircraft.

Ms. Margaret Purdy

Ms. Margaret Purdy had a 30-year career as a national security professional in the federal public service. Her assignments included Director General of Counter Terrorism at the Canadian Security Intelligence Service, Deputy Secretary to the Cabinet (Security and Intelligence) in the Privy Council Office, and Associate Deputy Minister of National Defence.

Ms. Purdy was one of four members of the independent panel advising the Minister of National Defence during the development of Strong, Secure, Engaged – Canada’s defence policy. She also served six years as Chair of the Departmental Audit Committee for the Canadian Security Intelligence Service, as well as terms as a member of the Department of National Defence and Communications Security Establishment Departmental Audit Committees.

Since retiring in 2006, Ms. Purdy has conducted compliance examinations, lessons learned reviews, post-incident inquiries, tabletop exercises, and strategic policy reviews for more than a dozen Government of Canada departments, agencies and regulatory bodies.

Ms. Christine Tovee

Ms. Christine Tovee has over 15 years of engineering leadership in aerospace development programs spanning early concept and requirements through detailed design, test, and validation. She then returned to research and development culminating in being promoted to Vice President of Research and Technology and Chief Technology Officer for Airbus Group Inc., in the United States (formerly known as EADS North America). Recently, she has taken the position of CTO at a Canadian space startup aiming to deliver hyperspectral imagery to new vertical markets.

At BAE Systems and EADS, Ms. Tovee held positions in Germany, France, and the United Kingdom, contributing to national and European space programs. She has led and collaborated on defence projects in all domains: land, air, sea and space, focusing on joint operations, C4ISTAR systems and military satellite communications.

Ms. Tovee was seconded to the UK Ministry of Defence to lead the technical aspects of the Joint Network Integration Body (JNIB). This programme combined the efforts of MoD and multiple defence contractors to identify and solve the integration challenges in providing a seamless information and communications system.

She is currently a member of the Government of Canada’s Space Advisory Board. She has also been appointed as a Fellow at the Creative Destruction Lab at the Rotman School of Business in Toronto.

Ms. Tovee holds a Bachelor of Applied Science in Aerospace Engineering from the University of Toronto, and a Master of Science in Aeronautics and Astronautics from Massachusetts Institute of Technology (MIT).

Annex F: IRPDAO Contact Details

Independent Review Panel for Defence Acquisition Office

Mailing address:

National Defence Headquarters – Pearkes BuildingFootnote *
101 Colonel By Drive
Ottawa, Ontario
K1A 0K2

Email
Website

Page details

Date modified: