Evaluation of the IDEaS Program

March 2023

1258-3-062 ADM(RS)

Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

Acronyms

Back to Table of Contents

Executive Summary

Evaluation Scope

The evaluation of the Innovation for Defence Excellence and Security (IDEaS) Program focused on the relevancy of the program, the program's ability to engage innovators, benefits of program outcomes to the defence and security community, and the efficiency of the program for the period between Fiscal Years (FY) 2017/18 and 2021/22. The evaluation also considered the extent to which the program has achieved expected results, in line with its short-term and medium-term objectives. Over the short-term, the program aimed to engage innovators and solicit concepts and initiatives to solve Department of National Defence/Canadian Armed Forces (DND/CAF) issued challenges (i.e., competitions in need of solutions to specific defence and security issues). In the medium-term, IDEaS aimed to engage Canadian innovators to solve defence and security challenges. The evaluation is in accordance with the DND/CAF Five-Year Departmental Evaluation Plan FY 2017/18 to FY 2021/22. 

Program Overview​

IDEaS was launched in 2018 consistent with the direction given through initiative 105 of Canada's defence policy: Strong, Secure, Engaged (SSE). IDEaS is based on three pillars: 1) to build an innovation community by bringing industry, researchers and academics together; 2) to find solutions through the open competition of ideas with the goal of finding multiple solutions to challenges faced by DND/CAF; and 3) to push technology forward through validating innovative solutions and advancing the most promising technologies along the innovation continuum. In 2018, the program received a funding commitment of $1.6 billion over the next 20 years, ultimately in support of the Department’s Science, Technology and Innovation Program. As of August 2022, the program has invested over $314.2 million.

Results 

The findings show that the IDEaS Program remains relevant in the evolving global defence and security environment, and has effectively engaged Small or Medium-sized Enterprises (SME) and academia, further cultivating those networks to strengthen the innovation ecosystem within Canada. The program has effectively facilitated the advancement of a range of solutions to address defence and security challenges, and has had varying levels of success in delivering its allocated funding throughout the evaluation period. Challenges were identified within the IDEaS Program, notably the need for better strategic direction, as well as a need to improve internal processes, governance of the program, and use of the Grants and Contributions (Gs&Cs) portions of program funding. Levels of engagement between innovators, clients and the program show room for improvement.  

Overall Conclusions

Overall, the program received positive feedback from all program stakeholders including program personnel, DND/CAF members, and innovators. IDEaS remains relevant in the context of SSE and in the face of the evolving defence and security environment and is currently funded until 2037 to continue to deliver on its mandate. IDEaS has been effective in cultivating innovation within Canada, specifically with SMEs and academia. Through these relationships, solutions have been curated to address a wide range of challenges. Nonetheless, some areas within IDEaS have been identified requiring improvement.  

Recommendations

1. Strengthen the strategic direction and governance of the IDEaS Program (particularly related to the generation of challenges) to ensure alignment with departmental priorities and enhance internal processes. 
2. To ensure the successful continuity of partnerships, increase engagement throughout the program processes. 
3. Explore the benefits of increasing the utilization of Transfer Payment Program (TPP) (Vote 10) funds.

Back to Table of Contents

Program Context

In accordance with initiative 105 of SSE, the IDEaS Program was launched in 2018 with a commitment of $1.6 billion until 2037. The IDEaS Program is a sub-component of the larger Defence Capability Development & Research activity that aims to develop and synthesize new knowledge and integrate advancements in Science and Technology (S&T) to provide a competitive advantage in military operations and enhance the effectiveness of other defence programs.^{Footnote 1} Underpinned by its three pillars to: Build an Innovation Community; Find Solutions; and Push technology forward, the program was developed to provide mechanisms to allow the government to better tap into technology development and innovation. IDEaS further supports the Departmental outcomes of “a robust Canadian innovation base that can support Canada’s defence and security is established and enabled”^{Footnote 2} and “current and future military capabilities and related areas of scientific investigation are established and fulfilled.”^{Footnote 3}

As of August 2022, the program has issued 70 challenges, and invested over $314.2 million into 624 projects across its five funding elements: Competitive Projects; Innovation Networks; Contests; Sandboxes; and Test Drives.^{Footnote 4} Funding recipients have spanned across Canada with 284 unique recipients being represented (either as part of an SME, large industry, academia, not-for-profit or even as an individual). 

Program Mandate: 

The overall objective of IDEaS is to shape, shift and promote innovation from Canadian innovators to benefit defence and related security challenges posed by DND/CAF. IDEaS aims to support creating a vibrant innovation community, open competition of ideas and validated innovative solutions.

* Vote 1 Expenditures exclude $22.74M spent on Test Drives.
** Vote 1 Expenditure data for FY 2022/23 is as of Oct 31, 2022.

Back to Table of Contents

Evaluation Scope

Coverage and Responsibilities

The evaluation of the IDEaS Program considered the extent to which the program has achieved expected results in line with its short-term and medium-term objectives. Over the short-term, the program aimed to engage innovators and solicit concepts and initiatives to solve DND/CAF issued challenges. In the medium-term, IDEaS aimed to engage Canadian innovators to solve defence and security challenges. The evaluation was structured around four main evaluation questions:

The IDEaS evaluation was undertaken in alignment with the Treasury Board of Canada (TB) Policy on Results, which requires Gs&Cs programs of $5 million or more per year, to be evaluated every five years. This is the first time this program has been evaluated; although in 2015, an evaluation of the Science and Technology Program was conducted. 

Out of Scope

The departmental procurement process was scoped out.

Methodology

The findings and recommendations within this report were informed by multiple lines of evidence, including: 

• Document Review; 
• Review and analysis of program data, including financial data;
• Key Informant interviews (n=6: external innovators engaged with IDEaS, n=9: program analysts);
• International Benchmarking: Czech Republic, France, Italy, Netherlands, New Zealand, Norway, UK and US; and
• Surveys (n=19: internal DND innovation clients, response rate of 54%; n=23: program analysts, excluding executives, response rate of 61%).

Further details on the evaluation methodologies can be found in Annex C of this report, while evaluation limitations are noted in Annex D.

Back to Table of Contents

Key Results

Q1: How can the program be supported to stay relevant?

The IDEaS Program remains relevant in the evolving global defence and security environment.

As outlined in SSE, adapting and innovating quickly are crucial to developing and maintaining capabilities that address the challenges presented by today’s evolving global defence and security environment. A program document on SSE^{Footnote 5} identifies IDEaS as a program that "provides the tools to access this capacity (knowledge, solutions, and technologies) to meet both immediate and future needs of defence and security communities.” In fact, since its inception, IDEaS has had well-balanced social impact targets (e.g., force protection, cyber security, pandemic preparedness) that aligned with SSE.^{Footnote 6}

The overall mission for the IDEaS Program is to help DND/CAF address critical challenges to Canada’s defence and security posture through a demand-side innovation approach that links DND/CAF needs with, and helps grow, Canadian innovation capabilities and opportunities.^{Footnote 7} According to foundational documents by DND/CAF, the program was designed to provide new opportunities to support science, technology and innovation for the defence and security of Canada, and facilitate the integration and adoption of new solutions and capabilities for the CAF and the Public Safety and Security communities. In doing so, it not only transforms the way DND/CAF generates solutions to the new threats it faces, but it can also stay ahead of potential adversaries and meet evolving defence and security needs, while generating economic benefits for Canada.

Challenge selection and prioritization may not reflect the highest priorities of DND/CAF.

Regardless of the intended alignment of IDEaS to departmental priorities, 60% of program analyst survey respondents did not believe that IDEaS has adequate policies in place to support the program. Similarly, results from the client survey showed that DND/CAF stakeholders are also unclear about the strategic alignment of the IDEaS Program to, or its impact on, Government of Canada priorities. Nonetheless, 94% agreed that the program is valuable to DND and to the defence and security community.​

Program analyst survey respondents also identified that there are difficulties with how the challenges are prioritized and governed, given the lead role Defence Research and Development Canada (DRDC) plays in this process. Specifically, the majority (74%) of respondents disagreed that IDEaS has the necessary strategic direction to effectively prioritize submitted challenges. ​

The lack of strategic direction and the program's reliance on DRDC to source and prioritize challenges is a problem that program managers are aware of and currently addressing.​

Concerns were raised about internal processes and lack of governance. 

In interviews with program managers, it was repeatedly raised that the IDEaS Program is not releasing enough challenges, as there is a need for a constant stream of solutions required from DND/CAF stakeholders. They also acknowledged that they do not have enough integrators (i.e., internal DND/CAF subject-matter experts who can help refine the development of solutions) to manage these challenges. 

However, it should be noted that the IDEaS Program is not responsible for developing challenges or sourcing its own integrators. It was acknowledged by program manager interviewees that the greatest obstacle to putting out the most relevant challenges is the reliance on the broader DRDC organization, which is responsible for distributing challenges to IDEaS.

Through interviews it was found that more frequent check-ins and direct communication between DND/CAF innovation clients (L1 level) and innovators would allow both sides to have a better idea of project progression throughout challenges, as well as anticipated next steps. While IDEaS Program managers are aware of these issues, they are careful not to overburden DND/CAF clients with additional meetings; instead, they are seeking ways to optimize governance internally and provide clearer and more consistent messaging to partners.
 

Back to Table of Contents

Q2: Is the IDEaS Program effective at fostering innovative networks and engaging innovators?

The IDEaS Program has effectively engaged SMEs and academia, further cultivating those networks to strengthen the innovation ecosystem within Canada.

The IDEaS Program has been successful in attracting a wide range of innovators to enrich innovation within Canada. In fact, program data show that since the outset of IDEaS, 100% of posted challenges have received proposals. This outpaced the 80% target outlined in the program’s foundational document. Further, benefitting from geographically dispersed innovators has positioned IDEaS to maximize established national and provincial systems of innovation –significant sources of knowledge, expertise, skills, facilities and equipment.^{Footnote 9} 

Program data also show that to date, SMEs and academia have been especially participatory in the funding elements (the various avenues to obtaining funding from the program, which includes Competitive Projects (CP), Innovation Networks, Sandboxes, Contests and Test Drives). In particular, SMEs and academia have been identified as having the highest involvement rate compared with other innovator types (e.g., Large Industry, Individuals and Not-for-profits), as well as those who have been awarded the most funding. The partnerships built during challenges have led all interviewed innovators to indicate that they would apply to the IDEaS Program again if the challenge aligned with their work.  

SMEs have also been awarded a higher funding percentage of second-milestone contracts, furthering the access to the creativity and knowledge of SMEs, and reflecting the effectiveness of the process for these innovators. In fact, 96% of program analyst survey respondents agreed that the IDEaS Program helped foster connections with Canadian industry and academia. 

The program has effectively attracted both new and repeat innovators to submit proposals. In particular, there were 71 unique return innovators to related domain challenges within the CP’s program element during the evaluation period. According to innovator interviewees, the main sources which attracted them to the program were industry meetings, innovator networks, conferences, previous engagement with DND, the program website and emails.
  

Gender-based Analysis Plus (GBA Plus) Considerations:  

The IDEaS Program attracts a wide range of innovators and as part of a government-wide initiative to examine GBA Plus indicators, the diversity of innovators was assessed during the evaluation. One-third of innovator interviewees felt that their companies naturally met the expectations for company diversity due to the types of individuals that work in their fields in terms of gender, neurodiversity and visible minority groups. IDEaS Program data show that there is a wide variety of individuals listed as owners in participating innovator organizations, confirming the interview responses that innovator organizations are diverse in ownership. Specifically, 51% of innovator organizations had some visible minority ownership, followed by 46% that had a portion of youth ownership, and 36% that included female ownership.

The program received positive feedback from innovators, clients and personnel alike. All innovator interviewees appreciated the broadness of the challenges, which allowed for creativity and stated that the challenge statements were clear and understandable. The overall application process was noted as being easy and clear by many innovator interviewees. Similarly, DND/CAF client survey respondents scored the IDEaS Program highly in: 

• Providing value for DND and the broader defence and security community; 
• Positively impacting domestic partnerships; 
• Effectively contributing to innovative designs and concepts; and 
• Transferring knowledge to DND/CAF.  

However, current levels of engagement with both IDEaS Program staff and DND/CAF clients was an area where many innovator interviewees hoped enhancements could be made. The Attracting Innovators consultation^{Footnote 10} produced similar results, with barriers for innovators listed as: the inability of innovators to engage end-users (i.e., DND/CAF members that could potentially utilize the solution in their day-to-day activities) and clients (i.e., those representing an L1 organization who identified/submitted a challenge and could potentially acquire the solution in the future); the lack of personal interaction between innovators and IDEaS Program staff and integrators, and insufficient feedback from integrators to innovators.^{Footnote 11}

Innovator – Program

The relationship between the program and the innovators is significant as it is the principal connection in achieving the intended outcomes of the program. However, half of innovator interviewees desired improvements to this partnership noting that it was is challenged by factors such as: the liaison between the two parties (known as the integrator), clarity on timelines and feedback levels.  

All interviewed innovators had positive experiences working with program integrators and benefited from its collaborative nature. However, access to those individuals was said to be limited with the frequency of engagements being perceived as minimal. Of innovators interviewed, some shared that they would like more frequent and descriptive communication as well as meetings with integrators between formal milestones. 

Clarity on timelines seemed to be the biggest challenge for innovator interviewees. While interviewees agreed that the program is responsive when contacted, it was stated that they would prefer to have clearly communicated timelines and proactive engagement during each stage of the process. Specifically, approximately half of the innovator interviewees felt there was still a level of uncertainty in the timelines for the selection process.  

Feedback given during milestone reviews was said to be helpful for innovators, many of whom indicated that the feedback they received was easy to understand. They mentioned that the scoring system was helpful for understanding which projects were selected and which aspects of the project were important to the client. However, half of innovator interviewees wanted more feedback from program personnel, indicating that a clear reason for not being chosen for follow-on phases would be valuable for future applications. 

Innovator – Client 

All innovator interviewees desired increased engagement with DND/CAF clients (i.e., those organizations that identified/submitted their respective challenges and are invested in the outcome of the challenge to potentially acquire the solution in the future), stating that it would be beneficial to receive feedback directly from clients and expand upon opportunities for improved visibility and exposure. This step was perceived to aid innovators in gaining access to the later stages in a process if they can further align their innovations with DND/CAF expectations. However, in interviews with program managers, they did not view innovator participation in IDEaS as either an advantage or disadvantage to future procurement. The key to fairness is ensuring no kind of information is shared that could preclude innovators from participating in procurement activities outside of IDEaS. 

The newly launched Marketplace^{Footnote 12} event in 2022 was an occasion whereby innovators presented their projects to defence and security stakeholders both in the public and private sectors and had the additional benefit of providing an opportunity for innovators to ask questions and get feedback on their solutions from DND/CAF stakeholders (e.g., both clients and end-users). Only a few innovator interviewees had participated in the Marketplace event, while the others had not heard of it. As this was the first piloted instance of Marketplace, participation and awareness is expected to increase in future years, especially in light of the informal engagement that occurred as a result of this event. 

Client – Program 

Both DND/CAF client and program analyst survey respondents agreed that current engagement levels within DND between the client and the IDEaS Program staff are insufficient, and that engagement throughout the process could be improved. There was further agreement that updates from both innovators and program staff to DND/CAF clients should be increased and consistent. Although 76% of program analyst survey respondents signaled that the IDEaS Program is well-known within DND/CAF, there are still desired improvements. Specifically, 67% of program analyst survey respondents disagreed that current engagement levels between the program and DND/CAF are sufficient, followed by 57% of respondents who disagreed that DND/CAF clients are engaged throughout the process effectively by the program.  

Sustained engagement between the client and the program was said to be challenged for several reasons, including a perceived insufficient number of dedicated integrators associated with the program. Similarly, program manager interviewees noted challenges in sustaining consistent and prolonged client buy-in, particularly in light of lengthy project timelines and cyclical CAF turnover. 

Back to Table of Contents

Q3: To what extent were the identified challenges successful in producing solutions that were of benefit to the defence and security community?

The IDEaS Program has effectively facilitated the advancement of a range of solutions to address defence and security challenges. However, concerns were raised regarding the next steps, including commercialization, for solutions to fully support defence innovation.

Overall, respondents to both the DND/CAF client survey and the program analyst survey are confident in the quality of the challenges to derive the solutions needed. 

Of DND/CAF clients surveyed, 93% agreed that the potential solutions accurately address the identified defence challenges, and 100% agreed that the submitted challenges are, or will be beneficial to DND/CAF. Similarly, the majority of program analyst survey respondents agreed that the innovations produced to date have largely matched the original intent of the client.

In addition to the 100% of posted challenges that received proposals, program data also show that 100% of posted challenges resulted in projects or initiatives to address them. This outpaced the 75% target indicated in the program’s cornerstone document. 

The program data also show that between FY 2018/19 and October 2022, 150 funding awards were granted to organizations who succeeded to the second milestone of a challenge (16 challenges total) within the CP program element, representing 79% of projects that have received funding for a higher level of development since the inception of IDEaS. Innovators and their solutions are moving further into the process, meaning that the solutions are increasing their Technology Readiness Levels (TRL), and the maturity of solutions is increasing. 

There is a lack of clarity about next steps, including commercialization, once a solution is developed.

In interviews with program managers, it was indicated that IDEaS has not yet reached the point where solutions, once developed, can be procured. It was also stated by some interviewees that the IDEaS Program is not funding projects that are market ready outside of Test Drives. As program manager interviewees acknowledged, the acquisition of the solution by DND/CAF was not the initial scope of the program. As such, the most asked question from innovators is “what happens next?” For this, the program has no clear answer. Commercialization of solutions was identified as an international best practice for similar innovation programs within the governments of the USA, UK, Australia and New Zealand. As such, this could be an area that is considered when further developing the IDEaS Program that would enable it to foster marketable innovation and to keep pace with equivalent programs. In fact, IDEaS Program manager interviewees shared that knowing how innovation fits into overall capability development within DND/CAF could benefit the program. As noted in open-ended survey questions, internal DND innovation clients are also interested in the commercialization of solutions. 

Effective performance metrics could further inform the direction of the program.

Of the program analysts surveyed, 60% disagreed that there are effective metrics to track the progress of IDEaS. As noted in open-ended survey questions from program analysts, most metrics are not outcome-based. Without awareness of effective metrics, program personnel cannot confidently speak to the success of the program and, in turn, cannot effectively determine if changes or adjustments are required to reach program goals. 

The program’s Vote 1 funding includes allocations for O&M costs as well as overhead costs such as personnel. As mentioned in interviews with program staff, the program does not have enough overhead funding within its annual Vote 1 allocation to alleviate their actual overhead costs and the pressures this creates in terms of staff capacity and administrative workload.  

The program reduced the number of projects following CFP1 (Call for Proposal – phase one). This was intentional as the number of challenges released and the volume of proposals received were far larger than initially anticipated. Lessons Learned and After Action Reviews completed following CFP1 recommended the total number of contracts awarded in a given fiscal year be reduced in the future to ensure the number of proposals and resulting projects were feasible for the program and DND capacity to support.

Although IDEaS' overhead costs in Table 1 appear to reflect a high measure of program efficiency, inadequate levels of overhead funding appear to be impacting the program’s capacity to continue to maintain the ability to generate and manage the initial volume of challenges and resulting program elements. 

IDEaS has not fully expended Vote 10 funds allocated in the IDEaS’ TPP which includes all Gs&Cs spending.​ According to data reported in the Public Accounts of Canada, the program had spent $31.7M, or 42%, of total allocated Vote 10 funding of $75M (per Main Estimates) as of the end of FY 2021/22.​ Program staff mentioned the unspent portions of Vote 10 funding were either reprofiled or lapsed at fiscal year-end.

As noted in program manager interviews, this funding is ideally suited to seed the initial Canadian innovation ecosystem to promote and support the research and development of lower TRL projects to address defence and security challenges that could eventually result in a procured solution.​

Program staff interviews cited an insufficient number of relevant challenges issued as one of the headwinds the program faced in spending its Vote 10 funding during the evaluation period. Furthermore, it was noted that the program was initially focused on competitive projects, funded through Vote 1 contracts. There was also a perception that the relevance of issued challenges through Vote 10 funds during this timeframe was potentially diminished given that this spending focused on lower solution-ready projects. ​

The COVID-19 pandemic presented unexpected challenges to the program and recipients of program funding. Particularly impacted were academic institutions (i.e., the majority recipient of Vote 10 funds) who had difficulties spending their funding as most universities had no students on campus, and few to no graduate students working in university laboratories. Program staff noted Vote 10 spending should improve in the coming years with the easing of COVID-19 restrictions.​

During four of the first five years following the program's launch, Vote 10 dollars were not fully expended.

Back to Table of Contents

Conclusions

Overall, the program received positive feedback from all program stakeholders including program personnel, DND/CAF members and innovators. IDEaS remains relevant in the context of SSE and in the face of the evolving defence and security environment, and is currently funded until 2037 to continue to deliver on its mandate. IDEaS has been effective in cultivating innovation within Canada, specifically with SMEs and academia. Through these relationships, solutions have been curated to address a wide range of challenges that the department is facing, and which DND/CAF members have found beneficial. 

Current barriers to fully supporting the future of innovation within the department include:
Governance; prioritization of challenges to align with departmental priorities; limited engagement opportunities between innovators and integrators/end-users; unclear processes between innovation and capability development; and the under-utilization of Vote 10 funds. Further developing the strategic direction for the program, enhancements with stakeholder engagement, and optimizing the program’s Vote 10 funds should alleviate some of the difficulties to ensure the future of IDEaS will effectively contribute to the departmental priorities and increase stakeholder confidence. 

Back to Table of Contents

Annex A: Management Action Plan

Management Action

Action 1.1 Director General of Research and Development Innovation (DGRDI) will develop a consistent method of defining its innovation program of work to better facilitate the prioritization, definition and sourcing of the required support for the challenges. Working with Director General Research and Development Program, DGRDI will develop new tools including a new standardized intake method for its innovation challenges through the DRDC call for S&T requirements, as well as a sourcing framework to identify the “best fit” for delivery for all new challenges as part of an integrated DRDC program. This approach will allow for efficiencies to be realized that will minimize the impact on stakeholders, and will take into account key considerations for streamlining the process including: the security classification of the S&T required; the Scientific/Technical Readiness Level; dollar value; expected economic benefit; and the current capability within DRDC research centre resources.

OPI: DGRDI
OCI: DG Research and Development Program, DG Research and Development Science and Engineering
Target Date: December 2024
Deliverable: DGRDI will develop an innovation program sourcing framework. This framework will ensure innovation challenges are consistently collected from clients and streamed to the most appropriate program, based on a flexible, yet pre-determined, set of criteria. 

Management Action

Action 2.1 The IDEaS program relies on successful partnerships and engagements with its stakeholders within the CAF and the DND; other government departments (OGD); academia and industry in order to meet the goals of the program. While IDEaS has demonstrated success in meeting the short and medium-term goals, there is recognition that there is a need for a refreshed and more deliberate approach to engagement in order to meet the long-term goals. The planned evolution of program governance will enable IDEaS to strengthen its relationship with its clients and stakeholders across the Defence Team and OGDs. IDEaS will also explore options to strengthen relationships with innovation partners (e.g., academic and industry) throughout the delivery of innovation projects in a manner that is more aligned with agile innovation project management. This will require consultation with colleagues at Public Services and Procurement Canada (PSPC) to ensure that such engagements would not present any risks to the integrity of the procurement process, and would remain aligned with the outcome-based approach to soliciting and investing in concept and advanced technology development and demonstration projects. It is important that all stakeholders remain engaged throughout this multi-year process, given that the development of solutions to innovations challenges requires increased financial investment as the maturity level of these solutions increases.

OPI: DGRDI
OCI: DGRDI
Target Date: September 2024
Deliverable: The IDEaS team will develop an options analysis report outlining possibilities to update its program delivery governance in order to reduce the programmatic burden on stakeholders, strengthen the relationship with DND/CAF and facilitate an agile innovation project delivery approach throughout its processes.

Management Action

Action 3.1 IDEaS is currently leveraging its Vote 10 capacity to advance projects in new ways to test different investment models. For example, a particular project under the CP element was no longer viable for higher Solution Readiness Level (SRL) testing/demo purposes based on the stakeholder’s needs; however, it was determined that there would be value in a deviation to the original project to acquire knowledge in the domain. Therefore, a contribution agreement that defined the new requirements is in development and will be managed using Vote 10 funds within the Innovation Networks element. In addition, a more optimized use of Vote 10 funding will help mitigate some of the capacity challenges faced by PSPC in processing IDEaS Vote 1 contracts.

OPI: DGRDI
Target Date: December 2024
Deliverable: IDEaS will research, develop and implement new methods of leveraging Vote 10 capacity to advance projects in innovative ways to test different investment models. The MAP will be considered fully implemented once a Concept paper has been finalized outlining an overview of the findings and resulting recommendations.

Back to Table of Contents

Annex B: Findings and Recommendations

Back to Table of Contents

Annex C: Evaluation Methodology

Data Sources

The findings and recommendations of this report were informed by multiple lines of evidence collected throughout the conduct phase of the evaluation, which were then triangulated to provide a fulsome overview. The research methodology used in the scoping and conduct of the evaluations are as follows:

Back to Table of Contents

Annex D: Evaluation Limitations

Back to Table of Contents

Annex E: Program Profile

Program Objectives

In the short term, the program aims to steer more external innovator expertise towards DND/CAF challenges by engaging with external innovators across Canada, ensuring these challenges provide good development opportunities and by streamlining the submission and review process, where possible. In the medium term, the IDEaS Program’s main goal is to mature relationships and partnerships among innovators and improve the quality of the solutions to the challenges issued by DND/CAF. Finally, in the long term the program will aim to demonstrate increased defence capabilities and increased economic benefits to Canada.

Program Outcomes

The IDEaS Program reports under the Defence Results Framework Program entitled “Science, Technology and Innovation,” with the outcomes of “[A] robust Canadian innovation base that can support Canada’s defence and security needs is established and enabled” and “[C]urrent and future military capabilities and related areas of scientific investigation are established and fulfilled.” 

Program Stakeholders

Assistant Deputy Minister (Defence Research and Development Canada (ADM((DRDC)) manages the IDEaS Program on behalf of DND. IDEaS challenges are derived from DND/CAF stakeholders. Level 1 organizations and public safety partners such as the Royal Canadian Navy, Canadian Army, Royal Canadian Air Force, Canadian Joint Operations and Command, Royal Canadian Mounted Police, Canadian Special Operations Forces Command, Veteran Affairs Canada and Crown Corporations are considered program clients.  

Internal to IDEaS Program 

• ADM(DRDC)'s Director General Innovation 

Internal to DND/CAF 

• Assistant Deputy Minister (Public Affairs) 
• Assistant Deputy Minister (Materiel) 
• Assistant Deputy Minister (Infrastructure and Environment)
• Assistant Deputy Minister (Information and Management)   
• Royal Canadian Navy
• Canadian Army
• Royal Canadian Air Force
• Canadian Joint Operations and Command
• Canadian Special Operations Forces Command

External to DND/CAF 

Key external partners include innovators from SMEs, industry, academia, non-governmental organizations and not-for-profits. The IDEaS Program maintains strategic partnerships with the Office of Small and Medium Enterprise, Canadian Intellectual Property Office, National Research Council, PSPC and Industrial Research Assistance Program. Clients include the Royal Canadian Mounted Police, Veteran Affairs Canada and Crown Corporations.

Foundational Document Performance Measurement Indicators

The program is expected to achieve the following results:

DRR Measurement: IDEaS results supporting DR 4.2: Defence and security challenges are addressed through innovative solutions. 

* For FY 2018/19, results are not available. New indicators as of 2018/19. 
** In FY 2019/20, projects were still in their first phase of work or had just transitioned to a second phase of funding (Component 1b). Therefore, most solutions were still in the development stage. In FY 2020/21, projects began completing the second phase of funding (Component 1b), and their broader solutions were delivered to DND/CAF for consideration. Therefore, DND/CAF began to benefit from completed Component 1b projects. 
The date to achieve the departmental result indicator is March 31, 2023. 
Source: Departemental Plan 2022-23

DRR Measurement: IDEaS Service Delivery

Program standard timeliness performance measures are publicly reported. For FY 2019/20, six performance measures were listed although no results data is available. This can be attributed to the relatively recent launch in 2018 of the IDEaS Program. For FY 2020/21, three performance results are available. The DRR indicates that the program has succeeded to meet one target over the last fiscal year. 

Source: Departmental Results Report, GC InfoBase

Back to Table of Contents

Annex F: Definitions

Back to Table of Contents