Language selection

Government of Canada / Gouvernement du Canada

Search

Evaluation of the Centre for Regulatory Innovation

On this page

Introduction

This report presents the results of an evaluation of the Centre for Regulatory Innovation (CRI), managed by the Regulatory Affairs Sector at the Treasury Board of Canada Secretariat (TBS). The evaluation was done by the Internal Audit and Evaluation Bureau with the assistance of Goss Gilroy Inc. in accordance with the Treasury Board Policy on Results. The evaluation was undertaken between April 2022 and March 2023 and covered the period from the implementation of the CRI in late 2019 to December 2022.

Results at a glance

Canada ranks close to the bottom within the Organisation for Economic Co-operation and Development in terms of regulatory burden (35th out of 38 countries) and produces less innovation output than other G7 countries. Regulatory innovation and experimentation require a skill set that is not common in the Government of Canada.

This evaluation will:

  • show that, through its two funds, the CRI:
    • meets an ongoing need
    • operates effectively in a regulatory system that does not often encourage innovation or taking risk
  • show that the CRI was able to adapt its model when it encountered challenges to better support capacity
  • detail how the two CRI funds ease the burden on regulatory departments and agencies (RDAs) and increase access to resources for innovation

The evaluation notes a potential for the CRI to:

  • showcase results from funded RDAs
  • strengthen partnerships in the innovation ecosystem
  • build greater awareness of the CRI’s products and services at the working level

The evaluation will also outline how the Regulatory Experimentation Expense Fund (REEF) was inefficient in how it disbursed funds and how exploring longer-term projects may help. Lastly, the evaluation highlights some enduring challenges faced by the Government of Canada in terms of regulatory innovation and experimentation:

  • the skills and knowledge needed to do regulatory innovation and experimentation are often underestimated
  • an environment that supports innovation and experimentation must be in place
  • successful innovation and experimentation are more than reaching an expected outcome because learning what does not work is equally important

Program context

The Regulatory Policy Division within TBS’s Regulatory Affairs Sector was established in 2017. The Cabinet Directive on Regulation was published in 2018 in line with Budget 2018, which outlined the Government of Canada’s commitment to a regulatory reform agenda. The Fall Economic Statement in 2018Footnote 1 announced initiatives to reform and modernize federal regulations to support innovation and business investment while continuing to protect the health and safety of Canadians and the environment. The initiatives included the establishment of regulatory reviews, the CRI, the Regulators’ Capacity Fund, and the Annual Regulatory Modernization Bill.

The CRI was originally conceived to help businesses connect with relevant RDAs and to manage regulatory sandboxes while new systems that support innovation and competitiveness were being tested. Over time, the CRI shifted its focus to building capacity.

The CRI:

  • supported RDAs in undertaking novel approaches to regulatory decision-making
  • helped build experimentation capacity
  • funded experiments

The CRI’s mandate is to help RDAs:

  • adopt innovative regulatory approaches that support regulatory competitiveness
  • work with industry to bring the application of new technologies into the Canadian marketplace

The CRI’s core activities, governance, proposal selection, and project funding are detailed in Appendix B. The CRI’s logic model is found in Appendix C.

Evaluation methodology and scope

Evaluators used a quasi-experimental evaluation approach to assess the implementation of the CRI, the CRI’s relevance, and the extent to which the CRI’s immediate outcomes were met. Evaluators consulted RDAs that used funds from the CRI and unfunded RDAs that did not apply for or use CRI funds, but that had hubs of innovation.

The evaluation used five lines of evidence:

  1. document review
  2. administrative data review
  3. interviews (n=24) with 13 departments and 11 TBS representatives
  4. interviews (n=9) with unfunded regulatory departments and agencies (RDAs)
  5. case studies (n=6)

Limitation of the evaluation

In all but one case, the evaluation team was unable to obtain perspectives from industry stakeholders and their experiences with the Regulatory Experimentation Expense Fund (REEF) despite making attempts to do so. The evaluation scope did not include the CRI’s assessments to select projects.

Relevance

Conclusion

There is an ongoing need to support regulatory innovation and experimentation given the demand for the Centre for Regulatory Innovation’s (CRI’s) services and funding, and the benefits that innovation brings to advance public policy. While some RDAs are innovating without the CRI, they are not likely to experiment.

Findings

The ongoing need for the CRI to facilitate regulatory innovation is demonstrated in part by the recommendations from the Economic Strategy Tables:

We’re calling for Canada to establish an agile, modernized regulatory system that ranks within the top quartile globally, is conducive to innovation, creates public trust and attracts investment. […] Underpinning this is a fundamental change in relationship between industry and regulators supported by ongoing dialogue and the use of pilot programs and sandboxes for experimentation. An agile regulatory process will support Canada’s desired environmental, health, safety and economic outcomes.Footnote 2

The literature review shows that experimentationFootnote 3 is a strategic tool for policy-makingFootnote 4 and a crucial part of innovationFootnote 5 as experimentation gives us a way to deal with complexity and uncertainty. Although departments and agencies do not have “enough access to experimental design expertise,”Footnote 6 failing to pursue novel regulatory approaches that support innovation in the public sector often means maintaining the status quo.

As heard in consultations with industry in the Targeted Regulatory Reviews, there is a need for novel regulatory approaches to support innovation. According to the publication Stagnant regulations impair innovation and economic growth, regulations in Canada are stagnant. In terms of regulatory burden, Canada ranks 35th out of 38 countries in the Organisation for Economic Co-operation and Development. Similarly, according to the most recent Global Innovation Index 2022 from the World Intellectual Property Organization, Canada produces less innovation output than other G7 countries relative to Canada’s level of investment.

The administrative data review shows that interest in both CRI funds is growing. In 2021–22, 66% more funds were requested than were available from the Regulators’ Capacity Fund (RCF). The amount requested for RCF funds also doubled between 2020–21 and 2021–22. Though funding requested from the Regulatory Experimentation Expense Fund (REEF) decreased in 2022–23, there were always more requests than the fund could support. In 2022–23, three times more funds were requested than were available from the REEF.

This is particularly critical in the federal government, which tends to feel ‘uncomfortable’ with experimenting and using innovation as a ‘buzzword’ while still staying ‘inside the lines.’

Key informants

Key informants are unanimous and case studies corroborate the continuing need for the CRI. Most key informants emphasized the importance of having a dedicated pool of money for innovation and experimentation. Some key informants related the need for the CRI to the context of continuous technological change and to the push for regulatory innovation via a safe space for experimentation. Key informants noted that the CRI enables the practice that innovation work requires. It was noted in the case studies that funding was particularly difficult to obtain through internal budget processes (for example, priority is typically given to program delivery, and experimentation is seen as very risky). For example, one key informant in a case study noted that RDAs are more likely to experiment if they have an available pool of money and space. Key informants also noted that available funding offset the need to seek funds through other mechanisms that might take longer or be more complex.

Regulatory innovation and experimentation without the CRI’s support

There is evidence that regulatory innovation is occurring in RDAs through regulatory agendas and Targeted Regulatory Reviews without the CRI’s support. Although evidence of regulatory experimentation is limited, RDAs that have undergone regulatory reviews since 2018 have developed regulatory initiatives based on their roadmaps.Footnote 7 In some cases, these initiatives are aimed at modernizing regulatory systems to enable innovation. The document review identified a few cases of departments and agencies using a regulatory sandbox to experiment. During the COVID-19 pandemic, in some RDAs, if the Minister believed that immediate action was required to deal with a significant risk to health, safety or the environment, the Minister could make interim orders that contain any provision in a regulation. For instance, under the Food and Drugs Act, Health Canada has authority to put in place interim orders in an emergency to allow for flexibility in requirements and processes. Health Canada’s regulatory innovation agenda also pursued other mechanisms, such as:

Interviewees from selected unfunded RDAs also provided evidence of some regulatory innovation and experimentation occurring without the CRI’s support.

For example:

  • most unfunded RDAs are involved in innovation that is unrelated to regulations
  • some RDAs are involved in regulatory innovation unrelated to the CRI mandate, such as through the Blue Economy Strategy or the Agile Regulations Table, which is described in Agri-food and aquaculture sector: Targeted regulatory review
  • a few unfunded RDAs are involved in the innovation hubs and labs of other departments, such as Employment and Social Development Canada, or other programs, such as Defence Research and Development Canada and the Canadian Safety and Security Program

Unfunded RDAs indicated three reasons why they may not be seeking support from the CRI:

  • (1) lack of capacity to dedicate resources to innovation or to seek support
  • (2) a preference to lead their own project without the reporting requirements and
  • application efforts required to seek funding or services from the CRI
  • (3) projects were not considered regulatory

Performance: implementation and outreach

Implementation

Conclusion

The Centre for Regulatory Innovation (CRI) evolved to its current role of building regulatory department and agency (RDA) capacity and enabling innovative approaches for regulatory innovation, though RDAs still face challenges meeting REEF’s criteria. The CRI could strengthen partnerships in the innovation ecosystem.

Findings

Adapting to evolving challenges and lessons learned

The document review shows that the CRI was initially conceived as a focal point for industry sectors interested in participating in regulatory experimentation. The CRI was mandated to connect with regulators, and to set up sandboxes and experiments to ensure that industry could introduce new products into the marketplace. The CRI was seen as a catalyst for change.

The CRI’s implementation evolved into a hybrid model to support RDAs interested in working with industry to facilitate innovation or experimentation and offset costs to RDAs. In this role, the CRI:

  • helps RDAs convene relevant parties
  • provides guidance
  • offers supplemental funding

While it was timely how the CRI evolved, the CRI’s status as an industry focal point did not materialize, and the implementation of its initiatives occurred later than planned. Some key informants indicated that most of the initiatives were delayed from 2019 to 2020, which coincided with the pandemic. As the Regulators’ Capacity Fund (RCF) was not part of the initial plan for the CRI, funding was provided to establish the RCF without additional resources to manage it. As a result, key informants indicated that establishing the RCF drew full-time equivalents (FTEs) away from the Regulatory Experimentation Expense Fund (REEF), which delayed implementation of the REEF.

For some interviewees, the lack of RDA capacity was another challenge. A lack of resources and knowledge of experimentation limited their ability to work with the CRI. Regulatory experimentation was not well known and understood. The range of skills, the maturity of those working in regulatory affairs, and their ability to differentiate innovation from experimentation were overestimated by the CRI.

The evaluation found that applicants to the REEF found it challenging to qualify for funding; interviewees said that it was difficult to meet the requirements set by the CRI. CRI key informants indicated that a high level of effort was needed to help applicants ensure that their projects met the criteria. Challenges included problem definition, description of project viability, experimental design, methodology, indicators, and associated metrics.

A few interviewees pointed out that the CRI is evolving to broaden the scope of the REEF to include nascent projects given that RDAs require support to develop proposals. In such cases, RDAs can use funds from the REEF to help them uncover opportunities for regulatory experimentation or innovation. Broadening the scope aligns well with the recommendation by Nesta, which is a partner of the CRI, to support projects at earlier stages of project design in order to help with problem definition and learning objectives.

More recently, CRI has received more proposals for the government to digitize processes or develop enterprise solutions. Based on a recommendation from the CRI Steering Committee, the CRI clarified its support for initiatives with significant information technology (IT) components. The CRI supports projects that are developing experiments to validate whether an IT solution is viable or the best solution to their problem. Because these IT proposals require large amounts of funding and REEF funding is limited, the CRI decided to limit such funding to projects aimed at developing prototypes.

Eligibility criteria for the Regulators’ Capacity Fund (RCF) and the Regulatory Experimentation Expense Fund (REEF)

RDAs had mixed feelings about the eligibility criteria for the REEF and the RCF.

For the RCF, most funded RDAs did not face challenges meeting the criteria and most key informants found the criteria were clear. Only a few struggled to differentiate between the RCF and the REEF.

For the REEF, most funded RDAs faced challenges meeting the technical criteria. Key informants said that while the process was long and intense, it was helpful in forcing RDAs to think systematically about experiments.

Other challenges that RDAs reported with regard to the REEF’s eligibility criteria include:

  • the complexity of defining an experiment, which lengthened the amount of time required to complete their application (this complexity was particularly challenging for those who are not researchers)
  • the definition of viability as a criterion in the early stages of a proposal because such a definition requires the assessment of a multitude of parameters to build the evidence and experimentation framework
  • a lack of consistency in the CRI’s application of criteria from one project to another

To be eligible for the RCF, a proposal is assessed for its economic impacts and competitiveness, while the REEF uses the criteria of public benefit and viability.

The role of partnerships

The document review shows that partnerships played an important role in the CRI’s effectiveness.

The CRI:

  • engaged in and coordinated efforts on regulatory innovation with partner departments and internationally
  • established an interdepartmental working group
  • had a steering committee that included:
    • the Community of Federal Regulators
    • Innovation, Science and Economic Development Canada (ISED)
    • Environment and Climate Change Canada
    • Transport Canada
    • Health Canada
    • the Privy Council Office’s Impact and Innovation Unit
    • Policy Horizons, which worked on horizontal initiatives

The CRI has also partnered with the TBS Experimentation Group, the Canada School of Public Service, the Office of the Chief Information Officer and ISED. The document review and interviewees indicated that, given the role of ISED Global Innovation Clusters in creating partnerships to build innovation ecosystems, the CRI has begun discussions with ISED Innovation Superclusters and ISED’s Global Hypergrowth Project to connect with regulators and identify regulatory barriers earlier. Doing so enables the CRI to explore opportunities for collaboration and to enlarge its network of new partners.

The CRIFootnote 8 coordinates Canada’s participation in Agile Nations, which is an intergovernmental forum that fosters cooperation in the global regulatory environment so that innovation can thrive. Interviews with two RDAs point to the need for TBS to facilitate communication and collaboration between RDAs for joint proposals and participation in Agile Nations. Interviewees indicated that they made their own connections to work with Agile Nations when the CRI could have made it possible given its “bird’s eye view of the regulatory innovation landscape.”

The jurisdictional review noted three best practices that TBS could pursue with Agile Nations:

  1. Consider using an engagement tracker to identify RDAs that are interested in or are good candidates for participation in activities related to Agile Nations. Doing so would enable TBS to solicit RDA participation in Agile Nations and would facilitate communication between RDAs that are working on related regulatory innovations.
  2. Provide more direct support, when possible, to alleviate capacity issues that prevent interested parties from participating in activities related to Agile Nations. An example is to provide more help with writing proposals.
  3. Continue to share information about Agile Nations when conducting workshops to build awareness and identify interested partners for future engagement.

Awareness and outreach

Conclusion

The Centre for Regulatory Innovation’s (CRI’s) outreach and awareness-building efforts are unevenly received, though demand for the CRI is growing.

Findings

Feedback received in 2020 showed that most regulatory departments and agencies (RDAs) knew about the CRI indirectly. Based on this feedback, the CRI established an engagement plan to promote a whole-of-government approach to regulatory experimentation in 2021. The CRI reached out to 21 large RDAs, as well as to private sector and international stakeholders, including Agile Nations. In 2022–23, the CRI’s engagement and outreach plan included five engagement objectives and activities to target RDAs and external stakeholders. The CRI reached out to other RDAs and some innovation teams, and targeted some large RDAs with experiences and tangible examples of experimentation to attract or help other RDAs. The CRI relied on existing structures (that is, steering committees and working groups composed of senior officials) or connections with RDAs to communicate the CRI’s work. A few interviewees indicated that largely relying on senior officials to disseminate information about the CRI has not been very effective.

Key informants were divided on the effectiveness of the CRI’s promotional efforts.

Half described the efforts as ineffective with most indicating they had never seen any direct outreach or promotion by the CRI. Interviewees became aware of the CRI indirectly, such as through colleagues, regulatory forums or word of mouth.

The other half, mainly those who have a working relationship with the CRI (for instance, members of the CRI working group), described efforts to promote the CRI as effective. Interviewees in this group became aware of the RCF and the REEF through the CRI distribution list, discussions, regular communications materials and call-outs for proposals.

Performance: effectiveness

Supporting innovation

Conclusion

The CRI has supported innovation to some extent. RDAs’ lack of capacity and authorities hindered the use of the CRI’s services, which slowed their innovation capability. Both the CRI and RDAs could do more to showcase projects and results. REEF timelines make it difficult to see lasting results.

Findings

Supporting regulatory departments and agencies (RDAs)

The document review indicates that the creation of the CRI itself is perceived as innovative. Regulatory experimentation and the use of sandboxes to innovate is new in government. Most interviewees agree that offering a safe place for RDAs to experiment or undertake pilot projects is useful, particularly when funding is available for that purpose. The CRI and some RDAs were not able to leverage these tools as expected given risk aversion and lack of capacity (FTEs and budget). There is also a lack of understanding of the difference between a sandbox and an experiment, and how to use sandboxes and experiments to address barriers faced by industry. Finally, there is a lack of legislative authorities in most RDAs to conduct regulatory experiments, which creates a need for exemptions to conduct such experiments.

Unfunded RDAs identified three ways that the CRI could support RDAs:

  1. Playing a role in “connecting the dots between departments” to facilitate discussion, improve awareness of key players, and coordinate work between different departments working in the same space
  2. Tailoring outputs to all audiences involved in regulatory innovation work (for example, by using plain language and making outputs easily understandable)
  3. Helping RDAs with their own internal “self-imposed barriers,” such as risk aversion
Sharing results

The document review shows that the CRI shares project results with RDAs through the CRI supported projects on GCcollab, the CRI Regulatory Experimentation presentation for regulators and through the Regulators’ Capacity Fund Lessons Learned Report. Though most interviewees agreed that the results of CRI-funded projects have been shared across government, some think that the CRI should take a more active approach by showcasing projects and the results externally, such as to ISED’s Innovation Supercluster, Global Hyper Growth Initiatives, Agile Regulations Initiatives or at the Director General–level Agile Regulations Table.Footnote 9 Although the CRI has begun discussions with ISED supercluster groups, these discussions have not progressed to the point of showcasing projects to private industry.

The evaluation found that RDAs themselves sometimes shared their project results across government and industry. Half of case studies presented their project results to some extent across government. In one case study, the project results were extensively shared through workshops and various outreach sessions with relevant industry associations and other agencies. In another case study, the results were shared with federal and provincial partners and the Canada School of Public Service and were made available under a Creative Commons licence for other agencies to leverage. The evaluation also found, in two case studies, that the industry partners involved were not aware of the CRI’s funding or participation.

Supporting innovation

Most of the projects funded under the Regulatory Experimentation Expense Fund (REEF) have a long life cycleFootnote 10 and are either two-year projects or in phases and have not been finalized. An assessment by Nesta in 2022 showed that two of five REEF projects supported industry innovation:

  1. a sandbox for a pilot artificial intelligence (AI) accreditation program
  2. a sandbox for the use of currently prohibited aircraft for pilot training in flight school

Interview and case study evidence show that even though the pilot school withdrew due to an issue with procuring the required aircraft during the pandemic, AI accreditation under the Standards Council of Canada continues. Interviewees involved in the AI project were confident that the project would result in important learning for the future of quality control standards in AI development and expected to broaden the scope to include additional AI applications in the future.

Other projects tested new approaches or changes to internal processes. In one case, the CRI helped ISED leverage AI and improve the way the ISED Competition Bureau does business. In another case, the CRI helped ISED fill an educational gap in the digital credentials space for people working in policy and business. The project experimented with different educational approaches that would be most relevant to this audience.

Departments acquiring capacity

Conclusion

The Centre for Regulatory Innovation’s (CRI’s) guidance and best practices strengthened RDAs’ experimentation capacity. Time will tell whether this strengthening is sustainable.

Findings

The document review shows that funded regulatory departments and agencies (RDAs) received advice, lessons learned, workshops and webinars from the CRI. In 2021 and 2022, the CRI conducted two workshops to share lessons with funded RDAs and to provide funded RDAs with an opportunity to share their experiences. The CRI posted the lessons learned report on GCcollab, which made it available to regulators. Interview and case study evidence show that recipients of funding from the RCF found the sessions helpful, particularly the breakout sessions that allowed recipients of funding to connect with others who had undertaken similar projects.Proactively sharing success stories more broadly is one area that interviewees felt the CRI could improve in.

The CRI partnered with Nesta to advise regulators and develop the Regulators’ Experimentation Toolkit.

The CRI partnered with Nesta to advise regulators and develop the Regulators’ Experimentation Toolkit which gave funded RDAs access to Nesta’s international best practices and guidance.Footnote 11 Some RDA interviewees said that the toolkit guided them on how to design and run regulatory experiments and sandboxes, and measure performance. The CRI also partnered with the Experimentation Working Group in TBS’s Priorities and Planning Sector. In response to the questions that the working group received about regulatory experimentation, the working group promoted the CRI’s regulatory experimentation guidance.

Case study evidence showed that the CRI largely supported projects during the initial stages. For example, most interviewees felt that the CRI supported RDAs by helping them identify regulatory experiments to pilot and by providing funding or examples. All interviewees agreed that the CRI’s support was useful in providing an iterative process for learning. Some interviewees suggested that having a case manager or person responsible for follow up within the CRI would connect the RDA with relevant contacts and help with solutions, which may often be outside the RDA’s jurisdiction.

A few interviewees agreed that the CRI helped build the capacity to undertake experiments, though it is too early to tell if the RDAs can sustain the capacity to undertake more experiments.

Some examples of how the CRI supported the development of departmental capacity include:

  • a Policy Lab with Environment and Climate Change Canada where the CRI and Nesta provided expertise on developing and innovation matrix to undertake a policy experiment
  • a Competition Tool with the ISED Competition Bureau where the CRI directed regulators to external experts who could assist them in developing the desired tool
  • a Pilot Accreditation Program for the assessment of AI management where the CRI and Nesta provided ongoing advice to undertake the experiment

Application and use of results

Conclusion

While most RCF recipients applied the results of their projects to varying degrees, it is too early to say for REEF recipients.

Findings

Evidence from most of the case studies and interviewees show that Regulatory Experimentation Expense Fund (REEF) recipients were not yet able to apply the results of their experiments, though the results were extensively shared throughout government and industry.

Regulatory Experimentation Expense Fund

The Centre for Regulatory Innovation’s (CRI’s) experimentation projects have a long life cycle and could include different phases; a few are still ongoing, and in one case, the project could not be realized. The case studies and interviews provided some examples, including:

  • one project just finished its first phase and planned to apply what it learned to other regulatory issues in the chemical sector
  • an experimentation project in AI received funding for two fiscal years and it is still ongoing (before REEF, the RDA used the Regulators’ Capacity Fund (RCF) for the same project)
  • for an experiment, the REEF’s two-year limit was incompatible with the type of research required and the reallocation of funds from one year to the next did not help with procurement issues; consequently, the experiment was not continued

In one case study, there was evidence that three different educational approaches (an educational portal, demonstrations and interactive sessions) and supporting materials that were developed and used for digital credentials were available through an ISED online learning portal and shared under a Creative Commons licence. These approaches were made available to encourage other organizations to leverage and improve upon them. An interviewee indicated that some of the content has already been repurposed by a third party.

Regulators’ Capacity Fund

Evidence from interviews and case studies show that most Regulators’ Capacity Fund (RCF) recipients were able to apply the results of their projects to varying degrees, including:

  • one RDA published their toolkit, which was rated as one of the most useful tools in the space during the first year that the toolkit was released
  • another RDA launched a new portal that provides information and services for online dispute resolution to support compliance with the Pay Equity Act
  • a few RDAs published project results on their websites and presented at conferences
  • some other projects were exploratory but resulted in recommendations for future work, for instance in producing a roadmap toward a collaborative approach to creating permits for mining projects

The limited project time frame prevented some RDAs from applying results as they had hoped.

Both funded and unfunded RDAs offered the following suggestions for the CRI’s consideration:

  • Consider ways to fund more long-term initiatives or projects. The one-to-two-year limit is incompatible with research projects that require procurement, which can be a lengthy process
  • Determine common barriers to regulatory innovation in the public sector (for example, offering targeted support to facilitate the hiring of consultants or experts)
  • Improve project scope by having a standard and clear (that is, less technical) definition of experimentation and innovation, including examples of results and outcomes
  • Broaden funding goals for the REEF to include other innovation tools besides experimentation in order to increase accessibility and enable innovation

Performance: efficiency

Conclusion

The data shows that the Regulators’ Capacity Fund (RCF) is in demand and that the current design is efficient. In contrast, the Regulatory Experimentation Expense Fund (REEF) continues to face challenges in its delivery and ability to dispense funds to RDAs.

Findings

The RCF received a total of $10 million between 2019 and 2022. Due to delays associated with the launch, most of the funding for 2019–20 had to be reprofiled to 2020–21 and 2021–22 (Appendix D, Table 1).

In 2020–21, the RCF was unable to spend its total available budget due to:

  • the withdrawal of some applicants, which resulted in only 18 applications for funding
  • the COVID-19 pandemic
  • administrative delays, which resulted in memoranda of understanding not being finalized until late 2020

These delays resulted in applicants requesting smaller amounts or postponing projects until the following fiscal year.

The CRI improved the administration of the RCF by beginning the intake process for 2021–22 sooner and by working closely with applicants to ensure that they had the internal capacity to proceed with their projects before approval. As a result, a total of 22 new requests were made and 12 were approved. The RCF disbursed $5.5 million or 86.9% of its budget.

The REEF has not been able to disburse all its funds since 2020. The funding available for REEF increased steadily between 2020–21 and 2022–23, while the percentage of funds lapsed remained high and has not fallen below 36% (Appendix D, Table 2). The high level of lapsed funds is due to a decline in the number of applications submitted between 2020–21 and 2022–23 and low approval rates (Appendix D, Table 3).

The evaluation found that applicants to the REEF faced challenges that may have dampened their interest in applying for the REEF, such as:

  • the COVID-19 pandemic
  • difficulty meeting REEF requirements: key informants from the CRI indicated that a high level of effort was needed to help applicants ensure that their projects met the necessary criteria
  • difficulty in reflecting the experimental approach in the proposal rather than in the implementation of the innovation: applicants were encouraged to apply to the RCF instead
  • of the seven projects diverted to the RCF between 2020 and 2023, five were subsequently approved under the REEF

The CRI indicated that in the REEF’s second year, the CRI made administrative changes to better align funded projects with the purpose of the REEF. These changes have not resulted in an increase in applications to the REEF compared with 2020–21. The changes introduced new challenges to some RDAs that were surprised by the tightening of requirements and the lack of consistency.

The tightened requirements reflected a relatively narrow conceptualization of experimentation, which Nesta identified as a possible risk in their final report to the CRI in 2022. Nesta cautioned that focusing on technical aspects too heavily would make regulators feel “out of their depth” and hesitant to participate. They noted that requiring a less technical approach may improve uptake of the REEF and enable the regulatory community to build skills to improve their ability to conduct more technical experiments in the future.

Recommendations

Given the ongoing need to support regulatory innovation capacity in Canada and to increase RDAs’ expertise, it is recommended that:

  1. The Centre for Regulatory Innovation (CRI) focus on innovation more broadly, with experimentation being only a component among many. This could be accomplished by developing a strategic framework for innovation and by creating a regulatory toolbox.
  2. The CRI improve awareness of their products and services to enable more departments and agencies to engage in regulatory innovation and connect with regulatory departments and agencies (RDAs) already working in that space.
  3. The CRI strengthen its partnerships with players in the innovation ecosystem, in and outside the Government of Canada, to inform the CRI’s suite of services.

Appendix A: summary of conclusions and recommendations

Relevance

Conclusion: There is an ongoing need to support regulatory innovation and experimentation given the demand for the Centre for Regulatory Innovation’s (CRI’s) services and funding, and the benefits that innovation brings to advance public policy. While some regulatory departments and agencies (RDAs) are innovating without the CRI, they are not likely to experiment.

Performance: implementation and outreach

Conclusions:

  • Implementation: The CRI evolved to its current role of building RDA capacity and enabling innovative approaches for regulatory innovation, though RDAs still face challenges meeting the Regulatory Experimentation Expense Fund’s (REEF’s) criteria. The CRI could strengthen partnerships in the innovation ecosystem.
  • Awareness and outreach: The CRI’s outreach and awareness-building efforts are unevenly received, though demand for the CRI is growing.

Performance: effectiveness

Conclusions:

  • Supporting innovation: The CRI has supported innovation to some extent. RDAs’ lack of capacity and authorities hindered the use of the CRI’s services, which slowed their innovation capability. Both the CRI and RDAs could do more to showcase projects and results. REEF timelines make it difficult to see lasting results.
  • Departments acquiring capacity: The CRI’s guidance and best practices strengthened RDAs’ experimentation capacity. Time will tell whether this strengthening is sustainable.
  • Application and use of results: While most RCF recipients applied the results of their projects to varying degrees, it is too early to say for REEF recipients.

Performance: efficiency

Conclusion: The data shows that the Regulators’ Capacity Fund (RCF) is in demand and that the current design is efficient. In contrast, the Regulatory Experimentation Expense Fund (REEF) continues to face challenges in its delivery and ability to dispense funds to RDAs.

Recommendations

it is recommended that:

  1. The CRI focus on innovation more broadly, with experimentation being only a component among many. This could be accomplished by developing a strategic framework for innovation and by creating a regulatory toolbox.
  2. The CRI improve awareness of their products and services to enable more departments and agencies to engage in regulatory innovation and connect with RDAs already working in that space.
  3. The CRI strengthen its partnerships with players in the innovation ecosystem, in and outside the Government of Canada, to inform the CRI’s suite of services.

Appendix B: program description

The Centre for Regulatory Innovation (CRI) is a capacity-building model thatsupports the ability of regulators to undertake novel approaches or regulatory experiments to inform regulatory decision-making.

The CRI supports federal regulators by helping them:

  • expand their knowledge of innovative approaches in a regulatory context
  • increase their ability to undertake experimentationFootnote 12 in orderto inform regulatory decision-making
  • try sandboxesFootnote 13 and other approaches to respond to regulatory challenges
  • provide support and advice to industry innovators

The CRI’s core activities are:

  • experimentation advisory services
  • international engagement on regulatory innovation

Experimentation advisory services

The CRI supports regulators by:

  • developing tools, policies and system-wide frameworks
  • providing advice to support the design and implementation of experiments and sandboxes run by regulators

The CRI supplements financial support by administering two funds:

  1. Regulatory Experimentation Expense Fund (REEF),Footnote 14 which offsets expenses incurred by regulatory departments and agencies (RDAs) conducting CRI-approved regulatory experiments. In 2023, the CRI expanded the scope of the REEF to support regulators with their pre-experimentation efforts, such as seed projects.
  2. Regulators’ Capacity Fund (RCF),Footnote 15 which supports projects that improve the capacity of regulators to incorporate economic and competitiveness considerations in the design or implementation of a regulatory regime.

International engagement on regulatory innovation

The CRI engages in and coordinates efforts on regulatory innovation within government with industry innovators, academics and international partners.

The CRI coordinates Canada’s participation in Agile Nations, which fosters cooperation on rule-making for innovation in order to make it easier for businesses to introduce and scale innovations across member markets. Member countries include Canada, Denmark, Italy, Japan, Singapore, the United Arab Emirates and the United Kingdom.

Funding decisions for the REEF

The CRI uses a two-step process to make funding decisions:

  1. RDAs submit a high-level expression of interest document that is used by the CRI to determine whether a project is eligible for funding.
  2. The CRI invites select projects to submit a more detailed proposal, which allows the interdepartmental director general–level steering committee to confirm the project’s eligibility and rank the project using the assessment criteria in the Regulatory Experimentation Expense Fund Guide.

Funding decisions for the RCF

How RDAs apply for funding from the RCF and how the CRI decides which RDAs receive funding.

  1. The Centre for Regulatory Innovation (CRI) releases a call for expressions of interest.
  2. Regulatory departments and agencies (RDAs) submit expressions of interest.
  3. The CRI invites select RDAs to submit proposals.
  4. Select RDAs submit proposals.
  5. The CRI makes funding decisions.
  6. The CRI and the funded RDAs sign memoranda of understanding
  7. The funded RDAs launch their projects.
  8. The CRI supports the funded RDAs during their projects.
  9. The funded RDAs provide feedback and reports to the CRI.
  10. The funded RDAs complete their projects.
  11. The CRI shares lessons learned from the projects.

The CRI governance structure

The CRI reports to the Executive Director, Regulatory Policy and Cooperation Directorate, Regulatory Affairs Sector, Treasury Board of Canada Secretariat (TBS). The CRI’s overall operations are governed by an interdepartmental steering committee, which is composed of director general–level representatives from:

  • the Regulatory Policy and Cooperation Directorate (TBS) (chair of the committee)
  • the Privy Council Office
  • Innovation, Science, and Economic Development
  • Transport Canada
  • Environment and Climate Change Canada
  • Health Canada
  • the Community of Federal Regulators

Appendix C: logic model for the Centre for Regulatory Innovation (CRI)

Appendix C. Text version below
Appendix C - Text version

Long-term outcomes (changes in state)

  • The federal regulatory system supports regulatory competitiveness while continuing to protect the health and safety of Canadians and the environment
  • The federal regulatory system supports industry innovation while continuing to protect the health and safety of Canadians and the environment

Intermediate outcomes (changes in behaviour)

  • Regulatory departments and agencies (RDAs) adopt innovative regulatory models, administration tools and practices
  • RDAs implement regulatory approaches and practices that help industry bring innovative products and services to the marketplace

Immediate outcomes (changes in capabilities)

  • Increase the knowledge of RDAs about innovative solutions and their regulatory application
  • Increase the ability of RDAs to innovate and use experimental methods to inform decision-making
  • Increase the ability of RDAs to use sandboxes to allow industry to safely test an innovation before the innovation enters the marketplace
  • Increase the ability of RDAs to provide regulatory support and timely advice to innovators

Appendix D: disbursement of funds

Table 1: funding available to the Regulators’ Capacity Fund (RCF), funding disbursed by the RCF and funding lapsed, from 2019–20 to 2021–22
Fiscal year Initial available funding Final available funding Funds requested Funds disbursed or spent Funds lapsed Percentage of funds lapsed
2019–20 $2,000,000 $250,000

Not applicable

$250,000Footnote *

$0Footnote †

 0%
2020–21 $4,000,000 $3,425,203 $4,728,513 $3,425,203

$0Footnote ‡

 0%
2021–22 $4,000,000 $6,324,797 $9,551,926 $5,497,000 $827,797 13.1%
Total $10,000,000 $10,000,000 $14,280,439 $9,172,203 $827,797 8.3%

Source: the Centre for Regulatory Innovation (CRI)

Note: The evaluation did not consider the renewal of the RCF for the efficiency analysis. The RCF was made available to the CRI late in September 2022, and the call for proposals was in October. As a result, not all 2022–23 funds have been committed as of the writing of this report. The CRI is exploring options to ensure that the historical demand for these funds can be met by disbursing funds as much as possible through 2024–25.

Table 2: funding available to the Regulatory Experimentation Expense Fund (REEF), funding disbursed by the REEF and funding lapsed, from 2019–20 to 2022–23
Fiscal yearFootnote * 2019–20 2020–21 2021–22 2022–23
Total funds requested $0.00 $4,503,704.60 $4,949,820.77 $3,210,733.66
Available funding $200,000.00 $500,000.00 $700,000.00 $1,000,000.00
Funding committed $0.00 $339,004.60 $569,520.00 $613,033.66
Funding disbursed $0.00 $272,000.00 $441,783.74 $612,726.34
Percentage of funds disbursed 0.00% 54.40% 63.11% 61.27%
Funds lapsed $200,000.00 $228,000.00 $258,216.26 $387,273.66
Percentage of funds lapsed 100.00% 45.60% 36.89% 38.73%

Source: the CRI
Table 3: number of projects that applied for and received funding from the Regulatory Experimentation Expense Fund (REEF), from 2020–21 to 2022–23
Fiscal year 2020–21 2021–22 2022–23
Number of projects that applied for funding from the REEF 22 6 9
Number of projects that received funding from the REEF 5 3 3

Source: the CRI

Appendix E: Management Response and Action Plan

The Centre for Regulatory Innovation (CRI) of the Regulatory Affairs Sector, Treasury Board of Canada Secretariat (TBS), has reviewed the evaluation report and agrees with the report’s recommendations. Proposed actions to address these recommendations are outlined in the tables below.

Recommendation 1

CRI should focus on innovation more broadly, with experimentation being one component among many. This goal could be accomplished by:

  • developing a strategic framework for innovation
  • creating a regulatory toolbox

Management response

The CRI agrees with the recommendation to expand its services to:

  • support more novel regulatory approaches beyond experimentation
  • support innovation-friendly regulatory environments more broadly
Proposed actions for Recommendation 1 Start date Targeted completion date Office of primary interest

1.1 Develop a strategic framework that defines the pre-conditions, skills and steps associated with regulatory innovation and use these as a basis for scoping the initiatives that CRI would support and the services that CRI would deliver to help regulators under a broadened innovation mandate.

January 2024

December 2024

TBS-RAS-CRI

1.2a. Identify a suite of novel approaches that supports innovation in regulatory design and administration, and assess their potential value as a CRI line of service for federal regulators.

Potential areas for exploration include:

  • regulatory foresight
  • co-development
  • behavioural insights and other non-binding or non-prescriptive regulatory approaches
  • regulatory technologies

August 2023

February 2024

TBS-RAS-CRI

1.2b. Informed by the assessment in response to action 1.2a, develop options and a recommendation for expanding the scope of experimentation-specific funding to support a broadened innovation mandate.

February 2024

April 2024

TBS-RAS-CRI

1.2c. Create a plan for the phased expansion of CRI services that is aligned with available resources. The intent is for the plan to be evergreen.

April 2024

March 2025

TBS-RAS-CRI

1.2d. Implement the plan created in action 1.2c. The intent is for the implementation to be ongoing.

April 2025

March 2026

TBS-RAS-CRI

Recommendation 2

CRI should improve awareness of its products and services to enable more departments to engage in regulatory innovation and connect with regulatory departments and agencies (RDAs) already working in this area.

Management response

The CRI agrees with the need to expand outreach and awareness efforts to:

  • more effectively reach the whole federal regulatory community
  • build awareness of its service offerings
Proposed actions for Recommendation 2 Start date Targeted completion date Office of primary interest

2.1 Develop an engagement strategy that includes:

  • setting a baseline frequency for promotional activities
  • outlining plans to reach RDAs more effectively, including and beyond those represented in the CRI Working Group

The intent is for the strategy to be evergreen.

July 2023

March 2024

TBS-RAS-CRI

2.2 Implement the engagement strategy developed in action 2.1. The intent is for implementation to be ongoing.

April 2024

March 2025

TBS-RAS-CRI

Recommendation 3

CRI should strengthen its partnerships with players in the innovation ecosystem, in and outside the Government of Canada, to inform the CRI suite of services.

Management response

The CRI agrees with the recommendation to further leverage partnerships with RDAs, and with other domestic and global groups, to improve understanding of needs, priorities, barriers and best practices for regulatory innovation. This improved understanding will inform the services that CRI delivers.

Proposed actions for Recommendation 3 Start date Targeted completion date Office of primary interest

3.1. Undertake an environmental scan to identify key targets and relevant initiatives in the domestic and international regulatory innovation landscape.

August 2023

December 2023

TBS-RAS-CRI

3.2 Engage with targets identified through the environmental scan and identify and document roles and products for collaboration and information-sharing on regulatory innovation experiences and best practices.

January 2024

June 2024

TBS-RAS-CRI

Page details

2024-06-13