Pre-market guidance for machine learning-enabled medical devices

Download the alternative format
(PDF format, 661 KB, 26 pages)

Organization: Health Canada

Date published: 2025-02-05

On this page

Introduction

Artificial intelligence (AI) is a broad term for a category of algorithms and models that perform tasks and exhibit behaviours such as learning and making decisions and predictions. Machine learning (ML) is the subset of AI that allows ML training algorithms to establish ML models when applied to data, rather than models that are explicitly programmed.

Medical devices that use ML, in part or in whole, to achieve their intended medical purpose are known as machine learning-enabled medical devices (MLMD). "Medical purpose" refers to parts (a) through (e) of the "device" definition within the Food and Drugs Act (act). MLMD are subject to the act and associated Medical Devices Regulations (regulations).

In this guidance, "transparency" describes the degree to which appropriate and clear information about a device (that could impact risks and patient outcomes) is communicated to stakeholders (such as, patients, users, health care providers and regulators). Transparency is an important aspect of the device's safety and effectiveness, and helps stakeholders make informed decisions.

This guidance introduces the concept of a predetermined change control plan (PCCP). A PCCP provides a mechanism for Health Canada to address cases where the regulatory pre-authorization of planned changes to ML systems is needed to address a known risk.

In the face of uncertainties and risks associated with ML and PCCPs, the ongoing safety and effectiveness of marketed MLMD can be strengthened by including terms and conditions (T&Cs) on medical device licences, as appropriate.

Health Canada has adopted the MLMD terms and definitions used by the International Medical Device Regulators Forum (IMDRF). Manufacturers are encouraged to review this document:

In this guidance, "ML training algorithm" refers to the software procedure that establishes the parameters of an ML model by analyzing data. The "ML model" represents a mathematical construct that generates an inference or prediction based on new input data and is the result of an ML training algorithm learning from data. The "ML system" refers to an ML-enabled software that meets the definition of medical device as per Section 1 of the regulations, including ML models and the associated ML training algorithms.

Scope and application

This document provides guidance to manufacturers who are submitting a new or amendment application for Class II, III and IV MLMD under the regulations.

The information in this guidance relates to the ML system of an MLMD. It does not cover the non-ML information required in a medical device licence application.

Manufacturers should also consult other relevant guidance relating to medical devices, including the following:

Policy objective

This guidance outlines supporting information to consider when manufacturers are demonstrating the safety and effectiveness of an MLMD:

Policy statements

An MLMD can be standalone software that meets the definition of a medical device. It can also be a medical device that includes software that meets the definition of a medical device.

An MLMD can be an in vitro diagnostic device (IVDD) or a non-IVDD. The risk classification of an MLMD can range from Class I to Class IV.

Manufacturers should clearly state that the device uses ML in their cover letter for all Class II, III and IV applications for an MLMD. Furthermore, for MLMDs that have a PCCP, manufacturers should clearly state in their cover letter that their device includes a PCCP. Excluding such statements could delay the application process.

Manufacturers should include a justification for the proposed medical device classification applied to the MLMD. This justification should reference the classification rules outlined in Schedule 1 of the regulations.

Medical devices must meet the applicable requirements of sections 10 to 20 of the regulations. Manufacturers must ensure that objective evidence is available to support the intended use of the MLMD, the safety and effectiveness of the device and the associated claims.

An application must demonstrate that the MLMD (including the PCCP, as appropriate):

Class II, III and Class IV applications must include the information listed in section 32 of the regulations. Additional information may be requested at any time during our review of an application (new or amendment) or after a device has been licensed.

Health Canada understands that manufacturers may use a variety of information, methodologies and evidence to demonstrate that their MLMD is safe and effective. Additionally, different intended uses or risk profiles may require different types or levels of evidence. As such, we have outlined information for consideration rather than prescribing the required information for all scenarios. Health Canada applies a risk-based approach to determine the evidence requirements.

The guidance for implementation section of this document outlines the information to consider including, or having available upon request, for an MLMD licence application. If any of the information identified in this section is not available, manufacturers should offer a justification or provide alternative information, as applicable.

Data referred to or used by manufacturers should be justified as adequately representative of the Canadian population and clinical practice. Any data used to develop the MLMD or demonstrate a device's safety and effectiveness should reflect the population for whom the device is intended. For example, this could include consideration of skin pigmentation, biological differences between sexes and other identity-based factors.

For those devices that are authorized with a PCCP, subsequent changes made according to the authorized PCCP do not require that you submit a medical device licence amendment application for a significant change. Those changes should be documented within the manufacturer’s Quality Management System. PCCP-driven changes are still subject to other licence amendment requirements, such as a change in identifier of the device, and relevant post-market regulatory oversight.

For amendments to a device that are outside of an authorized PCCP, including changes to the PCCP itself, the regulations and relevant guidance documents should be consulted before implementation. It's important to determine whether the change constitutes a significant change and requires an application for a medical device licence amendment.

A PCCP may be submitted with applications for a new medical device licence or a medical device licence amendment. Manufacturers may consider the pre-submission process, as appropriate, to discuss a proposed PCCP prior to submitting a licence application.

This guidance represents Health Canada’s current thinking. We may revise this guidance and adapt our policy approach as the technology matures and the regulatory oversight has been optimized.

Guidance for implementation

Health Canada considers product lifecycle information to be essential in demonstrating the safety and effectiveness of an MLMD. From our perspective, the MLMD lifecycle includes the following components:

Figure 1 gives a visual overview of the content areas discussed in this document. The iterative components reflected in this lifecycle schematic are not mutually exclusive and may not occur in the order indicated.

Figure 1: MLMD product lifecycle
Figure 1. Text version below.
Figure 1: Text description

The MLMD product lifecycle is represented in this diagram with 9 components, illustrating an iterative process where each stage is not mutually exclusive. The components are:

  • good machine learning practice
  • design
  • risk management
  • data selection and management
  • development and training
  • testing and evaluation
  • clinical validation
  • transparency
  • post-market performance monitoring

Good machine learning practice

Good machine learning practice (GMLP) is important when designing, developing, evaluating, deploying and maintaining an MLMD. This helps to ensure safe, effective and high-quality medical devices. Additional information can be found in the Good Machine Learning Practice for Medical Device Development: Guiding Principles.

The evidence provided with an application for an MLMD should include a description of how the manufacturer has considered GMLP within the organization and implemented it throughout the product lifecycle, as applicable. If applicable, this description should outline the quality practices implemented to ensure that the PCCP change description will be realized by following the PCCP change protocol.

Sex and gender-based analysis plus

Sex and gender-based analysis plus (SGBA Plus or GBA Plus) is an analytical process used to assess how a product or initiative may affect diverse groups of people. This process can be incorporated into the risk management approach used across the lifecycle of the device.

Evidence demonstrates that biological, economic and social differences between diverse groups of people contribute to differences in health risks and outcomes, their use of health services and how they interact with the health system. Integrating SGBA Plus throughout the lifecycle of a medical device will lead to more equitable health outcomes for Canada's diverse population.

Over the lifecycle of the MLMD, manufacturers should apply SGBA Plus and consider the unique anatomical, physiological and identity characteristics of patients. This includes:

Design

Indications for use, intended use and contraindications

For any Class II, III or IV MLMD, the intended use or medical purpose should be made clear in the application. Provide all relevant information, including the following:

Device description

Provide a detailed description of the MLMD, including any ML systems used to achieve an intended medical purpose. Consider including the following information in the description of the device or software:

Predetermined change control plan:

A PCCP is the documentation intended to describe changes that will be made to the MLMD as well as the device bounds or limits (for example, performance envelope) and how the changes will be implemented and assessed. The changes described in a PCCP include those that would otherwise require a medical device licence amendment application for a significant change prior to implementation. A PCCP includes a change description, change protocol and impact assessment. If included, a PCCP is considered part of the device design.

PCCPs should be risk-based and supported by evidence, take a total product lifecycle perspective and provide a high degree of transparency. Additional information can be found in the Predetermined change control plans for machine learning-enabled medical devices: Guiding principles.

All modifications listed in a PCCP must ensure that the device continues to operate within its intended use. Changes listed in a PCCP should not include changes to the medical conditions, purposes or uses of an MLMD. Such changes require a medical device licence amendment application prior to implementation.

Appropriate changes to list in a PCCP include those where pre-authorization addresses a known risk while upholding the benefits to the patient. An example of such a change would be the maintenance or improvement of performance to address the risk of ML performance degradation over time. This performance degradation can be due to changes to the environment, such as to the input data or the relationship between the input variables and the target variable.

The use of a PCCP allows timely and ongoing management of risks while ensuring device safety and effectiveness.

A PCCP consists of the following 3 components:

  1. Change description
  2. Change protocol
  3. Impact assessment

The detailed PCCP, if applicable to the device, should:

1) Change description

The change description is the documentation that characterizes the device and the proposed changes. It includes, but is not limited to:

2) Change protocol

The change protocol describes the set of policies and procedures that control how changes, as outlined in the change description, will be implemented and managed. The protocol ensures ongoing safety and effectiveness.

Aspects of the change protocol that may need to be part of the licence application include, but are not limited to, plans for ongoing:

Each change in the change description should be clearly traceable to the relevant aspects of the change protocol (for example, through a traceability table).

3) Impact assessment

The impact assessment outlines the potential influence and implications of the changes listed in the PCCP. Aspects it should consider include, but are not limited to:

Risk management

Manufacturers should conduct the necessary risk management across the lifecycle of the MLMD and consider providing descriptions of:

The following items, as applicable, should be considered in the risk analysis:

When performing the risk management for an MLMD, consider referring to the current version of the following resource:

Data selection and management

The quality of the datasets used to develop an MLMD will influence the quality of the device. The data characterization is an important aspect of the device evaluation.

When describing the selection and management of data for an MLMD, consider providing the following elements:

Development and training

Consider providing descriptions of the ML development, training and tuning approaches, including the following elements:

Testing and evaluation

Consider including the following information on ML system performance testing as part of the performance/bench testing or software verification and validation:

Clinical validation

In a medical device licence application for a Class III or IV MLMD, manufacturers should provide the appropriate clinical evidence, including clinical validation studies, to support the safe and effective clinical use of their device. This information should be available upon request for Class II MLMD.

For more information on clinical evidence requirements, consult:

The clinical evidence should support that the trained, tuned and tested ML system, and the MLMD with that ML system, is safe and effective and performs as intended in the intended population.

Examples of clinical evidence that can be used include:

The clinical evidence should accompany a justification to support the level of evidence. This justification should establish that the evidence is sufficient to demonstrate:

Transparency

Transparency requirements should consider the various stakeholders involved in a patient’s health care across the lifecycle of the device (for example, patients, users, health care providers and regulators). Relevant aspects include the intended use and information about device development and performance and (when available) how an output or result is reached or the basis for a decision or action (sometimes referred to as explainability, which is one part of transparency). Additional information can be found in the Transparency for machine learning-enabled medical devices: Guiding principles.

Transparency should be considered throughout the device lifecycle and within the:

Transparency should continue to be considered during device use and upon device changes.

The following subsection outlines transparency considerations for MLMD labelling for the end-user.

Labelling

Manufacturers should provide copies of the labelling, including those pertaining to the ML system, within applications for Class II, III and IV MLMDs. Health Canada will review the labels against the requirements outlined in sections 21, 22 and 23 of the regulations.

Directions for use or instructions for use for the device, as well as product brochures, websites and marketing material with claims related to the ML system, are all considered labelling.

The following ML system information should be considered for inclusion in MLMD labelling, as applicable:

Manufacturers should consider including a structured summary of the key ML system information in the labelling (sometimes referred to as a model card, data card or model facts).

Post-market monitoring

Manufacturers should consider including a description of the processes, surveillance and performance monitoring plans and risk mitigations in place to ensure ongoing performance and inter-compatibility of the ML system.

This should consider the impact on ML system outputs or clinical workflows that could result from:

This should include references to any related information in the risk analysis and the PCCP, if applicable.

Licence terms and conditions

Terms and conditions (T&Cs) may be imposed on some medical device licences. This can help ensure that the device continues to meet the applicable safety and effectiveness requirements of the regulations after it's been approved.

As per subsection 36(2) of the regulations, the Minister may impose T&Cs requiring:

As per subsection 36(3) of the regulations, the Minister may amend T&Cs imposed on a medical device licence to take into account any new development with respect to the device.

The holder of a medical device shall comply with T&Cs of the licence as per subsection 36(4).

The level of risk, uncertainty and/or complexity of a specific situation will be considered when imposing or amending T&Cs, and when determining requirements for individual T&Cs.

Related links

Page details

Date modified: