Draft guidance: Pre-market guidance for machine learning-enabled medical devices

Download the alternative format
(PDF format, 687 KB, 17 pages)

Organization: Health Canada

Date published: 2023-09-18

On this page

Introduction

Artificial intelligence (AI) is a broad term for a category of algorithms and models that perform tasks and exhibit behaviours such as learning and making decisions and predictions. Machine learning (ML) is the subset of AI that allows ML training algorithms to establish ML models when applied to data, rather than models that are explicitly programmed.

Medical devices that use ML, in part or in whole, to achieve their intended medical purpose are known as machine learning-enabled medical devices (MLMD). "Medical purpose" refers to parts (a) through (e) of the "device" definition within the Food and Drugs Act (act). MLMD are subject to the act and associated Medical Devices Regulations (regulations).

In this guidance, "transparency" describes the degree to which appropriate and clear information about a device (that could impact risks and patient outcomes) is communicated to stakeholders. Transparency is an important aspect of the device's safety and effectiveness, and helps stakeholders make informed decisions.

This guidance introduces the concept of a predetermined change control plan (PCCP). A PCCP provides a mechanism for Health Canada to address cases where the regulatory pre-authorization of planned changes to ML systems is needed to address a known risk.

In the face of uncertainties and risks associated with ML and PCCPs, the ongoing safety and effectiveness of marketed MLMD can be strengthened by including terms and conditions (T&Cs) on medical device licences, as appropriate.

Health Canada has adopted the MLMD terms and definitions used by the International Medical Device Regulators Forum (IMDRF). Manufacturers are encouraged to review this document:

In this guidance, "ML training algorithm" refers to the software procedure that establishes the parameters of an ML model by analyzing data. The "ML model" represents a mathematical construct that generates an inference or prediction based on new input data and is the result of an ML training algorithm learning from data. The "ML system" refers to an ML-enabled software that meets the definition of medical device as per Section 1 of the regulations, including ML models and the associated ML training algorithms.

Scope and application

This document provides guidance to manufacturers who are submitting a new or amendment application for Class II, III and IV MLMD under the regulations.

The information in this guidance relates to the ML system of an MLMD. It does not cover the non-ML information required in a medical device licence application.

Manufacturers should also consult other relevant guidance relating to medical devices, including the following:

Policy objective

This guidance outlines supporting information to consider when manufacturers are demonstrating the safety and effectiveness of an MLMD:

Policy statements

An MLMD can be standalone software that meets the definition of a medical device. It can also be a medical device that includes software that meets the definition of a medical device.

An MLMD can be an in vitro diagnostic device (IVDD) or a non-IVDD. The risk classification of an MLMD can range from Class I to Class IV.

Manufacturers should clearly state that the device uses ML in their cover letter for all Class II, III and IV applications for an MLMD. Furthermore, for MLMDs that have a PCCP, manufacturers should clearly state in their cover letter that their device includes a PCCP. Excluding such statements could delay the application process.

Manufacturers should include a justification for the proposed medical device classification applied to the MLMD. This justification should reference the classification rules outlined in Schedule 1 of the regulations.

Medical devices must meet the applicable requirements of sections 10 to 20 of the regulations. Manufacturers must ensure that objective evidence is available to support the intended use of the MLMD, the safety and effectiveness of the device and the associated claims.

An application must demonstrate that the MLMD (including the PCCP, as appropriate):

Class II, III and Class IV applications must include the information listed in section 32 of the regulations. Additional information may be requested at any time during our review of an application (new or amendment) or after a device has been licensed.

Health Canada understands that manufacturers may use a variety of information, methodologies and evidence to demonstrate that their MLMD is safe and effective. Additionally, different intended uses or risk profiles may require different types or levels of evidence. As such, we have outlined information for consideration rather than prescribing the required information for all scenarios.

The guidance for implementation section of this document outlines the information to consider for an MLMD. If any of the information identified in this section is not available, manufacturers should offer a justification or provide alternative information, as applicable.

Data referred to or used by manufacturers should adequately represent the Canadian population and clinical practice. Any data used to develop the MLMD or demonstrate a device's safety and effectiveness should reflect the population for whom the device is intended. For example, this could include consideration of skin pigmentation, biological differences between sexes and other identity-based factors.

For those devices that are authorized with a PCCP, subsequent changes made according to the authorized PCCP do not require that you submit a medical device licence amendment application. PCCP-driven changes are subject to relevant post-market regulatory oversight.

For amendments to a device that are outside of an authorized PCCP, including changes to the PCCP itself, the regulations and relevant guidance documents should be consulted before implementation. It's important to determine whether the change constitutes a significant change and requires an application for a medical device licence amendment.

A PCCP may be submitted with applications for a new medical device licence or a medical device licence amendment.

This guidance represents Health Canada's current thinking. We will revise this guidance and adapt our policy approach as the technology matures and the regulatory oversight has been optimized.

Guidance for implementation

Health Canada considers product lifecycle information to be essential in demonstrating the safety and effectiveness of an MLMD. From our perspective, the MLMD lifecycle includes the following components:

Figure 1 gives a visual overview of the content areas discussed in this document. However, the iterative components reflected in this circular lifecycle schematic are not mutually exclusive and may not occur in the order indicated.

Figure 1: MLMD product lifecycle
Figure 1. Text version below.
Figure 1: Text description

The MLMD product lifecycle is represented in a circle with 8 stages, illustrating an iterative process. The stages are:

  • design
  • risk management
  • data selection and management
  • development and training
  • testing and evaluation
  • clinical validation
  • transparency
  • post-market performance monitoring

Good machine learning practice

Good machine learning practice (GMLP) is important when designing, developing, evaluating, deploying and maintaining an MLMD. This helps to ensure safe, effective and high-quality medical devices.

The evidence provided with an application for an MLMD should include a description of how the manufacturer has adopted GMLP within the organization and implemented it throughout the product lifecycle. If applicable, this description should outline the quality practices implemented to ensure that the PCCP change description will be realized by following the PCCP change protocol.

Pre-determined change control plan: concept

A PCCP is the documentation intended to characterize a device and its bounds, the intended changes to the ML system, the protocol for change management and the change impacts. If included, a PCCP is considered part of the device design.

PCCPs should be risk-based and supported by evidence, take a total product lifecycle perspective and provide a high degree of transparency.

All modifications listed in a PCCP must ensure that the device continues to operate within its intended use. Changes listed in a PCCP should not include changes to the medical conditions, purposes or uses of an MLMD. Such changes require a medical device licence amendment application prior to implementation.

Appropriate changes to list in a PCCP include those where pre-authorization is necessary to address a known risk while upholding the benefits to the patient. An example of such a change would be the maintenance or improvement of performance to address the risk of ML performance degradation over time. This performance degradation can be due to changes to the environment, such as to the input data or the relationship between the input variables and the target variable.

The use of a PCCP allows timely and ongoing management of risks while retaining high regulatory standards to ensure device safety and effectiveness.

Sex and gender-based analysis plus

Sex and gender-based analysis plus (SGBA Plus or GBA Plus) is an analytical process used to assess how a product or initiative may affect diverse groups of people. This process can be incorporated into the risk management approach used across the lifecycle of the device.

Evidence demonstrates that biological, economic and social differences between diverse groups of people contribute to differences in health risks and outcomes, their use of health services and how they interact with the health system. Integrating SGBA Plus throughout the lifecycle of a medical device will lead to more equitable health outcomes for Canada's diverse population.

Over the lifecycle of the MLMD, manufacturers should apply SGBA Plus and consider the unique anatomical, physiological and identity characteristics of patients. This includes:

Design

Indications for use, intended use and contraindications

For any Class II, III or IV MLMD, the intended use or medical purpose should be made clear in the application. Provide all relevant information, including the following:

Device description

Provide a detailed description of the MLMD, including any ML systems used to achieve an intended medical purpose. Consider including the following information in the description of the device or software:

Predetermined change control plan: content

A PCCP consists of the following 3 components:

  1. Change description
  2. Change protocol
  3. Impact assessment

The detailed PCCP, if applicable to the device, should:

Change description

The change description is the documentation that characterizes the device and the proposed changes. It includes:

Change protocol

The change protocol describes the set of policies and procedures that control how changes, as outlined in the change description, will be implemented and managed. The protocol ensures ongoing safety and effectiveness.

Aspects of the change protocol that may need to be part of the licence application include plans for ongoing:

Each change in the change description should be clearly traceable to the relevant aspects of the change protocol (for example, through a traceability table).

Impact assessment

The impact assessment outlines the potential influence and implications of the changes listed in the PCCP. It should consider:

Risk management

Manufacturers should conduct the necessary risk management and consider providing descriptions of:

The following items, as applicable, should be considered in the risk analysis:

When performing the risk management for an MLMD, consider referring to the current version of the following resource:

Data selection and management

When describing the selection and management of data for an MLMD, consider providing the following elements:

Development, training and tuning

Consider providing descriptions of the ML development, training and tuning approaches, including the following elements:

Testing and evaluation

Consider including the following information on ML system performance testing as part of the performance/bench testing or software verification and validation:

Clinical validation

In a medical device licence application for a Class III or IV MLMD, manufacturers should provide the appropriate clinical evidence, including clinical validation studies, to support the safe and effective clinical use of their device. This information should be available upon request for Class II MLMD.

For more information on clinical evidence requirements, consult:

The clinical evidence should support that the trained, tuned and tested ML system, and the MLMD with that ML system, is safe and effective and performs as intended in the intended population.

Examples of clinical evidence that can be used include:

The clinical evidence should accompany a justification to support the level of evidence. This justification should establish that the evidence is sufficient to demonstrate:

Transparency

Transparency requirements should consider the various stakeholders involved in a patient's healthcare across the lifecycle of the device (for example, patients, users, healthcare providers and regulators).

Transparency should be considered throughout the device lifecycle, including in the:

The following subsection outlines transparency considerations for MLMD labelling for the end-user.

Labelling

Manufacturers should provide copies of the directions for use or instructions for use for the device, including those pertaining to the ML system. Health Canada will review the labels against the requirements outlined in sections 21, 22 and 23 of the regulations.

The following ML system information should be considered for inclusion in MLMD labelling, as applicable:

Product brochures, websites and marketing material with claims related to the ML system should also be provided, as these are also considered labelling.

Post-market performance monitoring

Manufacturers should consider including a description of the processes, surveillance plans and risk mitigations in place to ensure ongoing performance and inter-compatibility of the ML system.

This should consider the impact on ML system outputs or clinical workflows that could result from potential changes in the inputs to the ML model, changes to how the ML system outputs are handled by compatible products or any other relevant information. This may be addressed as part of the risk analysis and the PCCP, if applicable.

Terms and conditions

Terms and conditions (T&Cs) may be imposed on some medical device licences. This can help ensure that the device continues to meet the applicable safety and effectiveness requirements of the regulations after it's been approved.

As per section 36(2) of the regulations, the Minister may impose T&Cs requiring:

As per subsection 36(3) of the regulations, the Minister may amend T&Cs imposed on a medical device licence to take into account any new development with respect to the device.

The holder of a medical device shall comply with T&Cs of the licence as per subsection 36(4).

The level of risk, uncertainty and/or complexity of a specific situation will be considered when imposing or amending T&Cs, and when determining requirements for individual T&Cs.

Related links

Page details

Date modified: