FDA permits pre-approval for changes to AI devices via Predetermined Change Control Plans

Background

An artificial intelligence-enabled device software function (AI-DSF) uses AI to analyze data and make predictions or decisions based on that analysis. Examples of AI and machine learning (ML) applications in medicine include earlier disease detection and diagnosis, development of personalized diagnostics and therapeutics, and development of assistive functions to improve the use of devices with the goal of enhancing user and patient experience.

Modifications to an AI-DSF that could affect the safety or effectiveness of the device require premarket authorization from FDA. However, in order to support the iterative development of AI-DSFs, FDA has described how a “Predetermined Change Control Plan” (PCCP) can be included in a premarket submission for a device that is [or includes] an AI-DSF. By submitting a PCCP, a sponsor may obtain pre-approval for intended modifications (and their method of implementation) to an AI-DSF without necessitating additional marketing submissions for each modification delineated and implemented in accordance with the PCCP.

In April 2023, FDA issued the draft guidance “Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence/Machine Learning (AI/ML)-Enabled Device Software Functions,” aiming to clarify the process by which AI/ML device sponsors could seek approval for modifications in advance by submitting a “PCCP document,” which would describe the anticipated changes and how they will be tested, implemented, and subsequently monitored. We summarized the draft guidance online here, explaining:

• the types of modifications that would generally be considered acceptable for authorization within a PCCP;
• what must be included in a PCCP premarket authorization request, including in a new “Description of Modifications” section;
• how a PCCP “Modification Protocol” should summarize a manufacturer’s data management practices, re-training practices, performance evaluation protocols, and update procedures; and,
• how a PCCP “Impact Assessment” should document the assessment of the benefits and risks of implementing a PCCP for an AI-DSF, as well as the mitigations of those risks.

Changes in the final guidance

The recently issued final version of the guidance mostly resembles the draft in terms of the key principles. FDA stresses that PCCPs must include the three key sections previously outlined (description of modifications, protocol for making modifications, and impact assessment). The agency also maintains that PCCPs are relatively narrow in scope, reiterating that a planned modification to a device which could significantly affect safety or effectiveness and was not covered by a pre-existing PCCP will require a new marketing submission.

One interesting difference form the draft is the final guidance’s omission of “/Machine Learning” from the title. Indeed, FDA repeatedly clarifies the broad scope of the final guidance, iterating that the recommendations for PCCPs “are intended to be broadly applicable to all AI-enabled devices,” even though “the majority of marketing submissions containing PCCPs that FDA has reviewed are submissions for devices that incorporate the subset of AI known as ML.”

Additionally, we observed the following changes in the final version of the guidance:

• The final guidance recommends that, to avoid misleading users about which device features are already available and which are still future options, the labeling should not include details about planned future modifications to the AI-DSF (i.e., the changes being authorized in the PCCP). New unique device identifiers (UDIs) are required, where applicable, when a new device version and/or model is released.
• There are several new references to the need for testing data to “be representative of the proposed intended use populations (e.g., with respect to race, ethnicity, disease severity, gender, age, or others, as appropriate) and intended environments.” This continues the trend of FDA’s heightened focus on ensuring health care equity and diversity in clinical trials, as we recently highlighted online here.
  • FDA advises that additional “methods to mitigate bias may be helpful, such as cross-validation, bootstrapping, bagging, ensembling, and the use of synthetic or augmented data” (emphasis added).
• The final guidance places greater import on sponsors ensuring that the data used to test the AI-DSF is separate and independent from the development process used to train and tune the AI-DSF. Such increased emphasis is consistent with our experience in product-specific discussions with FDA about devices incorporating AI, as the Agency becomes more aware of the importance of independence to ensure adequate performance of an AI-enabled device.
• FDA adds a section emphasizing the importance of “version control” and maintenance of a PCCP, placing stakeholders on notice that review of a marketing submission that includes a PCCP “will be interactive.”
  • FDA adds guidance for a manufacturer who would like to modify their PCCP for a previously authorized device with a PCCP, again directing sponsors with questions to the agency’s Q-Submission Program.
• FDA recommends that the PCCP should specify how planned changes will be implemented (automatically, manually, or a combination of both), as well as how the manufacturer plans to communicate information about updates to device users. The PCCP should also describe the expected frequency of updates, which could range from infrequent (resulting in a device that is primarily “locked”) to continuously implemented as the device adapts to new data during its use.
  • FDA adds that it “anticipates that manufacturers will monitor their device’s safety (e.g., adverse events) and effectiveness (e.g., performance) over time as modifications are implemented consistent with their authorized PCCP.”
• FDA asks sponsors to consider what information they should provide to users “about the safety performance of the device following implementation of modifications,” and if the answer is “none,” to explain why it is not necessary.
• FDA now asks AI sponsors to provide criteria and/or a plan to roll-back an update to reset devices to a previous version, if applicable.

CDRH risk/benefit analysis of non-device software functions

FDA’s Center for Devices and Radiological Health (CDRH) also published last week a “Report on Risks and Benefits to Health of Non-Device Software Functions,” which FDA is required to create annually pursuant to the 21st Century Cures Act. The report examines information on the risks and benefits to health associated with certain software functions that were omitted from CDRH’s regulatory oversight per section 520(o)(1) of the Federal Food, Drug, and Cosmetic Act (FDCA), including software functions:

• for the “administrative support of a health care facility,”
• for “maintaining or encouraging a healthy lifestyle,”
• to serve as electronic patient records,
• to transfer, store, display, or convert format of clinical data or other device data/results, or
• serving as clinical decision support and meeting certain additional criteria.

Similar to the 2022 version of the report, this year’s report found more benefits than risks to patient safety and health related to these software functions. However, FDA notes, that because there is no requirement to disclose adverse events from non-device software to the agency, the report may be underrepresenting adverse events.

Relying on peer-reviewed literature, public comments, and interviews with experts, FDA’s 2024 analysis reported numerous findings, a few of which are highlighted below:

• AI is a “promising” tool for producing first drafts of administrative documents for human review. However, AI is not currently able to engage in independent analysis of administrative functions.
• Mobile health combined with multiple emerging technologies can potentially improve physical activity. Patients could particularly benefit from mobile apps that featured personalized goal setting, selective feedback, motivational messages, and interactions with clinicians. Of interest, FDA noted that use of wearables and wearable-related interventions led to an initial increase in exercise and physical activity, but use decreased over time.
• Although researchers have observed a “noticeable” impact of digital health interventions on women’s healthy behaviors, there is a need to address digital equity (e.g., subpopulations may be unable to take advantage of digital tools due to cost).

Next steps

We expect the final PCCP guidance to encourage innovation and delivery of AI-enabled medical devices by enabling manufacturers to make certain updates to their devices without re-engaging FDA prior to their implementation. However, the guidance also puts the onus on medical device manufacturers to plan ahead and approach FDA through the Q-submission process to discuss a proposed PCCP, and then include a lengthy change control section in their marketing submissions in order to obtain pre-authorization for such updates, as well as commit to rigorous post-market monitoring of how their authorized device performs in the real world.

If you wish to submit a comment, or have any questions on PCCPs, AI-enabled medical device regulations, or FDA product submissions more generally, please contact the Hogan Lovells attorney with whom you regularly work or any of the authors of this alert.

Authored by Kelliann Payne, Suzanne Levy Friedman, Eriko Yoshimaru, and Evelyn Tsisin.