Breaking Down the New FDA Guidance: Human Factors in AI-Enabled Medical Devices Software

Introduction to human factors in AI-enabled medical devices software

Artificial Intelligence (AI) in medical devices has introduced new possibilities, yet it also raises challenges in usability. The FDA’s draft guidance on AI-enabled medical devices highlights the criticality of human factors (HF) engineering throughout the total product lifecycle. If you want to learn more about human factors in the medical device or pharmaceutical industry, I recommend you to check our article “Introduction to Human Factors in Medical Devices and Combination Products”.

In the following sections, I will share my personal insights and expert analysis as a human factors professional on the new FDA AI guidance, which has sparked significant discussions within the industry.

The FDA’s Perspective on Human Factors in AI-Enabled Devices

As a human factors expert, I find very interesting that the new AI guidance references the term 'Human Factors' 34 times (including in some footnotes) and 'Usability' another 34 times, though not always in the same context. It is not common to find an FDA guidance, outside the traditional HF ones, that makes such frequent and deliberate calls to the Human Factors or Usability field. In my view, this recognizes the  role of human factors  in ensuring safety and effectiveness in medical devices, but it also shows how seriously the FDA is about recognizing this AI-enabled devices as acceptable solutions—a focus I didn’t entirely anticipate, especially considering how permissive the FDA has sometimes been in software-related human factors (see premarket device-software guidance), but I am really happy to see this insistence on robust usability and safety considerations. 

Now, let's go back to the guidance. The FDA emphasizes that HF integration ensures that users—from clinicians to patients—can effectively and safely interact with AI-enabled devices. The guidance outlines key considerations:

1. User Interface (UI) Design:

Regulatory submissions should prioritize detailed user interface (UI) descriptions, encompassing graphical elements, operational sequences, and user workflows. In my professional view, this focus is essential to ensure transparency, as highlighted in Appendix B of the guidance. The aim is to enhance user comprehension of AI outputs and streamline decision-making processes, which are critical for the safe and effective use of AI-enabled devices.

2. Usability Validation:

Usability evaluations should ensure that all users, regardless of their expertise, can safely and effectively interact with the device. This includes a focus on comprehensive testing to address diverse user needs and scenarios. The guidance emphasizes the importance of human factors validation testing for tasks considered critical—those where errors could result in significant harm—ensuring that risks are identified and mitigated through robust design and usability strategies.

3. Risk Mitigation:

Risk assessments should comprehensively address user interactions, placing particular emphasis on identifying and mitigating risks related to information interpretation errors. FDA's guidance  prioritizes design solutions over reliance on training or labeling as the primary means of risk mitigation which is aligned with ISO14971 and IEC62366 recommendations. By embedding safety and usability into the device design, manufacturers can proactively address potential hazards, ensuring that the devices are intuitive and error-resistant across diverse user scenarios.

Validation activities for AI-enabled medical devices

According to the FDA, the validation of AI-enabled medical devices must ensure that it is safe and effective, this is exactly the same requirements presented in the human factors guidance for medical devices and it is the traditional process followed for over a decade in medical devices development.

I find it very interesting that the guidance distinguishes between human factors validation and usability evaluation, despite stating that "usability describes whether the device can be used safely and effectively by the intended users, including whether users consistently and correctly receive, understand, interpret, and apply information related to the AI-enabled device." In my view, this separation indicates that the FDA expects manufacturers to implement usability programs in alignment with IEC62366 standards, even for devices whose severity classification does not mandate a formal human factors validation. In addition, the guidance reference the main FDA human factors guidance "Applying human factors and usability engineering to medical devices", which is the bread and butter for any human factors practitioner, claiming that even thought the guidance provides recommendations for device with critical tasks, it can be helpful to demonstrate appropriate control over other risks. This approach reflects an emphasis on ensuring comprehensive usability practices regardless of risk classification.

Usability to support control of risks in AI-enabled medical devices software

In the Appendix D of the guidance, the FDA clearly states that even when a device do not have critical tasks associated to it's use and manufacturers don't need to submit a HF validation report, they should consider following HF practices recommended in the HF guidance as a way of evaluating usability of the AI-enabled medical device. 

In the absence of a required HF validation, the HF process must be used to demonstrate effectiveness of the proposed risk control measures. This is something we are already doing in the industry frequently and, according to IEC62366, the HF process must confirm that the risk control measures implemented in labeling must be identifiable, understandable and support safe and effective use of the device. 

I must admit that I am a little bit concern about this, I do believe that it is a good approach, but at the same time I can expect really lengthly human factors evaluations since the technology is not mature enough to not rely on labeling-based mitigations at this moment. I highly recommend any manufacturer to engage soon with the available HF providers to get advise on how to define proper risk mitigations that can be validated. At PharmaHF we don't provide any service for manufacturers, since the only purpose of this blog is informative, but we love to have a chat about human factors so feel free to contact us in our email if you want to chat about this or any other topic.

Reflections about the role of human factors according to the new FDA guidance for AI-enabled medical device software

The new FDA guidance makes it clear that human factors or usability engineering is indispensable for the development of AI-enabled medical devices. It strongly recommends following HF processes for all AI medical devices, even those without critical tasks. This highlights the FDA's recognition of HF as a vital element in ensuring safe and effective device use.

Manufacturers of AI-enabled medical devices should engage human factors practitioners from the earliest stages of development. Failing to do so could jeopardize their ability to pass HF validations. Risk control measures must be designed to support safe and effective use, with usability evaluations serving as a key method for demonstrating their effectiveness. The manner in which these measures are established directly influences the success of validation efforts.