The U.S. Food and Drug Administration’s Center for Devices and Radiological Health (CDRH) has released a comprehensive discussion paper outlining a proposed regulatory approach for generative artificial intelligence (AI) in the medical field. This move signals a pivotal shift in how the federal government intends to oversee a technology that is rapidly transforming clinical workflows, diagnostic accuracy, and patient engagement. Published on August 19, 2026, the paper arrives at a critical juncture when healthcare providers are increasingly integrating large language models (LLMs) and multimodal AI into frontline care, despite lingering questions regarding the reliability and safety of these "black box" systems.
The FDA’s latest communication acknowledges that while generative AI holds transformative promise for reducing administrative burdens and enhancing precision medicine, it introduces "unique risks" that traditional software-as-a-medical-device (SaMD) frameworks are ill-equipped to handle. Unlike traditional AI, which typically follows rigid algorithmic paths to produce predictable outputs, generative AI is characterized by its ability to accept open-ended inputs, perform a vast array of multi-step tasks, and produce variable outputs that can change even when given the same prompt. This variability, combined with the well-documented phenomenon of "hallucinations"—where AI generates factually incorrect but highly convincing information—has necessitated a reevaluation of the agency’s premarket and postmarket requirements.
A New Tiered Risk Framework for Generative Systems
Central to the FDA’s discussion paper is a proposed risk framework that categorizes generative AI-enabled devices based on the level of autonomy and the potential impact on patient safety. The agency suggests a spectrum of risk that begins with "informational and non-directive" tools and culminates in "fully autonomous" medical systems.
In the lowest risk category are devices that provide background information or predictive scores without mandating a specific clinical path. An example provided by the CDRH includes software that analyzes a patient’s historical health record to generate a risk score for a future cardiovascular event. In this scenario, the AI acts as a supplemental tool for the clinician, who remains the primary decision-maker.
The risk profile escalates significantly when AI begins to direct clinicians or patients toward specific actions. The paper distinguishes between "directive" tools—such as an AI that suggests a specific dosage for a medication—and "autonomous" tools that perform actions without human intervention. The CDRH acknowledged that the context of use is as important as the technology itself. For instance, an AI that autonomously prescribes antibiotics for a minor infection carries a different risk profile than one designed to initiate life-saving thrombolytic therapy during a stroke workflow. The latter requires a much higher threshold of evidence and more rigorous safety buffers due to the narrow window for error and the severity of potential adverse outcomes.
Addressing the Challenge of "Hallucinations" and Performance Drift
One of the most significant hurdles identified by the FDA is the inherent instability of generative AI outputs. In traditional medical device manufacturing, consistency is a hallmark of safety. However, generative models are designed for creativity and flexibility, which can lead to "performance degradation" over time as the model interacts with new data—a phenomenon often referred to as "model drift."
The discussion paper highlights the difficulty in defining the "intended use" boundaries for a device that can theoretically answer any question or generate any type of content. If a clinician uses a generative AI tool designed for radiology to ask about a dermatological condition, the device may provide an answer that it was never validated to give. To mitigate this, the FDA is exploring requirements for "guardrails" that would force the AI to decline queries outside its validated scope and provide transparent explanations for how it arrived at a specific conclusion.
Furthermore, the agency is tackling the issue of "hallucinations" head-on. Because generative AI can produce authoritative-sounding medical advice that is entirely fabricated, the FDA is considering new labeling requirements. These would mandate that manufacturers provide clear warnings regarding the probabilistic nature of the outputs and require human-in-the-loop verification for high-stakes clinical decisions.
A Competency-Based Approach to Premarket Evaluation
To ensure that these devices are safe before they reach the public, the CDRH is proposing a "competency-based" evaluation model. This approach moves away from the static validation of code and toward a more dynamic assessment of the AI’s "skills" or "competencies."
Under this proposed model, generative AI devices would be subjected to rigorous benchmarking against standardized datasets. Once a device clears these benchmarks, it would move into clinical setting testing, where the final, user-facing version of the software is evaluated in real-world scenarios. The goal is to observe how the AI interacts with human users, checking for "automation bias"—the tendency for humans to over-rely on automated suggestions—and ensuring that the AI maintains its accuracy across diverse patient populations.
This competency-based model reflects a shift toward a "total product lifecycle" (TPLC) approach. By testing the AI’s ability to handle complex, multi-layered medical queries rather than just checking its underlying code, the FDA hopes to create a more realistic safety profile for tools that are constantly evolving.
The Trade-off: Premarket Uncertainty vs. Postmarket Vigilance
Perhaps the most controversial element of the discussion paper is the FDA’s suggestion that it may be willing to accept "greater premarket uncertainty" regarding the benefits and risks of a generative AI-enabled device in exchange for more robust postmarket monitoring.
This represents a departure from the traditional regulatory philosophy, which generally requires exhaustive proof of safety and efficacy before a device is commercialized. However, the CDRH argues that the rapid pace of AI development may make exhaustive premarket testing impossible without stifling innovation. Instead, the agency envisions a system where manufacturers are held to stringent post-launch reporting requirements.
According to the paper, the primary responsibility for postmarket monitoring would fall on the manufacturers. They would be required to track real-world performance, identify instances of "drift" or "hallucination," and report adverse events with a level of granularity not previously required. The CDRH also noted that clinicians, healthcare institutions, and payers would play a vital role in this ecosystem, acting as the frontline observers who can flag when a tool begins to provide suboptimal care.
Chronology of FDA Action on Digital Health and AI
The release of this discussion paper is the latest step in a multi-year effort by the FDA to modernize its digital health oversight. The journey began in earnest in 2021 with the publication of the "Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan."
In 2024, the CDRH’s Digital Health Advisory Committee held its inaugural meeting to discuss the unique challenges posed by generative AI. This was followed by a 2025 session focused specifically on digital mental health devices, where the committee debated the ethics of using LLMs for therapeutic interventions and crisis management.
The 2026 discussion paper synthesizes the feedback from those meetings and sets the stage for formal guidance documents. The FDA has indicated that it will accept public comments on this paper for 90 days, after which it will begin drafting the regulatory framework that will likely govern the next decade of medical AI innovation.
Industry and Stakeholder Reactions
While official responses from industry giants are still being formulated, early reactions from the medtech community suggest a cautious welcome for the FDA’s transparency. Organizations like AdvaMed (the Advanced Medical Technology Association) have previously advocated for a "least burdensome" regulatory approach that allows for iterative updates to AI software.
However, patient advocacy groups have expressed concerns about the "premarket uncertainty" clause. "The idea of using the public as a testing ground for evolving AI models is inherently risky," said a spokesperson for a leading patient safety non-profit. "Postmarket monitoring is notoriously difficult to enforce, and by the time a problem is identified, thousands of patients could already be affected."
Conversely, tech developers argue that a rigid, slow-moving approval process would render AI tools obsolete by the time they reach the market. They point to the "versioning" nature of AI, where a model like GPT-4 or its successors can be updated weekly to improve accuracy and safety.
Broader Impact and Global Implications
The FDA’s stance on generative AI is expected to have a ripple effect across the global regulatory landscape. As the U.S. formalizes its approach, international bodies like the European Medicines Agency (EMA) and the International Medical Device Regulators Forum (IMDRF) are likely to look toward the CDRH’s framework as a blueprint.
The implications for healthcare equity are also significant. The FDA paper explicitly mentions the need for AI models to be tested across diverse demographics to prevent algorithmic bias. If generative AI is trained primarily on data from wealthy, urban populations, its "competency" in treating underserved or rural communities may be compromised.
As the healthcare industry stands on the precipice of an AI-driven revolution, the FDA’s discussion paper serves as both a roadmap and a warning. It acknowledges that the era of static, predictable medical devices is coming to an end, replaced by a new generation of dynamic, intelligent, and inherently unpredictable tools. The challenge for the agency will be to foster an environment where these tools can flourish without compromising the fundamental mandate of protecting public health.
