A radiologist in Houston pulls up a chest X-ray and watches an algorithm flag a suspicious nodule before she has finished her first sip of coffee. In Philadelphia, a Medicare Advantage enrollee opens a denial letter for a rehabilitation stay, unaware that an automated screening tool helped generate the decision. Across the country, artificial intelligence is weaving itself into the daily machinery of American health care, touching diagnostics, treatment planning, and insurance approvals alike.
Federal regulators have noticed. The Food and Drug Administration, the Centers for Medicare and Medicaid Services, and the Department of Health and Human Services have each released guidance, action plans, or proposed rules aimed at governing these tools. The White House has also weighed in: an October 2023 executive order on AI directed HHS to develop frameworks for responsible AI deployment in health care, including standards for safety testing, equity assessments, and the use of AI in benefits administration. Yet a stubborn gap remains between the pace of adoption and the evidence that any of it is making patients healthier. As of spring 2026, the regulatory scaffolding is rising fast. The proof that it protects anyone is still under construction.
A growing federal footprint
The FDA has built a formal oversight track for AI and machine-learning software classified as medical devices. The agency’s dedicated AI and software hub documents an oversight timeline stretching back to a 2019 discussion paper, through a 2021 action plan, and into final guidance on Predetermined Change Control Plans. That last piece matters because it lets manufacturers update their algorithms after clearance, provided they spell out in advance what can change, how changes will be controlled, and what verification and validation steps will follow. It is, in effect, a rulebook for software that never stops learning.
On the payer side, CMS has drawn its own lines. The agency’s responsible-use framework directs staff and contractors to assess bias risks, document limitations, and communicate uncertainty whenever AI tools inform agency operations. More consequentially for patients, CMS used its February 2024 Medicare Advantage final rule (CMS-4201-F) to insist that insurers cannot lean solely on algorithmic outputs when approving or denying care. Every coverage determination must include individualized clinical review. That requirement landed after high-profile reporting revealed that some insurers were using predictive models to cut short post-acute care stays, sometimes overriding the judgment of treating physicians.
HHS, meanwhile, has turned its attention to the security risks that accompany AI’s expanding data appetite. The Office for Civil Rights published a proposed update to the HIPAA Security Rule focused on cybersecurity. AI systems often require new vendor integrations, larger data flows, and cloud-based processing, all of which widen the attack surface for breaches. The proposal would tighten protections around those vulnerabilities by reinforcing risk-analysis obligations, clarifying expectations for business-associate agreements, and emphasizing contingency planning for cyber incidents that could disrupt clinical operations. As of spring 2026, the rule has not been finalized, leaving health systems to prepare for requirements whose final shape remains uncertain.
The evidence gap
Regulatory activity, however vigorous, is not the same as clinical proof. No publicly available longitudinal dataset tracks whether FDA-cleared AI devices have improved patient outcomes at scale. The agency maintains a safety-reporting portal for medical products, but aggregated adverse-event data specific to AI and ML-based software has not been released in a form that allows independent assessment of real-world harms or benefits.
Academic research has not filled the void. A 2022 BMJ analysis by Wong et al. examining AI-enhanced early warning scores against traditional clinical methods found mixed results, with the AI tools showing no clear advantage in improving patient outcomes. Related evaluations indexed by the National Library of Medicine echo that pattern of modest or uncertain benefit. The studies tend to examine narrow use cases at single institutions over short time horizons, and their outcome measures often focus on intermediate markers like prediction accuracy rather than mortality, quality of life, or financial burden. If AI tools are not consistently outperforming simpler approaches in controlled settings, the case for their rapid, broad deployment becomes harder to make on clinical grounds alone.
Cost-benefit data is similarly thin. Vendors market efficiency gains, but independent evaluations of workflow impact, staffing changes, or downstream utilization are sparse. CMS expects responsible use, yet it has not published compliance rates showing how many health plans or providers meet those expectations, or how often clinicians override algorithmic recommendations during coverage reviews. Without that transparency, the word “responsible” functions more as aspiration than as an auditable standard.
Accountability without a yardstick
The National Institute of Standards and Technology maintains an AI Risk Management Framework that CMS references for governance standards. Translating those high-level principles into measurable accountability across thousands of health systems, though, remains an unfinished project. Most organizations lack standardized reporting on model performance over time, disaggregated by patient demographics, which makes it difficult to detect algorithmic drift, emerging inequities, or unintended consequences. In practice, oversight often depends on local champions, internal committees, and vendor assurances rather than on uniform federal metrics.
The unfinished HIPAA Security Rule adds another layer of uncertainty. Health systems face the prospect of significant compliance investments, from upgrading technical safeguards to revising contracts and expanding security training, without knowing the final requirements, timelines, or enforcement mechanisms. Smaller providers, already stretched thin, may struggle to absorb those costs even as they are pushed to adopt more digital tools. The result is a landscape where the organizations with the fewest resources face the steepest compliance curves.
Where the guardrails stand as of spring 2026
For patients navigating the system right now, the most concrete protection comes from CMS: Medicare Advantage coverage decisions must reflect individualized clinical judgment, not just an algorithm’s score. That standard offers a clear line in settings where automated tools might otherwise be used to limit care.
The strongest public record comes from primary federal documents: the FDA’s guidance on algorithm changes, the CMS directives on responsible use and individualized coverage review, the executive order’s health-care provisions, and the proposed HIPAA rule. These confirm that regulators are building oversight structures and clarifying expectations. They do not contain data proving that AI tools deliver better care in everyday practice.
Peer-reviewed research, such as the BMJ analysis on early warning scores, offers controlled comparisons but with narrow scope and mixed results. Drawing broad conclusions about AI’s value from any single study requires caution, especially when the populations studied may not reflect the diversity of patients seen nationwide.
For developers and health systems, the FDA’s evolving device framework and HHS’s proposed security updates outline a regulatory floor, but not a guarantee of clinical value. Until more robust outcome data emerges, the honest reading is straightforward. Health-care AI is a set of tools under active construction, not a proven advance. The scaffolding of oversight is going up. The evidence that it will hold weight has not yet arrived.
More from Morning Overview
*This article was researched with the help of AI, with human editors creating the final content.
