A major health system suspends its generative-AI clinical tools
Will NHS England, the US Veterans Health Administration, or any other health system serving at least 5 million patients order the use of generative-AI clinical tools suspended across the whole system?
Chance by Q4 2031
39%
Cumulative probability
- Latest forecast
| Year | Q1 | Q2 | Q3 | Q4 |
|---|---|---|---|---|
| 2026 | 2% | |||
| 2027 | 4% | 6% | 8% | 10% |
| 2028 | 12% | 14% | 16% | 18% |
| 2029 | 20% | 22% | 24% | 26% |
| 2030 | 27% | 29% | 31% | 33% |
| 2031 | 34% | 36% | 38% | 39% |
Model reasoning
I estimate a low near-term chance, about 10% by end-2027, rising to about 39% by end-2031. Health systems are rolling out AI note-takers and chatbots fast, and near-misses happen roughly once a year: England's NHS told staff to stop using non-compliant AI scribes in 2025, and Israel blocked public chatbots in 2026. But orders that are published, mandatory, system-wide, and in a big enough system are rare. A major AI-scribe data breach or harm scandal would most raise the odds.
Where things stand (as of 17 Oct 2026)
I found no qualifying order published since the question opened on 16 Sept 2026. Both named systems are expanding these tools quickly:
- NHS England
- July 2026 press release: a national rollout of AI scribes ("ambient voice technology", AVT), AI triage in the NHS App, and Microsoft Copilot for more than 500,000 staff.
- Jan 2026: a self-certified AVT supplier registry. It is not mandatory; each trust still runs its own procurement.
- 29 Jul 2026: the medicines regulator (MHRA) decided that purely administrative scribes are not medical devices. NHS England dropped its earlier position that all scribes needed device registration.
- 6 Oct 2026: the government accepted all 44 recommendations of the National Commission on regulating AI in healthcare. They focus on monitoring tools over their whole life, not on halting them.
- The patient-safety investigator HSSIB is looking at AVT and is due to report in summer 2027.
- US Veterans Health Administration (VHA)
- Abridge and Knowtex scribes are being expanded under a $776m enterprise contract (Nextgov, Sept 2026).
- The VA inspector general (OIG report 11 Jun 2026) found staff using VA GPT and Copilot Chat to draft clinical notes. VHA agreed to issue guidance by April 2027 on "requirements … and penalties for inappropriate use". That reads as conditions on use rather than a ban, and leadership's stated policy is to "aggressively deploy" AI.
Reference class and base rate
The closest precedents all predate the opening date:
- NHS England, 9 June 2025. A "Priority Notification" letter ordered organisations to "pause or stop any implementation or use of AVT" that did not meet NHS standards, noting such tools were "widely used in clinical practice". It is cited everywhere from a third-party copy (ig.n3i.co.uk), which suggests it was never posted on an NHS domain. If so, it would probably have failed this question's source rule.
- Israel's Health Ministry, March and June 2026. It directed all health organisations to stop using public chatbots on their networks, then blocked them on government hospital computers. Whether this counts is unclear: much of the system is independently governed, and personal-device use continues.
- Health New Zealand. A March 2026 memo banning ChatGPT for clinical notes was local only. National "must not" guidance has existed since 2023. Its 5.1 million enrolled patients would meet the size threshold.
- US federal ban on Anthropic (Feb 2026). The VA's Claude use was for cybersecurity, not clinical care.
So there have been roughly 0.5–1 near-misses a year at national level, but very few that clear every condition at once: published on the system's own site, mandatory, covering the whole system, showing the tool was already in clinical use, and in a system with official counts of at least 5 million patients. I estimate about 15–30% of near-misses would fully qualify. That gives a hazard of about 8–10% a year.
Paths to YES
- NHS England or its successor, the Department of Health and Social Care. For example: requiring only registry-listed scribes; a national patient safety alert on an AI product; a cyber-driven block of AI tools or agents; or pausing the NHS App AI triage after an incident.
- VHA. For example: the April 2027 guidance banning chat-tool note drafting; suspending a product after an incident; or a federal vendor ban that hits a clinical tool.
- Other eligible systems. Health NZ, NHS Scotland, Kaiser, the US Military Health System, Spanish or Italian regional services, Israel, Mexico's IMSS and others, typically reacting to a vendor breach, patient harm, cyber threats (for instance the OpenAI agent breach of Australia's Medicare portal), or a geopolitical ban on a vendor.
Paths to NO
- Strong pro-AI policy in both the UK and US.
- Governance is moving toward approved-tool registries, so "stop using" orders become standing policy rather than new orders.
- Systems tend to suspend tools through internal memos that are never posted on official sites.
- The patient-count evidence excludes many systems.
Hazard model
I used about 1.5% for the rest of Q4 2026, about 2.0–2.3% a quarter in 2027 (when the VA guidance, the HSSIB report and EU AI Act deadlines cluster), rising slowly to 2.5% a quarter by 2030–31 as use and incidents grow. Compounded, that gives about 10% by end-2027 and about 39% by end-2031.
Strongest case against my estimate (higher): if near-misses keep coming at about one a year and a third of them qualify, the five-year chance would be 60% or more. Strongest case for lower: arguably no event in nearly four years has met this question's strict source rule. I settled between the two.
Sources
- NHS England Priority Notification, 9 Jun 2025 (ig.n3i.co.uk PDF); BMJ, 23 Jun 2025.
- Bird & Bird and Hogan Lovells on the MHRA scribe guidance and NHS registry (Jul–Aug 2026).
- NHS England press release via wired-gov, 6 Jul 2026.
- gov.uk, Commission response, 6 Oct 2026.
- VA OIG report 26-00182-140, 11 Jun 2026; Nextgov and FedScoop, Jun–Sep 2026.
- ynetnews on the Israeli ministry directive, Jun 2026; RNZ on Health NZ, 26 Mar 2026; health.govt.nz enrolment figure.
- Guardian on Healthwatch findings, 31 Aug 2026; CNBC on the Anthropic designation, 25 Sep 2026.
Resolution criteria
Question. By the last day of each quarter, has NHS England, the US Veterans Health Administration, or another health system serving at least 5 million patients issued a system-wide order suspending the use of generative-AI clinical tools?
Definitions.
- Health system: a publicly or privately administered network responsible for delivering or commissioning patient care, with a central authority able to direct clinical-tool use throughout that network. NHS England means the NHS in England under its national directing authority; the Veterans Health Administration (VHA) means the nationwide VHA system. Successor bodies exercising the same system-wide authority count. An insurer without responsibility for administering care delivery, a vendor's customer base, or an aggregation of independently governed providers does not count.
- Serving at least 5 million patients: for systems other than the two expressly named, the latest official annual report, statistical release, or official system profile available when the order is published must report at least 5,000,000 distinct patients receiving care in a twelve-month period, or at least 5,000,000 currently enrolled or registered patients for whom the system has ongoing care responsibility. Encounters, cumulative historical patient totals, and the population of a geographic catchment alone do not count.
- Generative-AI clinical tools: software using a generative model to produce patient-specific text, images, recommendations, or other content for diagnosis, treatment, triage, clinical communication, or clinical documentation. Generative ambient scribes and general-purpose chatbots used for these purposes count. Purely administrative scheduling, billing, research-only use, and non-generative prediction or image classification do not.
- Order suspending use: a mandatory direction from the system's central governing authority to cease an existing clinical use, temporarily, indefinitely, or permanently. The direction must take effect immediately or specify a definite future effective date; an announcement that an order may be issued does not count. No minimum suspension duration or particular reason is required.
- Across the whole system: the prohibition applies to every facility, service, and clinician under that authority wherever the specified tool or category would otherwise be used. Suspending one identified product or a defined category, including unapproved generative-AI clinical tools, counts; suspending all generative AI is not required. The tools need not previously have been deployed at every facility. Continued nonclinical testing is permissible, but exemptions allowing continued clinical use of the suspended product or category in particular parts of the system do not count as a whole-system suspension.
Resolves YES if an official order, circular, or statement confirms that a qualifying system has issued the mandatory suspension described above. The record must establish that the affected clinical use was occurring within the system before the order, whether authorized or unauthorized. A public statement reporting an already-issued internal order counts even if the internal order itself is not published.
Resolves NO / Exclusions.
- Advice to exercise caution, voluntary recommendations, proposed suspensions, and requests by unions, watchdogs, or professional associations are insufficient.
- A freeze only on procurement, new deployments, or expansion does not count if existing clinical use may continue.
- Local hospital, regional, departmental, or individual-clinician suspensions do not count unless that independently governed health system itself meets the patient threshold and the order covers its entire system.
- Vendor outages, vendor withdrawal of a product, and regulator warnings or recalls alone do not count without a qualifying system's own mandatory direction. A government ministry's direction counts when it is acting as the governing authority of the health system.
- A publication merely repeating a pre-opening suspension without confirming a newly issued suspension does not count. Subsequent lifting of a qualifying suspension does not undo YES.
Resolving source. Official orders, circulars, and statements published by the health system or its governing ministry only: for NHS England, england.nhs.uk, digital.nhs.uk, or an official successor or Department of Health and Social Care publication on gov.uk; for VHA, va.gov or department.va.gov; for other systems, their official domains or their governing ministries' official publication repositories. Official annual reports, statistical releases, and system profiles establish the patient threshold. This is a narrower source requirement than Public confirmation (A2); third-party reporting alone cannot resolve YES. Apply Resolution date (A3) and the Resolution window convention. The patient-count evidence is an eligibility check, not the event being forecast, and may predate opening.