An independent evaluator finds AI has made a lab's research 5× faster

Where things stand (12 Oct 2026)

The channel that could resolve this already exists, but the measured number is far below 5. METR has an AI R&D assessment agreement with Anthropic. A separate METR team with elevated access gave a preliminary estimate of "~1.5X overall acceleration in capabilities due to AI (i.e. 1.5 years in 1 year), with perhaps 30% chance of 2X." METR said more public output from that investigation would follow "in the coming weeks" (METR, Opus 5.5 summary, 22 Sep 2026). That format (a central estimate, a named developer, access under an agreement) already meets most of the criteria. What it lacks is a public method and a number of 5 or more.

Other evidence points to the same level:

• Anthropic's August 2026 Risk Report puts acceleration at less than 2x. It says Anthropic "may cross" its 2x threshold in the coming year and plans "deeper integration of external reviewers" (report).
• The Opus 5.5 system card shows Anthropic's capability index rising about 14.75 points a year historically. Since Mythos there has been either a one-time jump or at most a 1.53x steeper slope; the slope has not doubled.
• METR's Frontier Risk Report (May 2026) asked four labs directly about acceleration. None reported dramatic speedups. METR plans another round in late 2026 (report).

Related figures are large, but most are excluded or are inputs rather than overall pace:

• Anthropic engineers merge 8x as much code per day, and staff self-report a median of 4x output (Anthropic, June 2026).
• OpenAI researchers use 3.1 agent-workdays per human workday (OpenAI, 6 Sep 2026).
• A METR staff note by Thomas Kwa estimates researcher uplift of about 2.5–2.8x but overall R&D speedup of only about 1.8x, because compute is a bottleneck (note, 8 Jul 2026). Anthropic told METR that 2x labour maps to about 1.15–1.3x output.

Access for outside evaluators is growing. Dario Amodei committed Anthropic to embedded evaluators who can publish without Anthropic's editorial control (essay, 12 Sep 2026). Faculty (Accenture) is the first; METR is in talks (CNBC, 18 Sep 2026). OpenAI says it will do the same. METR raised about $71M, partly for "tracking recursive self-improvement." There are counter-signals: OpenAI kept legal rights to block METR's risk conclusions on GPT-5.6, and fired three staff partly over contacts with METR (NPR, 9 Oct 2026).

Reference class and model

There is no direct base rate: no independent estimate has ever exceeded about 2x. I therefore decomposed the probability into two parts.

1. Does the true AI-attributable overall pace at a lab with evaluator access reach 5x by mid-2031? That means five years of progress per year with compute held fixed. It roughly requires near-full automation of AI research, including research judgment.
   • Forecasters' estimates: Kokotajlo's automated-coder median is mid-2028 and Lifland's is mid-2030; Greenblatt's median for full automation is early 2031, with a 25th percentile in mid-2028 (Q1 2026 update).
   • Reality is tracking about 70% of AI 2027's pace. AI 2027 had 1.5x in early 2026; METR's 1.5x came in about Sep 2026. Scaling AI 2027's 5x date by that pace puts 5x around mid-2028 in the fast case.
   • Compute bottlenecks, the industry push to deliberately slow progress (which explicitly targets this rate), and skeptical economic analyses pull the estimate down.
   • My estimate: about 30–33% by mid-2031.
2. Given that, does a qualifying publication follow? It needs a central estimate of at least 5, a named developer, a public method, and no withdrawal. Ranges are read at their lower bound, measurements lag, labs can restrict disclosure, and government secrecy is possible in a takeoff. I put this at about 0.6.

That gives about 0.20, plus about 2% for edge-case estimates, so about 22–23% by end-2031.

I set quarterly hazards rising from 0.3% (late 2026) to a peak of about 1.7% in 2029, then easing slightly. Slower worlds still cross later, while bottlenecked or paced worlds never do. The cumulative series was computed in Python as 1 − ∏(1 − h).

Strongest case against

• Higher: lab chief scientists signed a statement that labs "could be close to automating AI research." OpenAI's internal task horizons grew 8x in six months. Claude went from leading under 1% to 26% of Anthropic's R&D work in six months. In an AI-2027-like takeoff the multiplier passes 5x quickly, and many evaluators would report it. That case supports 35% or more.
• Lower: overall pace measured with compute fixed has barely moved despite huge labour gains. The criteria are strict, and the leading labs are trying to slow down. That case supports about 10%.

I weigh these roughly evenly.

Question. By the last day of each quarter, has an independent organisation published an estimate, giving a number and a method, that AI systems have made the overall pace of a covered developer's AI research at least 5 times faster than it would be with human researchers alone?

Definitions.

• Independent organisation: a named organisation that is not part of, controlling, controlled by, or under common control with the covered developer concerned (control meaning a majority of equity, votes, or board appointments), and neither it nor any entity controlling it has made a direct equity investment in that developer. A government body is assessed apart from its government's other holdings. Paid work for the developer, compute or model access, grants, donations, and the developer's pre-publication review or redaction do not affect independence; an author employed by the developer does. It must have produced the estimate with access to the developer's non-public research data, staff interviews or surveys, or unreleased models under an agreement or memorandum of understanding with the developer or a legal or regulatory requirement, as stated in a publication by either.
• Overall pace of a covered developer's AI research: the developer-wide rate of substantive progress in developing or improving AI models and methods, measured per unit of calendar time. For a covered developer that is a named AI lab or model organization within a larger company, the scope is that organization's AI research, not the parent's unrelated work. An aggregate estimate from a sample counts only if the organisation explicitly presents it as an estimate of the developer's overall AI research pace.
• Human researchers alone: a counterfactual in which the same human research workforce has ordinary non-AI software and research infrastructure but no AI systems performing or assisting research work. This does not exclude training or evaluating AI models as objects of research. The comparison must distinguish AI assistance from changes in human staffing, training compute, or other non-AI resources; a historical baseline counts only if the method explains how those differences are accounted for.
• At least 5 times faster: a ratio of AI-assisted research progress per unit time to the human-only counterfactual of at least 5.0, or equivalent time to the same progress of at most one-fifth. The AI systems need not have been built by the developer whose research is estimated.
• Giving a number and a method: the publication reports the multiplier (or quantities explicitly equated to it), identifies the research-progress measure and observation period, and explains how both the AI-assisted result and human-only counterfactual were estimated and aggregated to the developer-wide level. Experimental comparisons, observational analyses, and explicitly described estimation models may count; public raw data, independent replication, and statistical significance are not required. Survey evidence counts only when the organisation uses a disclosed method to translate it into and endorse an overall research-progress estimate, rather than merely reporting respondents' output estimates.

Resolves YES if a resolving source gives, as the independent organisation's own estimate, an achieved or currently observed overall multiplier of at least 5.0, explicitly applied to a named covered developer, with the method published there or in public documentation it cites. A central or point estimate counts despite an uncertainty interval extending below 5.0; “about,” “roughly,” or “~” figures are read at face value; a range given alone is read at its lower bound; and if several estimates for the same developer and period include none identified as central or overall, the lowest governs. Preliminary or experimental estimates count.

Exclusions (resolve NO).

• Multipliers for coding speed or volume, experiment counts, or agent labor (such as agent-workdays per human workday).
• Unadjusted averages or medians of staff's self-reported output gains.
• Figures such as “up to 5x” or “nearly 5x,” and a stated probability that the multiplier reaches 5.

Resolving source. Only the organisation's website, including staff-authored notes hosted there; reports, papers, preprints, testimony, or submissions under its name or affiliation; and sections of developer-published documents explicitly presented as its own text, not a developer's paraphrase, summary, or quotation. This replaces Public confirmation (A2): developer disagreement does not prevent YES, but the organisation withdrawing the estimate, or revising it below 5.0, within 14 days of publication does. Apply the Resolution window, Observation period, and Resolution date (A3) conventions; the date is when the estimate and its method are both public.