takeoff.watch

An independent evaluator finds AI has made a lab's research 2× faster

Will an independent organisation publish an estimate, giving a number and a method, that AI systems have made the overall pace of a covered developer's AI research at least 2 times faster than it would be with human researchers alone?

Chance by Q2 2029

51%

Cumulative probability

0%25%50%75%100%50% · Q2 2029
  • Latest forecast

Forecast history

Forecast made on 11 Oct 2026

Latest of 2 forecasts · Methodology v2.1

Model reasoning

Forecast made 11 Oct 2026 · Methodology v2.1

The best independent estimate so far, from the evaluation group METR, puts AI's speed-up of Anthropic's research at about 1.5 times, with roughly a 30% chance it is already 2 times. METR is gaining deeper access inside AI companies, and AI's share of research work is climbing fast. So a published estimate of at least 2 times looks likely within five years, but not soon. The biggest swing factor is whether METR's upcoming reports show the speed-up rising toward 2 times.

Where things stand (as of 11 Oct 2026)

Who could publish. The obvious candidate is METR. It has an unpaid "AI R&D assessment agreement" with Anthropic, and a separate METR team with extra access is assessing how much AI is accelerating AI R&D inside Anthropic. METR's Claude Opus 5.5 summary (22 Sep 2026, https://metr.org/blog/2026-09-22-claude-opus-5-5/) quotes that team's preliminary estimate: "~1.5X overall acceleration in capabilities due to AI (i.e. 1.5 years in 1 year), with perhaps 30% chance of 2X." It gives no observation period and no method. METR says it expects "further public outputs… in the coming weeks."

Publication channels are growing:

  • Dario Amodei's "We Must Pace the Frontier" (12 Sep) commits Anthropic to embedded outside evaluators with employee-like access and the right to publish findings on risk levels without Anthropic's editorial control.
  • Anthropic says it plans to have embedded evaluators "monitor key metrics." It has signed Faculty/Accenture first and is in talks with METR (CNBC, 18 Sep). Altman says OpenAI will match.
  • Ryan Greenblatt joined METR in late September to investigate how close labs are to self-improvement speeding up AI progress.
  • Anthropic's Responsible Scaling Policy v3.4 requires a published external review of any risk report covering a model that crosses the automated-R&D threshold. That threshold is a doubling of the rate of aggregate capability progress.

The number is still below 2.

  • Anthropic's August 2026 Risk Report: research is "significantly faster… but not yet by a factor of 2."
  • Opus 5.5 System Card: a trend-break reading of Anthropic's capability index gives a 1.53x slope increase (95% range 1.20–1.82). A better-fitting reading says capabilities jumped once at Mythos Preview and then returned to the old slope.
  • Thomas Kwa's METR note (8 Jul, published before the window opened) estimates researcher uplift at about 2.5x but overall R&D uplift at about 1.77x, and says 2x "could happen in the next year or so."
  • Greenblatt (Apr 2026) thinks 2x overall progress needs roughly 5x faster research labour.
  • The fastest-growing published measures don't count here: OpenAI's 3.1 agent-workdays per human workday, Anthropic's 8x code per engineer, and staff self-reports of about 4x.

Reference class and model

There is no historical base rate, so I extrapolated the trend.

  • Inputs are rising fast. Claude "leads" 26% of Anthropic's R&D work (Aug 2026), up from 0–1% in Feb–Mar. Code output per engineer went 2.5x → 5.8x → 8x over three quarters.
  • Growth model. Treat overall speed-up as roughly 2/3 × (labour speed-up)^0.55 + 1/3, using Greenblatt's split, and let labour speed-up double every 6–9 months. That reaches 2x around mid-2027.
  • Measurement lag. Capability-trend evidence lags by a few months, publication adds more, and "pacing" slows things further. Together these push the median toward 2028.

I mixed three scenarios:

  • Fast self-improvement (~35%): 2x reached in 2027–28.
  • Gradual (~40%): reached in 2028–30.
  • Stall or deliberately capped (~25%): stays under 2x.

Given 2x is reached, I put roughly 80–88% on a qualifying publication: a central figure with a method and a recent observation period. I turned this into quarterly hazards: 3% now, peaking at about 7–8% per quarter in late 2027–2028, then falling to about 2.5%. The cumulative product gives 0.24 by end-2027, 0.44 by end-2028 and 0.67 by end-2031; a separate scenario-mixture calculation gave nearly the same curve.

Paths to YES and NO

  • YES: a METR report (embedded team, model-release summary, Frontier Risk Report, or required external review) gives "~2x" or more for Anthropic or OpenAI. "~2x" is read as 2.0, which qualifies.
  • NO:
    • Labs' pacing caps the rate of AI-driven progress.
    • Compute stays the binding constraint; Anthropic's own production function maps 4x labour to about 1.7x output.
    • Methods stay too confidential to meet the "number and method" bar.
    • Government secrecy limits what can be published.

Near term is low. The latest figure is about 1.5x, and the observation period has to end after 4 October.

Strongest counterargument

The criteria are demanding, and the newest evidence shows the trend has not steepened further. Persistent compute bottlenecks plus deliberate pacing could keep credible central estimates below 2x for years, which argues for roughly 40%. Against that, METR's institutional capacity and access are expanding quickly, AI's share of R&D work is rising steeply, and five years is a long window. I settled in between.

Sources

Resolution criteria

Opened 4 Oct 2026Not yet resolved

Question. By the last day of each quarter, has an independent organisation published an estimate, giving a number and a method, that AI systems have made the overall pace of a covered developer's AI research at least 2 times faster than it would be with human researchers alone?

Definitions.

  • Independent organisation: a named organisation that is not part of, controlling, controlled by, or under common control with the covered developer concerned (control meaning a majority of equity, votes, or board appointments), and neither it nor any entity controlling it has made a direct equity investment in that developer. A government body is assessed apart from its government's other holdings. Paid work for the developer, compute or model access, grants, donations, and the developer's pre-publication review or redaction do not affect independence; an author employed by the developer does. It must have produced the estimate with access to the developer's non-public research data, staff interviews or surveys, or unreleased models under an agreement or memorandum of understanding with the developer or a legal or regulatory requirement, as stated in a publication by either.
  • Overall pace of a covered developer's AI research: the developer-wide rate of substantive progress in developing or improving AI models and methods, measured per unit of calendar time. For a covered developer that is a named AI lab or model organization within a larger company, the scope is that organization's AI research, not the parent's unrelated work. An aggregate estimate from a sample counts only if the organisation explicitly presents it as an estimate of the developer's overall AI research pace.
  • Human researchers alone: a counterfactual in which the same human research workforce has ordinary non-AI software and research infrastructure but no AI systems performing or assisting research work. This does not exclude training or evaluating AI models as objects of research. The comparison must distinguish AI assistance from changes in human staffing, training compute, or other non-AI resources; a historical baseline counts only if the method explains how those differences are accounted for.
  • At least 2 times faster: a ratio of AI-assisted research progress per unit time to the human-only counterfactual of at least 2.0, or equivalent time to the same progress of at most one-half. The AI systems need not have been built by the developer whose research is estimated.
  • Giving a number and a method: the publication reports the multiplier (or quantities explicitly equated to it), identifies the research-progress measure and observation period, and explains how both the AI-assisted result and human-only counterfactual were estimated and aggregated to the developer-wide level. Experimental comparisons, observational analyses, and explicitly described estimation models may count; public raw data, independent replication, and statistical significance are not required. Survey evidence counts only when the organisation uses a disclosed method to translate it into and endorse an overall research-progress estimate, rather than merely reporting respondents' output estimates.

Resolves YES if a resolving source gives, as the independent organisation's own estimate, an achieved or currently observed overall multiplier of at least 2.0, explicitly applied to a named covered developer, with the method published there or in public documentation it cites. A central or point estimate counts despite an uncertainty interval extending below 2.0; “about,” “roughly,” or “~” figures are read at face value; a range given alone is read at its lower bound; and if several estimates for the same developer and period include none identified as central or overall, the lowest governs. Preliminary or experimental estimates count.

Exclusions (resolve NO).

  • Multipliers for coding speed or volume, experiment counts, or agent labor (such as agent-workdays per human workday).
  • Unadjusted averages or medians of staff's self-reported output gains.
  • Figures such as “up to 2x” or “nearly 2x,” and a stated probability that the multiplier reaches 2.

Resolving source. Only the organisation's website, including staff-authored notes hosted there; reports, papers, preprints, testimony, or submissions under its name or affiliation; and sections of developer-published documents explicitly presented as its own text, not a developer's paraphrase, summary, or quotation. This replaces Public confirmation (A2): developer disagreement does not prevent YES, but the organisation withdrawing the estimate, or revising it below 2.0, within 14 days of publication does. Apply the Resolution window, Observation period, and Resolution date (A3) conventions; the date is when the estimate and its method are both public.

Download all forecasts (JSON)