A training run confirmed at 10^28 FLOP

Where things stand (11 Oct 2026)

I downloaded Epoch's all_ai_models.csv (https://epoch.ai/data/all_ai_models.csv; page says it was updated 8 Oct 2026). Its highest training-compute estimate is 1.0001e27 FLOP for GPT-6 Astra (OpenAI, released 3 Sep 2026, rated "Likely", last modified 23 Sep). Epoch's note: "at least 100,000 GB200s in Abilene… ~100k GB200s over 90 days at 25% FP8 MFU", confidence interval about 5e26–2e27. That rests on Greg Brockman's Stratechery interview and Jensen Huang's "~100K+" post (PC Gamer, Sep 2026). No entry is at or above 1e28, and no lower bound is above 2.1e26. Astra cannot qualify in any case, because its training ended before the 16 Sep opening date. So the question needs a new run whose Epoch estimate is about 10 times the current record.

Disclosure is the binding constraint. Almost every 2026 frontier model has no Epoch compute estimate: Claude Opus 5/5.5, Mythos, Gemini 3.x and 4 Argon, Grok 4.5–4.7, GPT-5.5, GPT-6 Sol and Muse Spark. That holds even where Musk gave parameter counts.

Base rate

Record-setting Epoch estimates from the dataset: GPT-3 3.1e23 (May 2020) → GPT-4 2.1e25 (Mar 2023) → Gemini Ultra 5e25 → Grok 3 3.5e26 (Feb 2025) → GPT-4.5 3.8e26 → Grok 4 5e26 (Jul 2025) → Astra 1e27 (Sep 2026).

• That is about 3.0x per year since GPT-4, but only about 1.8x per year since early 2025.
• I resampled the 10 historical (gap, jump-size) pairs to simulate how long it takes to reach 1e28. Median about 2–2.3 years, i.e. late 2028. Roughly 6–8% by end-2027, 51–59% by end-2028 and about 89% by end-2029.
• Epoch's own May 2025 projections (https://epoch.ai/publications/model-counts-compute-thresholds) put the first model above 1e28 in 2027 (aggressive scenario), 2028 (median) or 2029 (conservative).

Physical capacity and candidate runs

Epoch's data-center timelines (https://epoch.ai/data/data_centers/data_center_timelines.csv) give each site's peak 8-bit operations per second. I computed how many days a 1e28 run would take at 25% utilization:

Site	Date	Days
Colossus 2	Oct 2026	210
Colossus 2	Q1 2027	128
QTS Cedar Rapids	Q4 2027	64
Meta Hyperion	Q1 2028	56
Stargate New Mexico	Q4 2028	46

Specific candidates:

• SpaceXAI. Musk says Colossus 2 will reach about 1.21M Blackwell chips by late December (Tom's Hardware, TIKR, Sep 2026). On Epoch's assumptions, 1.21M chips for 90 days is about 1.2e28. But about half of the capacity is being rented out (SemiAnalysis, Aug 2026; contracts reported by BASENOR). Grok 5 has no specifications or date.
• OpenAI. Huang says OpenAI plans 400k GPUs for its next run. On Epoch's method that is about 3.9e27 for 90 days or 7.8e27 for 180 days, still short unless OpenAI claims something like "10x Astra" or the run is much longer. Reports say a 10-trillion-parameter base model, "Bel", has finished pretraining. OpenAI restarted a large reinforcement-learning run on 28 Aug (https://openai.com/index/path-to-astra/).
• 2028. Vera Rubin clusters arrive; about 350k Rubin GPUs for 90 days is about 1.1e28.

Paths to YES and NO

YES:

• Musk or Nvidia publicise a giant Grok 5 run.
• OpenAI publishes GPU counts or relative-compute claims for its Bel-based models; Epoch has used relative claims before for GPT-4.5 and Grok 4.
• 2028 runs at the new gigawatt-scale sites.
• Initial public offering filings or open-weight technical reports.

NO:

• Labs stay secretive.
• Epoch's conservative assumptions (lower-bound chip counts, 25% utilization, 90 days) put borderline runs at 5e27–9.7e27.
• Compute is diverted to very profitable inference and rentals.
• Safety pacing. OpenAI held back reinforcement-learning runs and pulled GPT-6.1 Astra. Amodei's "pace the frontier" essay drew public support from Altman and Musk. A thin Manifold market gives 36% to an international training pause by 2028. Against that, the White House accord of 29 Sep is voluntary, and Trump and Beijing reject slowing down.

Forecast shape

The quarterly chance of first resolution rises from about 1–3% in late 2026 and early 2027, to about 6% by late 2027, to a peak of about 13% in late 2028 and early 2029. It then falls to about 8% as the remaining "no" worlds become mostly ones with structural barriers (a pause, total opacity, or Epoch changes). That gives about 15% by end-2027, 45% by end-2028, 67% by end-2029 and 84% by end-2031. I put more weight on these judgement-based figures than on the pure trend simulation, because the simulation ignores disclosure and pause risk.

Strongest case for a higher forecast: if this world is on Epoch's aggressive path (10 GW of SpaceX capacity, OpenAI saying it is working towards GPT-7 and GPT-8, a culture of bragging about GPU counts), a qualifying estimate could come in 2027. Strongest case for a lower one: the record has grown only about 2x per year since 2025, which would push the date towards 2030.

Question. By the last day of each quarter, has Epoch AI publicly estimated that a single AI training run used at least 10^28 FLOP?

Definitions.

• Epoch AI’s estimate: the numeric Training compute (FLOP) field in Epoch AI’s AI Models Dataset, available as all_ai_models.csv, or an explicit estimate in an Epoch AI-authored publication identifying a particular model or run. Any developer and any model domain qualify; public access to the model is not required.
• Single training run: training along one model’s checkpoint lineage, including resumed training and sequential pretraining, fine-tuning, or other post-training stages. Use Epoch’s total-training-compute accounting, as described in its Records and Estimation documentation, but exclude totals that reach the threshold only by combining independently trained models, separate experimental runs, or separately trained components. Distributed training across multiple sites counts as one run if it trains the same model jointly.
• FLOP: total floating-point operations, not FLOP per second, with no adjustment for numerical precision or algorithmic efficiency. Use Epoch’s published numeric value without upward rounding. A point estimate of at least 10^28 qualifies even if its uncertainty interval extends below the threshold. If only a range is published, its lower bound must be at least 10^28. A dataset estimate labeled Confident, Likely, or Speculative counts; an entry marked Wrong, Unknown, or Unverified does not count unless Epoch separately endorses the value in an authored publication.

Resolves YES if an eligible Epoch publication or dataset update gives a qualifying estimate for compute already used in an identifiable single run. The run may be complete or still in progress, but a planned final compute budget is insufficient. Apply the Resolution window and Observation period: the qualifying publication must fall within the window, and the training or measured training-to-date period must end after the opening date. A new or revised estimate can qualify subject to those rules. Once YES, always YES under the Resolution window convention, including if Epoch later revises its estimate downward.

Resolves NO if no qualifying publication exists by the quarter-end. Exclude forecasts, hypothetical models, planned runs, aggregate laboratory compute, hyperparameter-search or model-family totals, inference or evaluation compute, and hardware capacity from which a resolver would have to calculate an estimate independently. A developer’s claim or another analyst’s estimate does not qualify unless Epoch adopts it as its own estimate. A publication that only repeats or revises estimates for training periods ending on or before the opening date does not qualify.

Resolving source. Epoch AI’s own dataset and Epoch AI-authored publications on epoch.ai only, a narrower resolving-source rule than Public confirmation (A2); independent developer confirmation is not required. Apply Resolution date (A3) using the first documented public availability of the qualifying estimate, not the dataset’s model Publication date field. Dated dataset snapshots, archived Epoch pages, or an Epoch changelog may establish availability; if the precise update date cannot be established, use the earliest dated public snapshot that demonstrates the qualifying value. A successor Epoch dataset or renamed field counts only if it reports the same training-compute quantity.