An AI beats superforecasters by as much as they beat the public

Where things stand (FRI's 2026-10-16 nightly tournament leaderboard)

I used the official CSV, leaderboards/csv/leaderboard_tournament.csv in github.com/forecastingresearch/forecastbench-datasets:

• Best AI: Google DeepMind's "fire hedgehog", Overall Brier Index 69.0 (Brier 0.096).
• Superforecaster median forecast (S): 68.7 (Brier 0.098).
• Public median forecast (P): 63.4 (Brier 0.134).

So S − P = 5.3, and the bar is A ≥ 2S − P = 74.0. The best AI is 5.0 points short. The Baseline leaderboard is much further off: its bar is 72.8 and its best model scores 61.9. In Brier terms, an AI needs to beat superforecasters by about 0.030 (bar ≈ 0.0676). Today the margin is 0.002. Because the scaling constant is additive, this Brier margin barely moves when FRI's scaling anchors shift, so it is a stable target.

Trend and reference class

Leaderboard history. I pulled about 100 versions of the leaderboard CSV from git. Since FRI adopted the Brier Index in March 2026:

• S − P has stayed between 5.2 and 5.7.
• The best AI's lead over S went from −2.6 (March) to +0.3 (October).
• Older entries fell by about the same amount as the human scores. So the gain comes from new systems, not from drift in the humans' scores.

State of the art by release date (FRI's sota_graph_tournament.csv): 63.9 (Oct 2024), 65.0 (Feb 2025), 66.0 (Oct 2025), 68.4 (May 2026), 69.0 (Jun 2026).

• Linear fits give about 2.5–4.8 points a year depending on the window. They put the 74 crossing between mid-2027 and late 2028.
• Reference class: AI took about 20 months to go from public level (Oct 2024, ≈63.9) to superforecaster level (Jun 2026). Repeating that pace would add another full public-to-superforecaster gap by about early 2028.

Next wave. Preliminary dataset-only scores for systems submitted Aug–Oct 2026 point to an overall frontier of about 69.5–70.5 by December 2026. Early preliminary scores fall back noticeably as more questions resolve.

Why gains may slow

The overall score weights market and dataset questions equally.

Market questions. On the tournament board a market question is scored against the market's own price on the due date. Superforecasters beat that price by 0.017 Brier; the best AI (Cassi) beats it by about 0.024. Since late September 2026, FRI has dropped Manifold, moved toward liquid Kalshi, Polymarket and Metaculus questions, and stopped sampling markets priced below 0.1 or above 0.9. Big edges over liquid markets are hard to get.

Dataset questions. Here the AI is already about 0.02–0.03 ahead, close to the superforecaster–public gap of 0.034. Reaching the bar roughly means about 71–72 on dataset plus about 77–78 on market in a single system, against today's best of about 66–68 and 76.

The superforecaster "rematch"

FRI has announced a fresh superforecaster round on the Oct 25, 2026 question set: 130 market and 70 dataset questions (X thread of Sep 22; GitHub commits). The leaderboard code treats "Superforecaster median forecast" as a single entity, so the new forecasts will likely be pooled with the 2024 ones, and the 130 market questions would dominate the market part. Superforecasters forecast days before the due date but are scored against the due-date price, and they face questions where AI is strong. That makes a drop in S somewhat more likely than a rise. If the public is not re-surveyed, a drop in S lowers the bar by twice as much. But a rise is possible too: superforecasters might have AI help or a better study design. I model the change in the bar as a mixture with mean about −0.4 points and sd about 1.6.

Model

I simulated a frontier that rises toward a ceiling:

• Starting level about 69.2 at end-2026.
• Initial growth about 2.3 points a year.
• Ceiling about 75 ± 2.5.
• Leaderboard noise of about +0.8, because the best of many entries over many nights is taken.
• The bar shift above.
• A 4–8% yearly chance that FRI stops publishing a usable comparison.

This gives about 18% by end-2027, 37% by end-2028 and 61% by end-2031. The chance per quarter peaks at about 6–7% in 2028, then falls as the remaining paths are mostly ones where a ceiling binds. The near term is tiny: any shift from the rematch only starts showing from mid-December 2026.

Strongest case against

Being 31% better than the superforecaster median in Brier is a very large margin where outcomes are partly unpredictable. As AI traders make markets more efficient, the market half may never contribute much. FRI keeps adding near-coin-flip question types, such as pairs of stock returns. A well-run rematch could raise S. Any of these could keep the leaderboard below 74 indefinitely, so I keep roughly 40% on NO even over five years.

Sources

• FRI leaderboard CSVs and git history, plus sota_graph_tournament.csv (forecastbench.org/assets/data) — accessed 2026-10-17.
• FRI, "AI models have likely reached parity with superforecasters on ForecastBench" (2026-07-16).
• FRI, "Introducing the Brier Index" (2026).
• FRI X thread announcing the rematch (2026-09-22).
• forecastbench GitHub commits fa04ba07, c73a6367, 0d800eef (Oct 2026).
• EA Forum, "The Copier Line" (2026-08-21).
• Good Judgment, "Not So Fast" and "What ForecastBench Doesn't Measure" (2026).
• Metaculus Spring 2026 FutureEval analysis.

Question. By the last day of each quarter, has the Forecasting Research Institute reported an AI system whose overall ForecastBench score beats the superforecaster median by at least as much as the superforecaster median beats the general-public median?

Definitions.

• ForecastBench: the benchmark developed and maintained by the Forecasting Research Institute (FRI), published at forecastbench.org. Either its Tournament or Baseline leaderboard can qualify.
• AI system: an entry FRI identifies as an AI model or automated AI forecasting system, from any developer. Tool use, scaffolding, fine-tuning, ensembles, and use of human-produced information or prediction-market prices are allowed. A human forecaster or human–AI team in which people select or edit the submitted forecasts does not qualify.
• Score: the Overall Brier Index, with higher values better, as described in FRI’s “Making Forecasting Scores Easier to Interpret: Introducing the Brier Index” and the leaderboard methodology. It equals 100 × (1 − √B), where B is the overall difficulty-adjusted Brier score, calculated by equally weighting the dataset and market category scores before taking the square root. Differences are absolute index-point differences, not relative percentage improvements or differences in rank. If FRI stops displaying the index, its published overall difficulty-adjusted Brier scores may be converted using this formula; an unrelated replacement metric does not qualify.
• Superforecaster median and general-public median: the overall scores of FRI’s human comparison entries labelled “Superforecaster median forecast” and “Public median forecast,” respectively, including renamed equivalents. These are the scores of the question-by-question median forecasts within each group, not the medians of individual participants’ scores. “Public” means FRI’s non-expert public comparison sample, not a nationally representative population or a prediction-market crowd.
• Comparable scores: the AI and both human entries must come from the same dated leaderboard release or a single FRI comparison using the same methodology and evaluation cutoff. FRI’s difficulty-adjusted comparisons across non-overlapping question sets qualify; identical questions or fresh human participation are not required.

Resolves YES if a qualifying FRI publication supplies comparable scores A for an AI system, S for the superforecaster median, and P for the public median satisfying S > P, A > S, and A − S ≥ S − P (equivalently, A ≥ 2S − P). A published FRI statement explicitly confirming this index-point threshold also qualifies.

Use the most precise scores supplied with that publication; if only rounded point estimates are supplied, use those as published. No statistical-significance test, confidence-interval separation, minimum duration at the threshold, or independent replication is required. The Observation period convention applies: the evaluated outcome period must end after the opening date; merely republishing or recalculating scores solely for earlier outcomes does not count.

Resolves NO / Exclusions. Resolve NO at a horizon if no qualifying publication has appeared by then. Parity, first place alone, projected future scores, a qualifying gap only on the dataset or market subset, and comparisons mixing leaderboard dates or scoring methods do not qualify. Neither an individual-human median nor a raw-Brier-score gap can substitute for the defined comparison. Third-party calculations from raw forecasts, model-developer claims, and results on other forecasting benchmarks do not qualify unless FRI publishes the required comparison.

Resolving source. FRI only: its official ForecastBench leaderboard and accompanying dated releases or data files, FRI-authored reports, or institutional write-ups on forecastingresearch.org or forecastingresearch.substack.com. This is narrower than Public confirmation (A2). An archived leaderboard snapshot or timestamped official release may establish a historical publication. Apply the Resolution window and Resolution date (A3) conventions; use the first qualifying publication date, not the model release, forecast submission, or outcome date. Later score changes do not reverse a qualifying YES.