An AI automates 50% of the Remote Labor Index

Where things stand (15 Oct 2026)

• Top score today: Scale's live RLI leaderboard (labs.scale.com/leaderboard/rli, checked 15 Oct 2026) puts Claude Opus 5.5 first at 21.25%. Next are GPT‑6 Astra at 20.83%, Fable 5.1 at 17.92%, Fable 5 at 15.83% and Opus 4.8 at 8.33%. Every figure is a whole number of projects out of 240 (21.25% = 51/240), so 50% means 120 of 240.
• When each entry appeared: I downloaded and decompressed Wayback snapshots of the leaderboard (CDX index for labs.scale.com/leaderboard/rli).
  • 18 Mar 2026: Opus 4.6 (CoWork) led at 4.17%.
  • Between 3 Jun and 5 Jul: Fable 5, Opus 4.8 and GPT‑5.5 were added, after CAIS moved to stronger scaffolds (Claude Code or Codex with computer use, a worker‑critic loop, 24 h runs, $50–150 budget per project). Source: safe.ai/blog/significant-increase-in-digital-labor-automation.
  • 22 Sep: GPT‑6 Astra (20.8%) and Fable 5.1 were added (CAIS post on X, 22 Sep 2026).
  • Between 29 Sep and 8 Oct: Opus 5.5 was added.
• No qualifying result yet. The question opened on 8 Oct. Since then nothing has been published at or near 50%.
• Publication lag: Frontier models are now scored and posted about 2–4 weeks after release, roughly once a month.

Base rate and trend (computed)

I fitted a straight line to the log‑odds of the best score over time, using model release dates from auerin.com/timelines/agi (table: 0.8% Jun 2025 … 21.25% on 22 Sep 2026). The rise works out to:

• 2.0–2.7 log‑odds units per year on the full history and on the stronger‑scaffold period, which puts the 50% crossing between March and June 2027;
• only about 1.3 per year on Jun–Sep 2026 alone, which puts the crossing in about September 2027.

Two other extrapolations:

• A straight line in percentage points (about 19–23 points a year) gives about 2028.
• Constant exponential growth gives early 2027, which is implausible near the middle of the scale.

Comparable agent benchmarks (OSWorld, SWE‑bench) rose about 1.7–2.3 log‑odds units a year. RLI's jump in mid‑2026 came partly from a one‑off scaffold change; without it the underlying rise is about 1.6 a year.

I then simulated a mix of scenarios, starting from Opus 5.5's release on 22 Sep 2026:

• Fast (60%): median rise of 1.9 log‑odds units a year.
• Slow (28%): median rise of 0.9, reflecting deliberate pacing of releases and a hard core of 3D, video and music projects.
• Never (12%): RLI discontinued, a pause, or Scale stops publishing.

I added a 2–5 week publication lag and some lumpiness from discrete releases. The result is about 1% by end‑2026, 9% by Mar 2027, 26% by Jun 2027, 45% by Sep 2027, 59% by Dec 2027, about 83% by end‑2028 and a ceiling of about 88%.

Adjustments specific to this case

Pushing toward YES:

• A new Anthropic checkpoint (rumoured Fable 5.5 or Opus 5.6) appears imminent. It is in the Azure catalog as of 13 Oct, and users report it building 3D Blender scenes and animations, which are weak RLI categories (testingcatalog.com, 13 Oct; wccftech, 14 Oct).
• Gemini 4 Argon reached general availability on 13 Oct and Meta's Muse Wonder Full launched on 12 Oct. Neither has been scored yet.
• In this fast‑moving world, recursive self‑improvement is said to have begun (Amodei, "We Must Pace the Frontier").

Pushing toward NO:

• OpenAI shelved GPT‑6.1 Astra (Guardian, 28 Sep 2026).
• Anthropic and OpenAI have endorsed pacing, and governments now review releases before launch.
• Frontier gains from June to October were modest.

On balance I put the near quarters slightly above the simulation: Q4 2026 at about 4%, because the next release is unlikely to more than double the score; Q1 2027 at about 13.5%.

Strongest case against

• For lower: If you take the slower recent pace, or a straight line in percentage points, 50% arrives in 2028. A one‑off scaffold change inflated the mid‑2026 jump. Human grading against professional work may be slow to saturate on creative and 3D tasks.
• For higher: A 32‑trader Manifold market ("best RLI by Dec 31, 2026") prices about 25% on ≥50% by year‑end and centres around 30–45%. That market is thin and inefficient: it still gives about 5% to outcomes below 20% that are already ruled out, and recent trades moved it down. I give it limited weight.

Main sources

• labs.scale.com/leaderboard/rli (live) and its Wayback snapshots, Mar–Oct 2026
• safe.ai/blog/significant-increase-in-digital-labor-automation (about 1 Jul 2026)
• x.com/CAIS/status/2102440645684322308 (22 Sep 2026)
• auerin.com/timelines/agi
• Manifold API, market slugs what-will-be-the-best-remote-labor and what-will-be-the-highest-reported-r
• Reuters and The Verge on Gemini 4 Argon (13 Oct 2026)
• The Guardian on GPT‑6.1 Astra (28 Sep 2026)
• darioamodei.com/post/we-must-pace-the-frontier
• pasqualepillitteri.it and testingcatalog.com on Fable 5.5 (13–14 Oct 2026)

Question. By the last day of each quarter, has an AI system achieved an automation rate of at least 50% on Scale AI's Remote Labor Index?

Definitions.

• Remote Labor Index (RLI): Scale AI's benchmark of paid, real-world remote freelance projects, built with the Center for AI Safety (CAIS) (arXiv:2510.26787) and scored on Scale's leaderboard at labs.scale.com/leaderboard/rli, including a renamed or relocated leaderboard, or a qualifying successor.
• Automation rate: the percentage of projects in the official evaluation set for which human evaluators judge the AI system's deliverable at least as good as the professional's reference deliverable: under RLI's 3-point scale, a final majority-vote rating of 2 or 3.
• Official evaluation set: the full set of projects on which Scale computes its headline RLI automation rates, not the public set or any other subset. Projects not attempted, not scored, timed out, or without a deliverable stay in the denominator as not automated. Where Scale states that a rate omits projects, recompute it as the number judged automated divided by the full set size Scale states.
• AI system: any entry Scale lists as an AI model, agent, or automated system, from any developer, public or not, with whatever harness, tools, multi-agent or worker-critic arrangement, time limit, and budget Scale uses or permits; Scale may list its best overall rate across separately evaluated environments. No person may produce, select, edit, or review its deliverables beyond submitting the project brief, and it may not access the reference deliverable or evaluators' judgements of its attempts. A rate crediting a project if any one of several attempts, configurations, or systems passes does not count.
• At least 50%: 50.00% or more, using the most precise figure Scale publishes for the entry, or the exact quotient where Scale publishes projects automated and set size. A figure published only in rounded form is used as published; do not round figures or infer unpublished digits.
• Qualifying successor: a benchmark that Scale, alone or with CAIS, explicitly designates as a new version of or successor to RLI, provided its projects are held-out, paid, multi-sector remote-work projects with professional reference deliverables, and its headline metric keeps RLI's pass/fail line (human evaluators judge the AI deliverable at least as good as the professional's) under any rating scale; the threshold then applies, without rescaling, to the share of the successor's official evaluation set classified as automated. A headline metric that gives partial credit or relies on automated graders does not qualify. The original RLI continues to count alongside any successor.

Resolves YES if a qualifying publication, first published within the Resolution window and on or before the quarter-end, reports an automation rate of at least 50% for an AI system on the official evaluation set of RLI or a qualifying successor, or explicitly states that a system automated at least 50% of that set. Point estimates govern; no significance test, replication, or minimum time on the leaderboard is required.

Resolves NO / Exclusions. Resolve NO at a horizon if no qualifying publication has appeared by then. A figure Scale withdraws or corrects as erroneous within 14 days of first publication does not count; later changes do not reverse a YES. If Scale stops publishing RLI results, no other benchmark, evaluator, or estimate substitutes.

Resolving source. Scale AI only, alone or jointly with CAIS: the RLI leaderboard (labs.scale.com/leaderboard/rli); RLI papers, reports, and posts, including revisions of arXiv:2510.26787 and remotelabor.ai; and CAIS-authored RLI results posts on safe.ai or remotelabor.ai that report human evaluations conducted by Scale. Figures from AI developers, other evaluators or aggregators, or users of the open-source RLI platform do not count, even if they cite a Scale evaluation. This is narrower than Public confirmation (A2). Apply the Resolution window and Resolution date (A3) conventions, dating a figure by its first dated release or, failing that, the earliest archived snapshot of the leaderboard showing it.