An AI automates 90% of the Remote Labor Index

Where things stand (checked 2026-10-16)

The live Scale leaderboard (labs.scale.com/leaderboard/rli) lists each entry with a createdAt timestamp. The best rate has gone:

• 2.50% (Manus, 2025-10-27)
• 3.75% (Opus 4.5, 2026-01-09)
• 4.17% (Opus 4.6 CoWork, 2026-03-05)
• 15.83% (Fable 5, 2026-07-01)
• 20.83% (GPT-6 Astra, 2026-09-17)
• 21.25% (Opus 5.5, 2026-10-07)

That is the current maximum. Gemini 4 Argon (released Oct 13) has not been added yet. Every figure is a multiple of 1/240, so the official set is 240 projects and 90% means at least 216 automated. Fable 5 was first posted at 16.10% (38/236) and later recomputed to 15.83% (38/240), which matches the question's rule for omitted projects. Nothing qualifying has been published since the 2026-10-09 opening date.

Trend arithmetic (computed from the leaderboard timestamps)

In logit terms (log-odds, a scale on which benchmark scores often rise roughly in a straight line):

• A least-squares fit over the frontier points gives +2.73 per year. Measured endpoint to endpoint it is +2.35 per year; over the last quarter (Jul–Oct) only +1.45 per year.
• Reaching 90% needs another +3.5. At 2.35 per year that lands around April 2028; at 1.45 per year, around March 2029.
• Part of the mid-2026 jump came from CAIS's new harness: Claude Code/Codex with computer use, 24 hours per project, a GPU, and a worker-critic loop (CAIS blog, 2026-07-01).

Reference class and model

Comparable benchmarks (GPQA, SWE-bench Verified, OSWorld, GDPval) climbed about 1.7–2.2 logits per year in their middle phase and slowed near the top. GDPval, OpenAI's human-graded "wins or ties against experts" benchmark, seems to have stalled around 85% (GPT-5.5 at 84.9%, April 2026; no human-graded figure published for Astra). That suggests human-judged ceilings can bind in the 80s.

I ran a Monte Carlo simulation with these assumptions:

• Yearly slope drawn from a lognormal with median 2.0 and spread 0.45, plus a 20% chance of a stall.
• An 18% chance the practical ceiling sits below 90%.
• An 8–13% yearly chance that Scale stops publishing.
• Sampling noise on 240 projects.

It gives about 34% by end-2028, 49% by end-2029 and 58% by end-2031.

I also used a scenario mix:

• Fast, AI-2027-like (42%). Progress roughly tracks the AI 2027 scenario, as Zvi notes. METR finds Opus 5.5 on-trend with about 1.5x acceleration of AI research. Dario Amodei's "pace the frontier" essay describes self-improvement starting to accelerate progress.
• Moderate (38%).
• Slow or stalled (20%).

The mix gives about 30% by end-2028 and 62% by end-2031.

Paths to YES and NO

YES: new model generations, cheaper compute, bigger budgets or multi-agent setups that Scale permits, and labs' strong incentive to advertise remote-work automation. CAIS has said it will keep adding major releases, and new models have reached the leaderboard within about two weeks of launch.

NO:

• 90% needs near-flawless reliability across 3D/CAD, video, games and architecture work, judged by three humans by majority vote, with every pass audited (arXiv 2510.26787). The paper itself expects agreement between evaluators to fall as deliverables get more complex.
• Some work types stay stubbornly hard, such as music transcription and playtesting real-time games (CAIS).
• Scale or CAIS could retire RLI or replace it with a harder successor.
• They could switch to AI judges, which would not count under the question's definition.
• Regulation or deliberate "pacing" could cap the models the public can use.

Strongest case against

RLI sits at 21% while models score 97.6% on FrontierMath Tier 4. That points to a hard core of taste- and multimodal-dependent work that may not follow a smooth logistic curve. Five-year benchmark survival is also uncertain. Together these could justify something near 40%. On the other side, thin Manifold markets (median about 38% by end-2026; 80% reached "in 2027" at 66%) and the AI Digest survey (median 18% for end-2026, already beaten) suggest consensus has underestimated the pace. I hold near 60%, with the hazard peaking in 2028–29.

Sources: Scale leaderboard and its embedded JSON (accessed 2026-10-16); safe.ai/blog/significant-increase-in-digital-labor-automation (2026-07-01); CAIS on X (2026-09-22); arXiv 2510.26787; HCAMAG interview with Scale's Sehwag (2026-06-16); METR on Opus 5.5 (2026-09-22); darioamodei.com "We Must Pace the Frontier"; Manifold markets (via API); openai.com GPT-5.5 (GDPval).

Question. By the last day of each quarter, has an AI system achieved an automation rate of at least 90% on Scale AI's Remote Labor Index?

Definitions.

• Remote Labor Index (RLI): Scale AI's benchmark of paid, real-world remote freelance projects, built with the Center for AI Safety (CAIS) (arXiv:2510.26787) and scored on Scale's leaderboard at labs.scale.com/leaderboard/rli, including a renamed or relocated leaderboard, or a qualifying successor.
• Automation rate: the percentage of projects in the official evaluation set for which human evaluators judge the AI system's deliverable at least as good as the professional's reference deliverable: under RLI's 3-point scale, a final majority-vote rating of 2 or 3.
• Official evaluation set: the full set of projects on which Scale computes its headline RLI automation rates, not the public set or any other subset. Projects not attempted, not scored, timed out, or without a deliverable stay in the denominator as not automated. Where Scale states that a rate omits projects, recompute it as the number judged automated divided by the full set size Scale states.
• AI system: any entry Scale lists as an AI model, agent, or automated system, from any developer, public or not, with whatever harness, tools, multi-agent or worker-critic arrangement, time limit, and budget Scale uses or permits; Scale may list its best overall rate across separately evaluated environments. No person may produce, select, edit, or review its deliverables beyond submitting the project brief, and it may not access the reference deliverable or evaluators' judgements of its attempts. A rate crediting a project if any one of several attempts, configurations, or systems passes does not count.
• At least 90%: 90.00% or more, using the most precise figure Scale publishes for the entry, or the exact quotient where Scale publishes projects automated and set size. A figure published only in rounded form is used as published; do not round figures or infer unpublished digits.
• Qualifying successor: a benchmark that Scale, alone or with CAIS, explicitly designates as a new version of or successor to RLI, provided its projects are held-out, paid, multi-sector remote-work projects with professional reference deliverables, and its headline metric keeps RLI's pass/fail line (human evaluators judge the AI deliverable at least as good as the professional's) under any rating scale; the threshold then applies, without rescaling, to the share of the successor's official evaluation set classified as automated. A headline metric that gives partial credit or relies on automated graders does not qualify. The original RLI continues to count alongside any successor.

Resolves YES if a qualifying publication, first published within the Resolution window and on or before the quarter-end, reports an automation rate of at least 90% for an AI system on the official evaluation set of RLI or a qualifying successor, or explicitly states that a system automated at least 90% of that set. Point estimates govern; no significance test, replication, or minimum time on the leaderboard is required.

Resolves NO / Exclusions. Resolve NO at a horizon if no qualifying publication has appeared by then. A figure Scale withdraws or corrects as erroneous within 14 days of first publication does not count; later changes do not reverse a YES. If Scale stops publishing RLI results, no other benchmark, evaluator, or estimate substitutes.

Resolving source. Scale AI only, alone or jointly with CAIS: the RLI leaderboard (labs.scale.com/leaderboard/rli); RLI papers, reports, and posts, including revisions of arXiv:2510.26787 and remotelabor.ai; and CAIS-authored RLI results posts on safe.ai or remotelabor.ai that report human evaluations conducted by Scale. Figures from AI developers, other evaluators or aggregators, or users of the open-source RLI platform do not count, even if they cite a Scale evaluation. This is narrower than Public confirmation (A2). Apply the Resolution window and Resolution date (A3) conventions, dating a figure by its first dated release or, failing that, the earliest archived snapshot of the leaderboard showing it.