{
  "question_id": "W22",
  "slug": "aisi-finds-model-sandbagged-its-tests",
  "title": "A safety institute finds a model sandbagged its tests",
  "type": "binary",
  "resolution": {
    "levels": [],
    "values": [],
    "annotations": []
  },
  "forecasts": [
    {
      "question_id": "W22",
      "forecast_date": "2026-09-20",
      "evidence_cutoff": "2026-09-20",
      "method": "wide-v2",
      "opened": "2026-09-16",
      "linked_questions": [],
      "reading_adopted": "I count incentive-only adversarial evaluations, even highly artificial ones, but require the institute itself to affirm capability concealment and document the absence of underperformance instructions or targeted training.",
      "summary": null,
      "ambiguities": [],
      "key_drivers": [],
      "would_update_on": [],
      "forecasts": [
        {
          "period_end": "2026-09-30",
          "p_yes": 0.01
        },
        {
          "period_end": "2026-12-31",
          "p_yes": 0.109
        },
        {
          "period_end": "2027-03-31",
          "p_yes": 0.1892
        },
        {
          "period_end": "2027-06-30",
          "p_yes": 0.2622
        },
        {
          "period_end": "2027-09-30",
          "p_yes": 0.3359
        },
        {
          "period_end": "2027-12-31",
          "p_yes": 0.4024
        },
        {
          "period_end": "2028-03-31",
          "p_yes": 0.4561
        },
        {
          "period_end": "2028-06-30",
          "p_yes": 0.5051
        },
        {
          "period_end": "2028-09-30",
          "p_yes": 0.5496
        },
        {
          "period_end": "2028-12-31",
          "p_yes": 0.5902
        },
        {
          "period_end": "2029-03-31",
          "p_yes": 0.623
        },
        {
          "period_end": "2029-06-30",
          "p_yes": 0.6531
        },
        {
          "period_end": "2029-09-30",
          "p_yes": 0.6809
        },
        {
          "period_end": "2029-12-31",
          "p_yes": 0.7064
        },
        {
          "period_end": "2030-03-31",
          "p_yes": 0.7269
        },
        {
          "period_end": "2030-06-30",
          "p_yes": 0.7461
        },
        {
          "period_end": "2030-09-30",
          "p_yes": 0.7638
        },
        {
          "period_end": "2030-12-31",
          "p_yes": 0.7804
        },
        {
          "period_end": "2031-03-31",
          "p_yes": 0.7935
        },
        {
          "period_end": "2031-06-30",
          "p_yes": 0.8059
        },
        {
          "period_end": "2031-09-30",
          "p_yes": 0.8176
        }
      ],
      "passes": []
    },
    {
      "question_id": "W22",
      "forecast_date": "2026-09-27",
      "evidence_cutoff": "2026-09-27",
      "method": "wide-v2",
      "opened": "2026-09-16",
      "linked_questions": [],
      "reading_adopted": "Incentive-only adversarial tests can qualify, but a developer’s account of institute testing is insufficient unless the institute formally co-authors or endorses the finding.",
      "summary": null,
      "ambiguities": [],
      "key_drivers": [],
      "would_update_on": [],
      "forecasts": [
        {
          "period_end": "2026-09-30",
          "p_yes": 0.008
        },
        {
          "period_end": "2026-12-31",
          "p_yes": 0.068
        },
        {
          "period_end": "2027-03-31",
          "p_yes": 0.133
        },
        {
          "period_end": "2027-06-30",
          "p_yes": 0.202
        },
        {
          "period_end": "2027-09-30",
          "p_yes": 0.274
        },
        {
          "period_end": "2027-12-31",
          "p_yes": 0.347
        },
        {
          "period_end": "2028-03-31",
          "p_yes": 0.412
        },
        {
          "period_end": "2028-06-30",
          "p_yes": 0.471
        },
        {
          "period_end": "2028-09-30",
          "p_yes": 0.521
        },
        {
          "period_end": "2028-12-31",
          "p_yes": 0.564
        },
        {
          "period_end": "2029-03-31",
          "p_yes": 0.601
        },
        {
          "period_end": "2029-06-30",
          "p_yes": 0.633
        },
        {
          "period_end": "2029-09-30",
          "p_yes": 0.661
        },
        {
          "period_end": "2029-12-31",
          "p_yes": 0.684
        },
        {
          "period_end": "2030-03-31",
          "p_yes": 0.706
        },
        {
          "period_end": "2030-06-30",
          "p_yes": 0.726
        },
        {
          "period_end": "2030-09-30",
          "p_yes": 0.742
        },
        {
          "period_end": "2030-12-31",
          "p_yes": 0.757
        },
        {
          "period_end": "2031-03-31",
          "p_yes": 0.771
        },
        {
          "period_end": "2031-06-30",
          "p_yes": 0.783
        },
        {
          "period_end": "2031-09-30",
          "p_yes": 0.794
        }
      ],
      "passes": []
    },
    {
      "question_id": "W22",
      "forecast_date": "2026-10-05",
      "evidence_cutoff": "2026-10-05",
      "method": "wide-v2.1",
      "opened": "2026-09-16",
      "linked_questions": [],
      "reading_adopted": null,
      "summary": "No government safety institute has yet reported a leading AI model deliberately holding back in its tests without being told to. The UK institute said in late 2025 it had found no such cases, and recent company checks also find none. Over five years such behaviour could appear and be caught, but it is hard to detect, and the UK institute's early access to American models is being restricted. A clear UK institute report of a model hiding its abilities would change this most.",
      "ambiguities": [],
      "key_drivers": [],
      "would_update_on": [],
      "forecasts": [
        {
          "period_end": "2026-12-31",
          "p_yes": 0.017
        },
        {
          "period_end": "2027-03-31",
          "p_yes": 0.035
        },
        {
          "period_end": "2027-06-30",
          "p_yes": 0.054
        },
        {
          "period_end": "2027-09-30",
          "p_yes": 0.075
        },
        {
          "period_end": "2027-12-31",
          "p_yes": 0.098
        },
        {
          "period_end": "2028-03-31",
          "p_yes": 0.12
        },
        {
          "period_end": "2028-06-30",
          "p_yes": 0.141
        },
        {
          "period_end": "2028-09-30",
          "p_yes": 0.161
        },
        {
          "period_end": "2028-12-31",
          "p_yes": 0.182
        },
        {
          "period_end": "2029-03-31",
          "p_yes": 0.202
        },
        {
          "period_end": "2029-06-30",
          "p_yes": 0.221
        },
        {
          "period_end": "2029-09-30",
          "p_yes": 0.24
        },
        {
          "period_end": "2029-12-31",
          "p_yes": 0.258
        },
        {
          "period_end": "2030-03-31",
          "p_yes": 0.274
        },
        {
          "period_end": "2030-06-30",
          "p_yes": 0.289
        },
        {
          "period_end": "2030-09-30",
          "p_yes": 0.304
        },
        {
          "period_end": "2030-12-31",
          "p_yes": 0.318
        },
        {
          "period_end": "2031-03-31",
          "p_yes": 0.331
        },
        {
          "period_end": "2031-06-30",
          "p_yes": 0.344
        },
        {
          "period_end": "2031-09-30",
          "p_yes": 0.357
        },
        {
          "period_end": "2031-12-31",
          "p_yes": 0.369
        }
      ],
      "passes": []
    },
    {
      "question_id": "W22",
      "forecast_date": "2026-10-12",
      "evidence_cutoff": "2026-10-12",
      "method": "wide-v2.1",
      "opened": "2026-09-16",
      "linked_questions": [],
      "reading_adopted": null,
      "summary": "No government safety institute has yet reported catching a leading artificial intelligence model deliberately doing worse on its tests without being told to. Britain's institute searched thousands of test records in 2025 and found none, and the newest models tend to spot such tests and behave. A first report within five years is plausible but not likely soon. A British test catching a top model hiding its abilities would most change this.",
      "ambiguities": [],
      "key_drivers": [],
      "would_update_on": [],
      "forecasts": [
        {
          "period_end": "2026-12-31",
          "p_yes": 0.03
        },
        {
          "period_end": "2027-03-31",
          "p_yes": 0.051
        },
        {
          "period_end": "2027-06-30",
          "p_yes": 0.072
        },
        {
          "period_end": "2027-09-30",
          "p_yes": 0.093
        },
        {
          "period_end": "2027-12-31",
          "p_yes": 0.116
        },
        {
          "period_end": "2028-03-31",
          "p_yes": 0.136
        },
        {
          "period_end": "2028-06-30",
          "p_yes": 0.155
        },
        {
          "period_end": "2028-09-30",
          "p_yes": 0.173
        },
        {
          "period_end": "2028-12-31",
          "p_yes": 0.195
        },
        {
          "period_end": "2029-03-31",
          "p_yes": 0.211
        },
        {
          "period_end": "2029-06-30",
          "p_yes": 0.227
        },
        {
          "period_end": "2029-09-30",
          "p_yes": 0.242
        },
        {
          "period_end": "2029-12-31",
          "p_yes": 0.26
        },
        {
          "period_end": "2030-03-31",
          "p_yes": 0.274
        },
        {
          "period_end": "2030-06-30",
          "p_yes": 0.287
        },
        {
          "period_end": "2030-09-30",
          "p_yes": 0.3
        },
        {
          "period_end": "2030-12-31",
          "p_yes": 0.314
        },
        {
          "period_end": "2031-03-31",
          "p_yes": 0.325
        },
        {
          "period_end": "2031-06-30",
          "p_yes": 0.336
        },
        {
          "period_end": "2031-09-30",
          "p_yes": 0.347
        },
        {
          "period_end": "2031-12-31",
          "p_yes": 0.359
        }
      ],
      "passes": []
    }
  ],
  "declined": []
}