Daybook AI news by event, demo desk
Demo

Demo desk. Every organisation, source and story here is fictional and frozen at the demo clock. Nothing on this site is current reporting. Demo clock: Thu 12 Mar 2026, 09:00 UTC, frozen. About the demo

Follow-up Rising +13.4 in 6 h incomplete: Builders Forum top threads is stale Demo event

ReasonBench v2: test items found in public training data; scores under review

Maintainers say about one in eight test items appear verbatim in a widely used web corpus. Leaderboard scores are frozen while affected items are removed.

Latest development, 12 Mar, 07:00

ReasonBench corrections reshuffle the top of the leaderboard

Two models swap places at the top after the cleaned set is applied; the gap between them is within the benchmark's stated margin.

Heat at demo clock
30.1
Independent sources
5
Reports
6
First seen
Last report
In briefings
Wed 11 Mar, Thu 12 Mar

Timeline and provenance

Every report on this event, oldest first. All times UTC.

  1. First report

    ReasonBench v2: test items found in public training data; scores under review

    Kestrel Benchmarks Notes Official, tier 1

    Maintainers say about one in eight test items appear verbatim in a widely used web corpus. Leaderboard scores are frozen while affected items are removed.

    Demo report: no original link exists.

  2. Coverage

    A popular reasoning benchmark may be contaminated, its maintainers say

    The Latent Weekly Press, tier 2

    Kestrel Benchmarks has paused ReasonBench scoring after finding test questions in public training data, raising questions about recent leaderboard gains.

    Demo report: no original link exists.

  3. Coverage

    ReasonBench contamination: what it means for the leaderboard

    Builders Forum top threads Community, tier 3

    A thread collecting reactions; several posters ask which published model scores rely on the affected items.

    Demo report: no original link exists.

How the heat adds up

10 × 0.5age / 24 h per independent source, newest report only, within 48 h.

At the demo clock, 12 Mar, 09:00 UTC.
SourceReportAgeWeightPointsWhy
Kestrel Benchmarks Notes10 Mar, 09:002.0 days–0.00older than 48 h
The Latent Weekly10 Mar, 12:0045 h0.2732.73counted
Builders Forum top threads10 Mar, 13:3043.5 h0.2852.85counted
Meridian Models Newsroom11 Mar, 15:0018 h0.5955.95counted
Kestrel Benchmarks Notes12 Mar, 06:003 h0.9179.17counted
Circuit Review12 Mar, 07:002 h0.9449.44counted
Total30.15 sources counted

6 hours earlier, at 12 Mar, 03:00 UTC, the heat was 16.7, so the trend is rising (+13.4). How trends are labelled

Why these reports are one event

Joined when similarity reaches 0.35: 0.6 × shared entities + 0.4 × shared wording, and at least one shared entity.

ReportDecisionEvidence
Kestrel Benchmarks Notes, 10 Mar, 09:00newStarted this event: no open event scored above the threshold.
The Latent Weekly, 10 Mar, 12:00joinedScore 0.682 against the Kestrel Benchmarks Notes report: entities 1.00, wording 0.21. Shared: Kestrel Benchmarks, ReasonBench.
Builders Forum top threads, 10 Mar, 13:30joinedScore 0.362 against the Kestrel Benchmarks Notes report: entities 0.50, wording 0.16. Shared: ReasonBench.
Meridian Models Newsroom, 11 Mar, 15:00joinedScore 0.457 against the The Latent Weekly report: entities 0.67, wording 0.14. Shared: Kestrel Benchmarks, ReasonBench.
Kestrel Benchmarks Notes, 12 Mar, 06:00joinedScore 0.680 against the Kestrel Benchmarks Notes report: entities 1.00, wording 0.20. Shared: Kestrel Benchmarks, ReasonBench. Closest other event: Introducing Orchard 3: open-weight models in three sizes at 0.189.
Circuit Review, 12 Mar, 07:00joinedScore 0.657 against the Kestrel Benchmarks Notes report: entities 1.00, wording 0.14. Shared: Kestrel Benchmarks, ReasonBench. Closest other event: Introducing Orchard 3: open-weight models in three sizes at 0.174.

Named in this event

ReasonBench, Kestrel Benchmarks, Meridian Models