Safety passed. Workload did not.

On the untouched 281-record holdout, LitSynth made zero false exclusions and protected every relevant record. But 82.9% still required manual review, missing the preregistered 80% workload ceiling. The holdout result is therefore a failure.

TOPIC / CD000996

TASK / Intervention

RECORDS / 281

POLICY / title-abstract-v3

PREREG COMMIT / 8fd1e19

BLIND HASH / 282a24a4830f…

STATUS / FAIL — NO RETEST

Protected recall

100.0%

9/9 relevant records protected · PASS

False exclusions

0.0% of relevant records

Manual review

82.9%

233/281 records · FAIL (limit ≤80%)

Overall result

FAIL

both gates had to pass; no composite accuracy score

01 / Locked before unblinding

The failure is part of the result.

The safety gate passed, but the workload gate failed by 2.9 percentage points. We did not revise the locked policy or rerun the corpus after seeing labels.

LOCK 01
Corpus lock
CD000996: Inhaled corticosteroids for bronchiectasis. The pinned source commit, topic hash, PMID-list hash, and 281-record count are public.
LOCK 02
Policy lock
title-abstract-v3, deepseek-v4-flash, temperature 0.1, batch size 5, and source hashes are fixed.
LOCK 03
Blind first, labels second
All 281 label-free decisions must be written and hashed before the runner is allowed to fetch qrels.
LOCK 04
Safety plus workload
False exclusion and protected recall are published beside UNSURE count and manual-review rate. A safe but unusable review queue cannot pass.

02 / Inspect the complete audit trail

Reproduce the order, counts, and failure.

Full scored predictionspredictions.csv False-exclusion ledgerfalse-excludes.csv Manual-review ledgermanual-review.csv Metrics and gatesmetrics.json Run order and blind hashrun.json Corpus and source hashesmanifest.json Blinded predictionsblinded-predictions.json Machine-readable preregistrationpreregistration.json Human-readable preregistrationpreregistration.md

The 277-record corpus is no longer a performance claim

Its labels informed policy revision, so it remains only as a transparent post-error-analysis regression archive. The untouched holdout was scored once. Its failed workload gate is published and the corpus will not be recycled as untouched evidence.

Open historical development archive

Loading…

Safety passed. Workload did not.

The failure is part of the result.

Corpus lock

Policy lock

Blind first, labels second

Safety plus workload

Reproduce the order, counts, and failure.

The 277-record corpus is no longer a performance claim

Safety passed. Workload did not.

The failure is part of the result.

Corpus lock

Policy lock

Blind first, labels second

Safety plus workload

Reproduce the order, counts, and failure.

The 277-record corpus is no longer a performance claim