Correct the record

Research claim dispositions

The previous coupling campaign is superseded for launch purposes. These are corrections to this repository’s analyses. Frozen inputs, estimators and outputs remain unchanged. CORRECT records the history transition while REJECT records the disposition; the claim IDs remain addressable.

Preserved-byte manifest · Reproducible audit · Constructed teaching example

Policy

1. Send the exact row ID, the disputed field, and a stable primary-source locator through a repository issue ↗ or email.
2. Every report is logged publicly the same calendar day it is received: either as a verified correction or as an explicit under-review entry. Silence is not a resolution.
3. A verified correction updates the source row, all mechanically derived counts, generated pages, and the revision history in one reviewable change. The report is credited in the affected row where consent permits.
4. If evidence remains ambiguous, the row is weakened to an explicit ambiguity rather than retained by confidence or rewritten criteria.

Public revision history

The full record is also embedded beside the census rows; this route exists so a citation, post, or correction request always has one stable destination.

• 2026-09-01 — ml6-2026 preservation annotation: the row now records archive capture 20260829070626 as ARCHIVE_URL_RECORDED. The capture is the one produced by the row's own 2026-08-28 Save Page Now acceptance; it was printed as an HTTP 200 capture meeting the reviewed-freshness bar by the corrected two-index wayback gate on 2026-09-01, and a same-day independent liveness re-check was throttled (HTTP 429) — which is why the state is ARCHIVE_URL_RECORDED rather than ARCHIVED_VERIFIED, exactly as the preservation vocabulary defines the distinction. No evidential field, classification, count, or ladder value changed; N/M/K remain 20/14/5. v0_row_lock re-minted.
• 2026-09-01 — Wayback precision. The pre-freeze-visibility-evidenced premise and the earlier entry of this date said checks on 2026-09-01 "found no web-archive capture"; the only archive queried was the Wayback Machine's CDX index, which held no capture of the repository at any date. archive.today and other archives were not queried. The premise now names the archive checked; the earlier entry stands as written rather than being silently reworded, per this record's correction rule. No row, classification, count, or sensitivity envelope changed.
• 2026-09-01 — Date-evidence sensitivity declared for the 2026-08-30 addition. The row multimodal-safeguard-bench-2026 entered on GitHub-recorded dates (repository created 2026-06-02, pinned snapshot dated 2026-07-13); neither stamp records when the repository became publicly visible, and checks on 2026-09-01 found no web-archive capture of the repository at any date and no forks. That gap was prose in the 2026-08-30 entry; it is now the declared sensitivity pre-freeze-visibility-evidenced, whose alternative envelope is mechanically asserted: excluding the row yields N/M/K 19/13/4 and an M ladder of 13/12/0, while the ABSENT classification count stays 14 because the excluded row is PRESENT. The two headline 14s — the no-joint-artifact classification count and the shared-basis ladder rung — are different partitions of the same 20 examined rows, and only the second depends on this row. A statement from the row's author is the evidence that would settle the reading either way. No row, classification, evidence field, or primary count changed.
• 2026-09-01 — Contact routes de-harvested. Nine rows printed 20 author email addresses verbatim in contact_route; census.yaml is served publicly as machine-readable text/yaml at a crawlable URL, which made those rows a scrapeable contact list. Addresses are now written in the [at]/[dot] form this file already used elsewhere, so a human can still route a correction while a harvester cannot lift the list. No evidential field, classification, count, or ladder value changed: N/M/K remain 20/14/5. Recorded here rather than as nine per-row corrections because nothing about any source's evidence was corrected — only how this file republishes third-party contact details. v0_row_lock re-minted.
• 2026-08-31 — Action-layer clarification for multimodal-safeguard-bench-2026. A source check of src/msbench/pipeline.py established that the harness uses `blocked` items to suppress target generation. A fixed block-on-any OR therefore remains a valid counterfactual harness-action calculation on the pinned rows; it does not restore a source-defined shared-event safety or catch interpretation. N/M/K and the M ladder remain 20/14/5 and 14/12/0. This corrects this record's scope, not the authors' counts, their disclosed pairwise compositions, or any deployed route.
• 2026-08-31 — Semantic-scope correction for multimodal-safeguard-bench-2026. The row's old same-event evidence incorrectly treated the harness's normalized `blocked` bit as the shared event. Under the frozen protocol, that field concerns the common HarmBench/XSTest ground-truth label and remains yes. A new native_action_translation annotation records that no source-defined map from all native guard predicates to one catch event was found. MC-004's counts are retained as adapter-bit arithmetic and no longer called a shared-event catch union; N/M/K and the M ladder remain 20/14/5 and 14/12/0. This corrects this record's semantic framing, not the authors' counts or their disclosed pairwise compositions.
• 2026-08-31 — v0 frozen as a historical snapshot. Every examined row annotated with protocol-v1 vocabulary (source_type, reconstruction class, preservation state); v0_row_lock minted so history cannot change silently. Two classifications narrowed by the annotation: 14 marginals-only rows are recorded as PARTIALLY_IDENTIFIED rather than implying the joint is unknowable (comparable marginals bound it by sharp Frechet-Hoeffding limits), and circleguardbench-2025 is recorded as UNVERIFIED because its shared basis is unstated in the examined surfaces. No N/M/K value changed; no row was reclassified. Prospective collection rules for v1 were written and committed before the first v1 observation (census_protocol_v1.yaml).
• 2026-08-27 — Census established. Criteria v1 inclusion wording was committed and is now locked in repository history; that lock is a reproducibility record, not an independent preregistration. The four starting cases entered as under_review pending primary-source examination.
• 2026-08-27 — First examination pass completed: 19 rows examined (one over the ~18 budget estimate, recorded here), 9 candidates excluded by rule, 15 surfaced candidates left unexamined. Pre-publication schema clarifications, made before any count was public: the ABSENT definition no longer presupposes comparability (that axis is counted by M); joint_scope value printed_pairwise_only generalized to printed_partial_stack. Corrections to the starting brief: the single BELLS row split into three artifacts (2024 framework — excluded as a framework paper; 2025 misuse-detection evaluation; 2026 BELLS-O), and the artifact remembered as "MSBench with two pairwise ensemble rows" is actually UnsafeBench (arXiv 2405.03486), which prints six OR-ensemble rows but evaluates image-platform safety classifiers and therefore fails frozen domain criterion 5 — recorded prominently under exclusions. LlamaFirewall added from the snowball hop.
• 2026-08-27 — Demonstration computed on the one per-item outcome release found by the bounded search: union, all-miss, and residual coverage for the five specialized supervisors in BELLS 2025's released 170-prompt subset, registered as claim MC-002 (file bound by commit and sha256, reproduction script in CI) and rendered on the disclosure page. Census counts are unchanged — MC-002 is this record's own computation, not something the examined artifact printed.
• 2026-08-27 — Fresh-context adversarial verification, run same day and before anything was published, independently reproduced N/M/K = 19/13/4 and every MC-002 count, then found: (1) the wainjectbench-2025 reason line falsely called its ensemble members "academic and open-source" — Ensemble-I includes GPT-4o-Prompt, a commercial closed API; the row is corrected, and claim MC-001's commercial-API clause is withdrawn rather than reinterpreted, exactly as its forbidden rescues require. (2) A generator bug was deleting the letter "n" from the rendered criteria and exclusion notes on the public page (a malformed regex character class) — fixed, with the second exclusion rule's YAML typing corrected and the verifier extended to type-check exclusion rules. (3) The frozen phrase "separately attributable" had not said whether a product name was required. Rather than calling a post-review clarification pre-frozen, the record now exposes both readings: the primary treatment counts consistently distinguished, anonymized systems as separately attributable; the named-products-only sensitivity excludes unit42 and mechanically yields N/M/K = 18/12/4. Census schema v2 carries that executed sensitivity and a frozen-wording lock; criteria v1 itself is unchanged. (4) The IBM row's same-items evidence is marked as paraphrase. The primary counts remain N=19, M=13, K=4.
• 2026-08-28 — M ontology correction, made before any merge to main. The single number M was doing work it had not earned: the frozen criteria define "comparable" as shared items plus a shared event definition only, but a reader meets that word expecting matched operating thresholds and full exposure. compute_counts now derives an M ladder mechanically from fields already recorded on every row — 13 shared basis, 12 with no stated threshold mismatch (ML6 states the mismatch), 0 documenting matched thresholds with full exposure — and the page renders all three rungs. The proposition template gained an inline gloss saying which reading M uses. No row's evidence or classification changed; this is a weakening of what the headline implies, not a rescue of it. The strongest rung being 0 is the honest headline result and is now printed as such.
• 2026-08-28 — Adversarial release audit found that the repair still used the word "comparable" in the primary proposition and described the criteria lock as though it proved pre-search timing. Both claims were too strong. The public proposition now names only the mechanical shared-item/common-event basis, calls the four a heterogeneous joint-evidence discovery count rather than one estimand, and says precisely what the repository-history lock proves. No row, classification, or count changed; this is a further narrowing of language before any merge or public post.
• 2026-08-30 — A post-release source audit found a separate qualifying pre-freeze artifact: Patrick Kollman's Multimodal Safeguard Bench. It is not the excluded UnsafeBench artifact named in the 2026-08-27 entry: the new row gates harmful text and rendered-image inputs before a VLM and commits guard outcomes for Llama Guard 4, Llama Guard 3 Vision, and ShieldGemma-2; UnsafeBench evaluates image-generation-platform safety classifiers. GitHub's currently public history records repository creation on 2026-06-02 and the pinned fb6f32e6 snapshot dated 2026-07-13, before the frozen as-of date; no independent pre-cutoff web archive was identified. The documented search missed it. Under unchanged v1 criteria, the examined PRESENT row changes N/M/K from 19/13/4 to 20/14/5 and the M ladder from 13/12/0 to 14/12/0. This satisfies the prior MC-001 falsifier and rejects its 19/13/4 envelope; MC-001 was re-reviewed as a corrected, superseding 20/14/5 proposition rather than preserving the old result. The frozen inclusion wording and the UnsafeBench exclusion are unchanged.
• 2026-08-30 — Schema v3, prompted by a public self-contradiction rather than by any evidence error. The flagship page derived its headline K from compute_counts and, four paragraphs later, stated a hand-typed K of 4 in its non-claims list. Every gate passed: the census was internally coherent, the HTML matched its generator byte-for-byte, the deployed checksum matched, and the deployed smoke test asserted N and the M ladder but never K. The record's promise is that public claims stay bound to their evidence, so a page publicly stating two values for its own headline quantity is a release blocker and not a typo. Three changes follow. (1) Rows may declare joint_scope_additional, so a row carrying two kinds of joint evidence says so as data; the counts of artifacts that print a composition result, that release aligned per-item outcomes, and that do both are now derived by compute_counts instead of counted by hand in prose. No row's classification, evidence, or primary joint_scope changed, and N, M, K and the M ladder are unchanged at 20/14/5 and 14/12/0. (2) Every census quantity the site states now carries a stable identity, checked against this file on every push and against the served bytes after every deploy. (3) Historical counts must date themselves in markup; an undated superseded envelope now fails CI. This entry records a verification failure, not a measurement failure.

What a correction can change

A correction can change a row, a count, or the scope of a claim. It cannot be used to retroactively narrow a criterion merely to preserve a preferred result. The census's claim envelope and forbidden-rescue rules are public in MC-001.