Pranav Bhave
Claim MC-002 — 9 of 21 in the registry Supported within scope

MC-002Warrant reviewed 2026-08-28 · expires 2026-10-27

Five supervisors on BELLS against the independence plug-in

On the 170-prompt per-item subset publicly released with BELLS's 2025 misuse-detection evaluation (pinned commit below), the five specialized supervisors — Lakera Guard, Prompt Guard, LangKit, NeMo Guardrails, LLM Guard — have an OR-union of released binary verdict columns that flags 73 of the 82 prompts labelled harmful, leaving 9 all-miss (11.0%). The product of their individual miss rates is 3.5%; the release-recomputed all-miss rate is about 3.1× that independence plug-in on this file. The same OR-union flags 19 of the 50 benign prompts.

01Falsifier — what changes this claim

Recomputing from the bound, hash-verified file yields any count different from the expected block, or the file at the pinned ref no longer matches the recorded sha256, or the released columns are shown not to be the labelled systems' verdicts.

Consequence REJECT

This is the condition and consequence recorded in the registry. The vocabulary this is published in defines what each consequence commits the author to.

02Scope

Exactly the released file at the bound commit: 170 of the evaluation's 990 non-adversarial prompts (the study's ~4,165 adversarial prompts have no per-item release), selected by the authors under an unstated rule. Counting arithmetic only, at the vendors' released binary verdicts; no population estimate, no uncertainty interval, and no claim about any system's true rates is offered or licensed. Computed by scripts/reanalyze_bells_subset.py from scripts/mjgd_reference.py; the expected block below is what CI re-asserts against the downloaded, hash-verified file.

03Forbidden rescues

Repairs declared unavailable in advance; using one after a failure would breach the recorded commitment.

04Non-claims — what this does not license
05Binding and freshness
Binding
non_adversarial_prompts.csv @ 507566c5
The upstream repository declares no license, so the file is cited and hash-verified (sha256 791dd4b0a168f2eb5831b308083a49 2e83200a9fa82585643c739023b03f57c3, asserted by the reproduction script), never redistributed from this record.
Reviewed
2026-08-28 · window 60 days
Expires
2026-10-27 — after this date the recorded review is overdue; this does not make the claim false
Triggers
  • executable fires when the released subset on the default branch changes
  • manual the authors release the full per-item dataset, state the subset's selection rule, or correct the released verdicts
Dimensions
visibilityPublicprovenanceMachine-generated, owner-executedsupport roleExecuted outputmaturityExperimental