Minimum Joint Guardrail Disclosure

The disclosure ladder. Each step up reports strictly more of the joint structure. Step 3 is the campaign's ask — one union row and one all-miss row beside the marginals every evaluation already prints. Step 4 makes every other step recomputable by anyone.

Turn this draft into a reviewable packet: the browser-local Stack Study Preflight records the system, observation mode, denominator, and static aggregate checks together. It computes only declared full-exposure static evidence; deployed routes and adaptive tests remain separate protocol objects.

The fourteen components

Components 1–6 make the marginals interpretable; most careful evaluations already publish them. Components 7–9 are the missing column. 10–13 make it trustworthy. 14 makes it reproducible.

Population and denominator
What set of items, how many, and where they came from. Every joint statistic below is a fraction of this set.
Event definition
What counts as a positive — the thing a guard should catch — stated once, identically, for every system.
Per-guard configuration
Version, threshold, and settings for each guard. A threshold moved between guards silently changes what a comparison means.
Same-items confirmation
An explicit statement that every guard was evaluated on the same items. Same benchmark name is not the same item set.
Full exposure
Every guard saw every applicable item. If an earlier guard's block gated later guards, say so — gated and ungated numbers answer different questions.
Per-guard counts
Catches among positives and false positives among negatives, as counts with denominators, not only as rates.
Union detection
Items caught by at least one guard, among positives on the stated full-exposure item set. It is unrecoverable from marginals.
All-miss rate
Items caught by no guard, among positives on that same static item set. It equals 100% minus union detection; it is not, by itself, terminal deployment risk under routing, gating, or adaptation.
Residual coverage
For each added guard: what it catches among the items the preceding set missed. This is the measured value of adding the guard.
Intersections
Pairwise (and higher-order, where feasible) overlaps of catches or misses. Pairwise alone does not determine the higher-order structure; it still constrains it.
Uncertainty
Intervals for the joint statistics, not only the marginals. A union estimate without uncertainty invites overreading.
Missingness
Errors, refusals, and timeouts, and how each was scored. A timeout scored as a catch is a decision, not an accident.
Order semantics
For sequential stacks: the order, and what a block at stage k means for the stages after it.
Per-item release
One row per item with each guard's decision, when license and safety permit. This single artifact makes every statistic above recomputable by anyone.

The paste-in row

For a results table that already lists per-guard rates, the minimum viable disclosure is two added rows and three lines of caption:

| System                  | Catch rate on the positive set |
|-------------------------|--------------------------------|
| Guard A (version, thr.) | 91.0%  (910 / 1,000)           |
| Guard B (version, thr.) | 88.0%  (880 / 1,000)           |
| Any guard — union       | __._%  (___ / 1,000)           |
| No guard — all-miss     | __._%  (___ / 1,000)           |

Denominator: 1,000 positives, defined as <event definition>.
Every guard scored every item independently (no gating).
Errors/timeouts: <n>, scored as <policy>.

The row is meaningful only if: the denominator and event definition are stated; every guard scored the same items; every guard scored every item (or the gating is declared as the object of measurement); and missingness is scored by a stated policy. Absent those, a union number is not evidence about the stack.

The row, demonstrated on public data

BELLS's 2025 misuse-detection study released 170 prompts with eleven systems' decisions as columns. The census records a second aligned per-item release — Multimodal Safeguard Bench — whose three-guard adapter-bit ORs, all-zero counts, leave-one-out bit ORs, and full native-label pattern tables are likewise computed from the bound files and registered as claim MC-004 (python scripts/reanalyze_msbench.py). This page demonstrates the BELLS-specific arithmetic because MC-002 binds this exact file and its five specialized supervisors:

The minimum joint disclosure computed on the released BELLS subset: per-guard catch rates, union, and all-miss over 82 harmful prompts.
System	Catch rate, 82 harmful prompts
Lakera Guard	63.4% (52 / 82)
Prompt Guard	6.1% (5 / 82)
LangKit	30.5% (25 / 82)
NeMo Guardrails	85.4% (70 / 82)
LLM Guard	0.0% (0 / 82)
Any guard — union	89.0% (73 / 82)
No guard — all-miss	11.0% (9 / 82)

The product of the five individual miss rates is 3.5%: an independence plug-in reference. The release-recomputed all-miss in this static OR aggregation is 11.0%: about 3.1× that plug-in on this subset. The same union flags 19 of the 50 benign prompts — a separate static benign-union column needed to interpret this aggregation, not a deployment utility assessment.

The complete co-miss partition

The table above reports five marginal catch rates. This grid keeps the same 82 harmful prompts but shows every exclusive miss cell: a row describes exactly which of the five released verdict columns missed a prompt, and no other displayed guard did. Rows are ordered by the number of misses, from all five to none.

Text-only cell counts

miss pattern	guards that miss	count	cell state
11111	Lakera Guard, Prompt Guard, LangKit, NeMo Guardrails, LLM Guard	9	observed
01111	Prompt Guard, LangKit, NeMo Guardrails, LLM Guard	3	observed
10111	Lakera Guard, LangKit, NeMo Guardrails, LLM Guard	0	observed zero
11011	Lakera Guard, Prompt Guard, NeMo Guardrails, LLM Guard	0	observed zero
11101	Lakera Guard, Prompt Guard, LangKit, LLM Guard	18	observed
11110	Lakera Guard, Prompt Guard, LangKit, NeMo Guardrails	0	structural zero
00111	LangKit, NeMo Guardrails, LLM Guard	0	observed zero
01011	Prompt Guard, NeMo Guardrails, LLM Guard	0	observed zero
01101	Prompt Guard, LangKit, LLM Guard	26	observed
01110	Prompt Guard, LangKit, NeMo Guardrails	0	structural zero
10011	Lakera Guard, NeMo Guardrails, LLM Guard	0	observed zero
10101	Lakera Guard, LangKit, LLM Guard	0	observed zero
10110	Lakera Guard, LangKit, NeMo Guardrails	0	structural zero
11001	Lakera Guard, Prompt Guard, LLM Guard	3	observed
11010	Lakera Guard, Prompt Guard, NeMo Guardrails	0	structural zero
11100	Lakera Guard, Prompt Guard, LangKit	0	structural zero
00011	NeMo Guardrails, LLM Guard	0	observed zero
00101	LangKit, LLM Guard	1	observed
00110	LangKit, NeMo Guardrails	0	structural zero
01001	Prompt Guard, LLM Guard	18	observed
01010	Prompt Guard, NeMo Guardrails	0	structural zero
01100	Prompt Guard, LangKit	0	structural zero
10001	Lakera Guard, LLM Guard	0	observed zero
10010	Lakera Guard, NeMo Guardrails	0	structural zero
10100	Lakera Guard, LangKit	0	structural zero
11000	Lakera Guard, Prompt Guard	0	structural zero
00001	LLM Guard	4	observed
00010	NeMo Guardrails	0	structural zero
00100	LangKit	0	structural zero
01000	Prompt Guard	0	structural zero
10000	Lakera Guard	0	structural zero
00000	no guard	0	structural zero

The all-miss value is one cell, not an endpoint. The 11111 row is the registered 9 / 82 all-miss count. LLM Guard catches 0 / 82, so the 16 cells where its dot would say “catch” are structurally empty; they are not measured zeroes. The other 8 zero-count cells were possible under those marginals but did not occur in this released file. Counts describe the author-selected BELLS subset at its released native verdicts, not a vendor, a population, or a deployed stack; no item identities are published.

Scope, stated before anyone asks: the released 170 prompts are an author-selected subset (of 990 non-adversarial prompts; the study's ~4,165 adversarial prompts have no per-item release) under an unstated selection rule — so these are counting facts about exactly that file, not estimates of any system's true rate, and no interval is offered because the sampled population is undefined. The full envelope, falsifier, and forbidden rescues: claim MC-002. Reproduce it: python scripts/reanalyze_bells_subset.py — the file is hash-verified before a single count is taken.

Machine-readable MJGD v1

The human template above now has a small structural JSON schema, implementation notes, and a standard-library validator. Use the validator for semantic conformance: it checks declared evidence boundaries and arithmetic, not safety, calibration, route risk, or adaptive robustness.

MJGD v1 conformance fixtures and their validated states
Illustrative fixture	Validator state	What that state means
parallel-full-exposure.json	recomputed static	Complete binary full-exposure outcomes; every printed positive and benign count is recomputed.
sequential-route.json	held: route declaration	A routed or gated declaration is held, not collapsed into static full-exposure arithmetic.
partial-release.json	recomputed aggregate patterns	A complete controlled aggregate-pattern table; positive static results are recomputed without item identities.
aggregate-only.json	not identified	Marginals only; the exact finite all-miss identified set is shown instead of inventing a realized joint result.
missing-data.json	held: missing data	An explicit timeout, error, or non-exposure cell is held rather than silently scored.

One recomputed fixture, not a benchmark result: 4 positives, A catches 2, B catches 2, any guard catches 3, and all guards miss 1. Its benign union is 1 of 1. These are synthetic fixture counts, included to make the contract replayable rather than to characterize any deployed system.

python scripts/validate_mjgd.py --test      # five fixture states + refusal tests
python scripts/validate_mjgd.py --fixtures  # inspect every committed fixture

The ask, for benchmark authors

You evaluated multiple guardrails on a common benchmark. Did you retain one binary decision per item for every system? If so, would you consider publishing the union detection rate and the corresponding all-miss rate, together with the denominator and event definition? Those two rows identify static all-miss for the declared full-exposure evaluation without assuming independence — and I will gladly supply the calculation or a small reporting patch: bhavepranavwork@gmail.com.

Reference implementation

scripts/mjgd_reference.py computes every static component above from one decision per item per guard — union, all-miss, residual coverage in stack order, and pairwise intersections — and asserts its own identities (union + all-miss = denominator; residual coverage telescopes to the union; intersections respect their feasibility bounds) against synthetic fixtures in CI. It is ~a hundred lines, and it is the entire cost of the disclosure when per-item decisions were retained.

What this page does not claim: that any organization has adopted this schema; that the missing statistics, once measured, would show strong dependence; or that disclosure alone makes a stack safe. The census tracks the first; measurement — not assumption — settles the second; nothing settles the third.