Pranav Bhave
Answer — for teams justifying a second detector evidence: MC-003

What does the second guardrail catch that the first one misses?

This is the question that justifies the second detector's cost, latency, and false positives. No column in a standard guardrail results table answers it.

Residual coverage is a different quantity

Say detector A catches 80% and detector B catches 70%. B's headline number is measured over all items. The question you actually have is narrower: among the 20% that A let through, how many does B stop?

That conditional rate can be anything from 0% to 100% while both headline numbers stay exactly as published. A second detector that is excellent overall and blind to precisely A's blind spots contributes nothing to the stack — and its published score looks identical to one that closes every gap.

Residual coverage is not derivable from marginal scores. It is a property of how the detectors fail together, and per-detector columns contain no information about that.

The measurement is cheap if you kept the items

With one decision per item per system, residual coverage is a filter and a count: take the items the first guard missed, and count how many the second caught. No new experiment, no new inference.

If you cannot release per-item outcomes, there is a compact alternative that identifies the same thing. Publish the union with each guard removed in turn — a leave-one-out union. The gap between the full union and the union without guard g is exactly the set of items only g catches: its exclusive contribution to the stack.

What that looks like on real data

On the harmful stratum of the BELLS 2025 released subset (82 prompts, five specialized supervisors, union catches 73), the registered leave-one-out unions identify each supervisor's exclusive full-stack coverage:

18nemo
3lakera guard
0prompt guard
0langkit
0llm guard

Three of the five contribute nothing exclusive on this stratum: remove them and the union is unchanged. Marginal catch counts alone would have bounded each guard's exclusive coverage only loosely — for example 0 to 21 items for the strongest member. The leave-one-out unions identify the realized values.

Scope, strictly. One author-selected subset, one stratum, at unstated default configurations. This is not a vendor ranking, not a causal attribution, and not evidence about any product outside this file. One supervisor fires exactly once in the whole released set. Leave-one-out unions identify exclusive coverage only — not pairwise overlap, Shapley values, or causal contribution.

The cost side of the ledger

Adding a guard also adds false positives, and that direction is partly identified by the marginals: if any member flags benign traffic at a positive rate, the stack's benign flag rate has a strictly positive floor. On the same released file, the five-supervisor union flags 19 of 50 benign prompts. A second detector that adds no exclusive coverage still adds burden.