Preregistration → Execution

The plane

An experiment is not a preregistration plus code plus data plus a result. It is a typed graph, and the edges are the scientific objects. A result is admissible only when every edge holds.

DECLARED P ── BOUND X ── EXECUTED R/D ── VALIDATED O ── CLAIMED C

A(C) = F(P) & B(P,X) & E(X,R) & U(P,D) & Q(P,D) & V(R,D,O) & T(O,C)

The founding cohort

Retrospective. These five motivated the plane; they do not measure it. The classification was partly built after the failures were visible, so 2 of 5 valid is diagnostic evidence, not an estimate. Of 5058 observation rows, 4800 stand and 258 belong to voided or rejected runs. Frozen inputs, estimators and outputs are unchanged in every case; 19 files are held in the preserved-byte manifest at baseline commit e7c6b9d2677b.

What now intercepts these

3 regression fixtures, each reconstructing a defect that actually voided a run here, on synthetic data with a known answer. Each asserts that the defective rule still gets it wrong, so a fixture that loses its teeth fails loudly instead of passing vacuously. 7 mutant declarations carry one of the three defects each and must be rejected: a detector nobody has watched fire has no demonstrated discriminatory power.

Prospective coverage

An experiment opts in by committing a machine-readable contract beside its preregistration, intended for its runner to consume rather than restate. The current scan checks declaration fields and a textual reference in the runner; it does not prove runtime consumption. Frozen preregistrations are reported as undeclared and never failed: gating a frozen file after its outcomes are visible is itself the rescue this repository forbids.

2 of 6 declared   e3 e3b e7 e7b e8 e9

Unresolved blind spots

The edges with nothing behind them. Naming them here is the only thing currently done about them.

• U — executed universe is the declared universe
• Q — design informative enough for its stated target
• V — execution validity
• T — claim mechanically extracted from that output

Reproduce

git clone https://github.com/Cubits11/cubits11.github.io && cd cubits11.github.io
python3 scripts/verify_prereg.py --test    # the fixtures and the mutants
python3 scripts/direction.py               # the heading this page projects
python3 scripts/generate_conformance_record.py --check   # this page, against its sources

Sources

Every scientific number above is read from one of these files at render time. Hashes are of the bytes this page was generated from.