The Proof Stops at the Signed Schedule

In the retained Qwen3.8 27B deployment comparison, the BF16 baseline scores 63.00% and the Q5_K_M subject scores 64.50%. The subject-minus-baseline effect is +1.50 pp, with a paired 95% interval of [−0.74, 3.75] pp.

That result passes the comparison's declared policy. But three statements that might appear in a review—“it passed,” “it improved” and “it will work on our cases”—need different evidence. Here, only the first is established under the declared requirement. The interval includes zero, so the higher observed score does not demonstrate improvement or establish equivalence.

A finite schedule is the exact ordered set of cases used for both sides of one comparison. It tells the reviewer which inputs the result covers—and prevents a pass on those cases from becoming a claim about “the model” in general.

One result binds exact identities

An InvarLock public evidence pack identifies one comparison: the two artifacts and runtimes, the ordered cases, the metric and the rule. Those details let a reviewer state the result precisely. The dataset-identity article explains how the exact inputs are bound to the evidence.

For this example, the defensible statement is that the named Qwen3.8 BF16 and Q5_K_M deployments met the recorded exact-match requirement on the retained 400-question MMLU-Pro schedule. Replacing that sentence with “Qwen3.8 is better” drops the cases, comparison and uncertainty that give the result its meaning.

Diagram

A signed 400-record schedule establishes integrity and replay inside one declared-policy boundary, while new tasks, safety, recipient acceptance, and deployment remain outside.

Where the comparison stops

Exact comparison

Artifacts, runtimes, metric, policy, and one ordered 400-record schedule.

Integrity and replay

Authenticate the retained bytes against independent trust inputs, then recompute the metric, interval, and policy.

Separate questions

New tasks and safety need other evidence. Recipient acceptance and deployment need their own authority.

Replaying retained records does not reproduce the original model execution.

The figure separates checking the recorded result from extending its claim. Verifying the same records again does not measure a new task or establish permission to deploy.

The paired result is useful because it is bounded

The v0.15.0 public index used here contains seven evidence packs. Each uses 400 paired records selected deterministically from a pinned public dataset. Each public-evidence entry points to one immutable pack and receipt; the seven rows are not samples to pool into one aggregate claim.

Pairing preserves which outcomes changed because both sides face the same cases. For the Qwen3.8 example, the 1.50-point gain is six more correct answers out of 400. Its interval still allows a small loss, and the policy tolerates a loss of up to two points.

An uncertainty interval does not answer every question about uncertainty. It describes the comparison under the declared method. It does not measure what happens when customer requests differ from the selected benchmark questions.

Report v3 closes one policy gap

The three BF16-to-Q5_K_M deployment packs use invarlock/comparison-report-v3: at least 400 records, an interval no wider than 10 points, a lower bound of at least −2 points, and at least 20% accuracy on each side. These controls make “pass” more specific. They do not make the cases representative of a different workload.

The four-pack article explains why the absolute accuracy floor matters: the earlier relative-only rule could pass two zero-scoring engines. Adding that floor addresses a policy gap; it does not establish that 20% is adequate for every use.

The other four public packs retain invarlock/comparison-report-v2 semantics. Verification replays them under their original contracts rather than silently adding v3 controls to historical evidence.

Wider questions need new evidence or new authority

Suppose a team keeps the same model files but wants to use them for customer support. The original 63.00% and 64.50% scores still describe MMLU-Pro. The team needs to identify the customer behavior it cares about and cases that exercise it; copying the old record count or tolerance does not justify those choices.

Write the intended claim before choosing the follow-up. If it concerns a new workload, gather evidence for that workload. If it concerns a different requirement on the same saved answers, the recheck-or-rerun example explains when reanalysis suffices.

The useful outcome of this review is a precise sentence about what the result supports, followed by the unresolved question that matters to your use. The recipient checks then help the receiving team decide whether that evidence is enough to accept the handoff.

Limitations

• The seven packs in this release snapshot are curated public examples, not a representative sample.
• Their MMLU-Pro and MMMU-Pro Vision schedules cover named evaluation surfaces, not general capability or safety.
• Verification reconstructs retained comparison evidence and report arithmetic; it does not rerun inference or independently prove the origin of the measurements.