The Proof Stops at the Signed Schedule
What a verified comparison establishes depends on its exact records, declared contract, and independently supplied trust inputs.
In the retained Qwen3.8 27B deployment comparison, the BF16 baseline scores 63.00% and the Q5_K_M subject scores 64.50%. The subject-minus-baseline effect is +1.50 pp, with a paired 95% interval of [−0.74, 3.75] pp.
Under its declared contract and independent trust inputs, that result establishes a policy pass for this comparison. The interval includes zero, so the positive point estimate does not demonstrate improvement or establish equivalence. The pass does not authorize either artifact to ship.
A finite schedule is the exact ordered set of records used for both sides of one comparison. Its fixed scope makes the result reviewable and keeps a policy pass from becoming a claim about “the model” in general.
One result binds exact identities
An InvarLock public evidence pack does not contain a free-floating score. It binds the baseline artifact, subject artifact, dataset bytes, selected record identities, ordered paired schedule, baseline and subject runtimes, metric, and policy. Together, those inputs and outputs form one evaluation transaction.
Those identities define the transaction. Change the artifact, runtime, task, prompt format, records, metric, or threshold and the old result does not automatically follow. The changed question needs a new transaction.
The signature is part of that boundary, not a substitute for it. A signature authenticates bytes and a signer identity. Independent verification then checks the closed inventory, hashes, trust inputs, cross-bindings, retained records, and report arithmetic.
Where the comparison stops
- Exact comparison
- Artifacts, runtimes, metric, policy, and one ordered 400-record schedule.
- Integrity and replay
- Authenticate the retained bytes against independent trust inputs, then recompute the metric, interval, and policy.
- Separate questions
- New tasks and safety need other evidence. Recipient acceptance and deployment need their own authority.
Replaying retained records does not reproduce the original model execution.
Read from the center outward. The finite schedule fixes the exact transaction. Integrity authenticates its bytes and bound identities. Replay recomputes the metric, interval, and policy arithmetic from the retained records. The outer boundary returns one result under the declared thresholds. None of those layers answers the four questions beyond it.
Verification can issue a fresh verifier-signed receipt and render the report without rerunning model inference. Replay reconstructs the retained records and arithmetic; it does not reproduce the original execution that created those records. A stronger execution claim needs separate attestation or an independently controlled rerun.
The paired result is useful because it is bounded
The v0.15.0 public index used here contains seven evidence packs. Each uses 400 paired records selected deterministically from a pinned public dataset. Each public-evidence entry points to one immutable pack and receipt; the seven rows are not samples to pool into one aggregate claim.
Pairing gives each transaction a strong local interpretation: baseline and subject face the same ordered semantic records, so the comparison preserves which outcomes changed on each record. The report can then recompute the effect and its paired uncertainty interval.
That Qwen3.8 result says something precise about its artifacts, runtimes, records, and scoring rules. Its paired records show which answers changed, and its interval measures uncertainty around the effect.
Report v3 closes one policy gap
Three of those packs compare full-precision BF16 checkpoints with source-derived five-bit Q5_K_M deployments in the GGUF local-inference format. They use invarlock/comparison-report-v3. Each declared policy requires:
- at least 400 records;
- a paired interval no wider than 10 percentage points;
- the paired interval's lower bound to be at least −2 percentage points; and
- at least 20% accuracy for both baseline and subject.
The tagged policy records those controls directly:
{
"resolved_policy": {
"metrics": {
"exact_match": {
"delta_min_pp": -2.0,
"maximum_interval_width_pp": 10.0,
"minimum_record_count": 400,
"minimum_side_accuracy": 0.2
}
}
}
}
The side-accuracy floor matters. Without it, two poorly scoring sides could satisfy a relative-change rule while remaining inadequate on the task. The floor makes that particular policy harder to pass.
The 20% floor belongs to this comparison policy; it is not a universal capability threshold or a safety test. A pass means that these four controls passed for the bound transaction.
The other four public packs retain invarlock/comparison-report-v2 semantics. Verification replays them under their original contracts rather than silently adding v3 controls to historical evidence.
Wider questions need new evidence or new authority
The signed schedule answers a reviewable question. Wider questions remain outside its established boundary:
- A new task, dataset, prompt format, or population needs a new schedule.
- A new artifact or runtime needs new bound identities.
- Safety, privacy, robustness, and compliance need evidence designed for those questions.
- Recipient acceptance needs recipient-owned policy and current trust inputs.
- Deployment approval remains an organizational decision.
This is the practical value of a finite-schedule claim: the result travels with enough identity and arithmetic for another reviewer to verify it, while its limits tell that reviewer what must happen next.
Limitations
- The seven packs in this release snapshot are curated public examples, not a representative sample.
- Their MMLU-Pro and MMMU-Pro Vision schedules cover named evaluation surfaces, not general capability or safety.
- Verification reconstructs retained comparison evidence and report arithmetic; it does not rerun inference or independently prove the origin of the measurements.
Sources
Website documentation explains the maintained workflow. The measurements, seven-pack inventory, and report contracts described here use InvarLock v0.15.0; the tagged index preserves that evidence basis.
More in Research Note
Explore nearby related posts.
Research note
A Passing Release Test Can Contain Policy Rejections
In the retained preflight example, all eleven references must reproduce their expected outcomes: eight policy passes and three integrity-valid policy rejections.
Research note
A Signed Score Is Not Yet Decision Evidence
A signed evaluator output becomes decision evidence only when its records can be independently replayed under an authorized contract.
Research note
Three GGUF Deployment Comparisons Under One Review Standard
Three BF16-to-Q5_K_M comparisons pass the same declared policy, each with its own evidence and receipt. The repeatable result is the review standard.