Back to blog

The Proof Stops at the Signed Schedule

What a verified comparison establishes depends on its exact records, declared contract, and independently supplied trust inputs.

Updated 5 min readInvarLock Team
A precise trace stays within a bounded grid, while loose contours and isolated points remain outside the measured field.

In the retained Qwen3.8 27B deployment comparison, the BF16 baseline scores 63.00% and the Q5_K_M subject scores 64.50%. The subject-minus-baseline effect is +1.50 pp, with a paired 95% interval of [−0.74, 3.75] pp.

Under its declared contract and independent trust inputs, that result establishes a policy pass for this comparison. The interval includes zero, so the positive point estimate does not demonstrate improvement or establish equivalence. The pass does not authorize either artifact to ship.

A finite schedule is the exact ordered set of records used for both sides of one comparison. Its fixed scope makes the result reviewable and keeps a policy pass from becoming a claim about “the model” in general.

One result binds exact identities

An InvarLock public evidence pack does not contain a free-floating score. It binds the baseline artifact, subject artifact, dataset bytes, selected record identities, ordered paired schedule, baseline and subject runtimes, metric, and policy. Together, those inputs and outputs form one evaluation transaction.

Those identities define the transaction. Change the artifact, runtime, task, prompt format, records, metric, or threshold and the old result does not automatically follow. The changed question needs a new transaction.

The signature is part of that boundary, not a substitute for it. A signature authenticates bytes and a signer identity. Independent verification then checks the closed inventory, hashes, trust inputs, cross-bindings, retained records, and report arithmetic.

Diagram
A signed 400-record schedule establishes integrity and replay inside one declared-policy boundary, while new tasks, safety, recipient acceptance, and deployment remain outside.

Where the comparison stops

Exact comparison
Artifacts, runtimes, metric, policy, and one ordered 400-record schedule.
Integrity and replay
Authenticate the retained bytes against independent trust inputs, then recompute the metric, interval, and policy.
Separate questions
New tasks and safety need other evidence. Recipient acceptance and deployment need their own authority.

Replaying retained records does not reproduce the original model execution.

What to noticeRead from the signed schedule outward: integrity, replay, and one policy result stop before new tasks, safety, acceptance, and deployment.

Read from the center outward. The finite schedule fixes the exact transaction. Integrity authenticates its bytes and bound identities. Replay recomputes the metric, interval, and policy arithmetic from the retained records. The outer boundary returns one result under the declared thresholds. None of those layers answers the four questions beyond it.

Verification can issue a fresh verifier-signed receipt and render the report without rerunning model inference. Replay reconstructs the retained records and arithmetic; it does not reproduce the original execution that created those records. A stronger execution claim needs separate attestation or an independently controlled rerun.

The paired result is useful because it is bounded

The v0.15.0 public index used here contains seven evidence packs. Each uses 400 paired records selected deterministically from a pinned public dataset. Each public-evidence entry points to one immutable pack and receipt; the seven rows are not samples to pool into one aggregate claim.

Pairing gives each transaction a strong local interpretation: baseline and subject face the same ordered semantic records, so the comparison preserves which outcomes changed on each record. The report can then recompute the effect and its paired uncertainty interval.

That Qwen3.8 result says something precise about its artifacts, runtimes, records, and scoring rules. Its paired records show which answers changed, and its interval measures uncertainty around the effect.

Report v3 closes one policy gap

Three of those packs compare full-precision BF16 checkpoints with source-derived five-bit Q5_K_M deployments in the GGUF local-inference format. They use invarlock/comparison-report-v3. Each declared policy requires:

  • at least 400 records;
  • a paired interval no wider than 10 percentage points;
  • the paired interval's lower bound to be at least −2 percentage points; and
  • at least 20% accuracy for both baseline and subject.

The tagged policy records those controls directly:

{
  "resolved_policy": {
    "metrics": {
      "exact_match": {
        "delta_min_pp": -2.0,
        "maximum_interval_width_pp": 10.0,
        "minimum_record_count": 400,
        "minimum_side_accuracy": 0.2
      }
    }
  }
}

The side-accuracy floor matters. Without it, two poorly scoring sides could satisfy a relative-change rule while remaining inadequate on the task. The floor makes that particular policy harder to pass.

The 20% floor belongs to this comparison policy; it is not a universal capability threshold or a safety test. A pass means that these four controls passed for the bound transaction.

The other four public packs retain invarlock/comparison-report-v2 semantics. Verification replays them under their original contracts rather than silently adding v3 controls to historical evidence.

Wider questions need new evidence or new authority

The signed schedule answers a reviewable question. Wider questions remain outside its established boundary:

  • A new task, dataset, prompt format, or population needs a new schedule.
  • A new artifact or runtime needs new bound identities.
  • Safety, privacy, robustness, and compliance need evidence designed for those questions.
  • Recipient acceptance needs recipient-owned policy and current trust inputs.
  • Deployment approval remains an organizational decision.

This is the practical value of a finite-schedule claim: the result travels with enough identity and arithmetic for another reviewer to verify it, while its limits tell that reviewer what must happen next.

Limitations

  • The seven packs in this release snapshot are curated public examples, not a representative sample.
  • Their MMLU-Pro and MMMU-Pro Vision schedules cover named evaluation surfaces, not general capability or safety.
  • Verification reconstructs retained comparison evidence and report arithmetic; it does not rerun inference or independently prove the origin of the measurements.

Sources

Website documentation explains the maintained workflow. The measurements, seven-pack inventory, and report contracts described here use InvarLock v0.15.0; the tagged index preserves that evidence basis.

More in Research Note

Explore nearby related posts.