InvarLock 0.13.0: One Request, One Signed Decision Trail

One immutable request now fixes every decision-bearing input before execution.

Model evaluation becomes difficult to trust when the important decisions are scattered. One command names the baseline. Another selects a subject. A report remembers the scores but not the exact schedule. The producer signs an output, but the verifier has no independent way to reconstruct the decision.

InvarLock 0.13 replaces that workflow with one evaluation transaction: a closed baseline-versus-subject comparison from request through signed evidence. You run or import both sides against the same ordered records, then let an independent verifier reconstruct the result from separately supplied trust inputs.

At a glance

• request.yaml fixes every decision-bearing input.
• evaluate publishes one signed evidence pack.
• verify treats that pack as untrusted and writes a separate signed receipt.
• report presents the authenticated comparison without creating another source of truth.

The three command shapes are shown below; this is a schematic workflow, not a complete runnable example. Strict verification also requires independent trust inputs, a verifier identity and signing key, and an explicit receipt destination. Use the complete v0.13 verification example for those inputs:

invarlock evaluate request.yaml
invarlock verify evidence/ ... --receipt verification.receipt.json
invarlock report evidence/

The request makes those bindings visible. This abridged import-mode excerpt names both artifacts and fixes the schedule, metric, and comparison policy. It omits required runtime, execution, signing, and output fields; run mode instead uses a pinned dataset object. See the complete tagged request shapes:

comparison:
  baseline: {artifact: {model_id: org/baseline}}
  subject: {artifact: {model_id: org/subject}}
  dataset: inputs/schedule.json
  metric: exact_match
  policy: policy/acceptance.json

Why we closed the transaction

Before v0.13, an operator could assemble a workflow from baseline and subject flags, guard settings, adapters, and report outputs. That flexibility also left too many opportunities for two people to believe they were reviewing the same comparison when they were not.

The new request groups its fixed inputs into three parts: the baseline and subject artifacts; the dataset, schedule, and runtimes; and the metric, policy, execution mode, and evidence destination. Changing any of them creates a new transaction rather than silently changing an existing result.

Diagram

A closed evaluation request binds the exact artifacts, paired schedule, policy, signers, and execution mode before work begins.

One request fixes the comparison

Identity and pairing

Exact baseline and subject artifacts, one ordered schedule, and a selected metric.

Policy and execution

The declared policy, signers, runtime identities, and run or import mode.

Canonical request digest

The authenticated request preserves these inputs. Changing them creates a new transaction.

The request fixes what will be evaluated before execution starts.

That may sound strict, but it removes a surprisingly common ambiguity: was a result produced under the policy being reviewed now, or under a nearby configuration that happened to generate a similar-looking report?

The Evaluation Request guide covers both execution modes. In run mode, InvarLock authenticates local artifacts and data, prepares the schedule, runs each side in a constrained OCI worker, validates the outputs, derives paired statistics, and signs the pack atomically. Import mode accepts authenticated sidecars and record-level results from another controlled execution. An aggregate score alone is insufficient.

Three commands, three different jobs

The short command sequence is not just CLI cleanup. Each command owns a different authority boundary.

Scroll horizontally to see every column.

The report command does not reinterpret the result. The verifier does not borrow expected values from the submitted evidence. The runtime workers never receive the evidence-signing key. Those separations are the point of the transaction, not implementation details around it.

Diagram

Verification stops at the first failed authority boundary, then recomputes pairing, metrics, and policy before signing a separate receipt.

Verify before interpreting the result

Check bytes and authority

Parse the canonical evidence and check signatures against accepted external trust anchors.

Match and recompute

Match the request, artifacts, runtimes, schedule, and policy. Replay paired records and arithmetic.

Sign the receipt

Record the technical result separately. A valid policy rejection is different from an integrity failure.

A structural or authority failure stops replay before the policy result.

The verifier owns replay

A producer can submit a correctly signed bundle that names the wrong artifact or policy. The signature proves which key authenticated the bytes; it does not tell the verifier what it should have expected.

Strict verification therefore gets its expectations from outside the bundle. A trust profile supplies the expected policy, artifacts, schedule, runtimes, and signer keys. The verification guide lists the exact fields.

Verification first checks the pack's inventory, signatures, and expected identities. It then checks record order and recomputes the paired scores, interval, policy arithmetic, and verdict. A structural or authority failure stops replay. A policy rejection after valid replay produces a signed rejection receipt.

Runtime-provider receipts record what the provider declared and what InvarLock bound into the transaction. They are not independent proof that an honest runtime executed. The Trust Model, Threat Model, and Assurance Case document that boundary.

The numbers remain paired all the way through

The request selects one comparison-policy path, and the verifier recomputes it from retained per-record material:

On one exact-match record, “baseline wrong, subject correct” is a subject-only improvement; the reverse is a baseline-only regression. Keeping those paired outcomes across the schedule lets the verifier measure the change instead of comparing two detached averages.

• Exact match asks whether both sides produced the expected answer for the same record. It records baseline-only regressions, subject-only improvements, the exact two-sided McNemar probability, and a paired Newcombe 95% interval. Policy reads the conservative lower bound.
• Normalized negative log-likelihood (NLL) per UTF-8 byte measures expected-continuation loss. It compares both arithmetic means across 2,048 deterministic paired resamples. Policy reads the interval's upper bound.
• Authorized deterministic text scorers may emit per-record values in [0,1], while core verification still owns aggregation, paired resampling, policy interpretation, and verdict calculation.

When tokenizer contracts and paired token counts are comparable, normalized-NLL evidence may include a verifier-derived token-weighted perplexity ratio to help a human interpret the result. That ratio has no independent policy authority. The exact arithmetic and legacy v1 replay rule live in Decision Semantics.

Runtime providers stay explicit

Verification and reporting install with the core package. Hugging Face text execution is the built-in provider, with its heavier dependencies kept optional:

python -m pip install "invarlock==0.13.0"
python -m pip install "invarlock[hf]==0.13.0"

Other first-party execution paths remain coordinated packages rather than silently expanding the core trust boundary.

Scroll horizontally to see every column.

The release retains complete journeys for evaluator import, model transformation, local and multimodal runtimes, TensorRT-LLM, and offline handoff. The boundary changes; the request → evidence → verification → report trail does not. The Runtime Providers and Change Scenarios guides name each implementation.

Diagnostics no longer vote

Spectral, random-matrix, and variance calculations now live in invarlock-diagnostics. They can still be useful. Their canonical JSON can travel inside signed evidence and appear in a human report. What they cannot do in v0.13 is determine whether the paired comparison passes policy.

We made that separation because a diagnostic becomes decision-bearing only after someone defines its semantics, calibrates a threshold, reviews the rule, and versions the resulting policy. Moving those calculations into an observation package prevents descriptive context from acquiring verdict authority simply because it was available during a run. See Optional Diagnostics.

What the public evidence qualifies

The v0.13 public index admits only invarlock/evidence-pack-v1 comparisons tied to a checked-in qualification suite. Each row binds one 400-record schedule, the model and runtime identities, the policy, both signers, and an independent receipt.

The initial entries exercise Hugging Face text, Hugging Face vision-text, and TensorRT-LLM paths under their recorded suite conditions. A missing large-sample comparison is labelled Evidence not yet created, not converted into a failure or an implied pass. The public evidence guide explains how to inspect those rows.

Moving from 0.12.1

The old catalog and calibration command families, built-in model editing and training operations, masked-LM and sequence-to-sequence execution, custom observability stack, and duplicate report formats are removed. Existing signed comparison-report v1 evidence remains replayable with its original interval method; new evaluations emit comparison-report v2.

What this release still does not decide

Version 0.13 makes one comparison replayable. Schedule coverage, organizational policy, runtime honesty, and deployment approval remain review questions around that signed technical result. The earlier release-gates paper note explains why a valid artifact is only one input to that wider decision.

The retained request, evidence, and receipt give another verifier an exact comparison to reconstruct and review.

Sources

Website documentation explains the maintained workflow. The changes described here belong to InvarLock v0.13.0; the tagged release record preserves that historical scope.

• Documentation
• Evaluation request
• Evidence and verification
• Trust model
• Threat model
• Assurance case
• Decision semantics
• Runtime providers
• Model-change workflows
• Optional diagnostics
• Public evidence guide
• InvarLock v0.13.0 release record
• Tagged v0.13.0 changelog