InvarLock 0.15.0: Replayable Deployment Evidence and Bounded Decisions

InvarLock 0.15 adds replayable deployment evidence, an absolute accuracy floor, four retained evaluator journeys, and a protected consumer example.

Published August 16, 2026Updated October 20, 20268 min readInvarLock Team

A candidate can stay close to a weak baseline and still perform poorly. InvarLock 0.15 lets an exact-match comparison require a minimum accuracy on both sides, alongside its rule about allowed regression.

The release also adds saved deployment comparisons to inspect and a five-minute CPU example that checks evidence without running a model. Three new cases compare full-precision checkpoints with five-bit Q5_K_M deployments in the GGUF local-inference format. Four signed evaluator transactions exercise two evaluator paths; a separate deployment example checks that the supplied candidate matches the receipt.

At a glance — v0.15 release snapshot

New evaluations emit comparison-report v3.

Exact match can require a minimum accuracy on both sides.

Three BF16-to-Q5_K_M GGUF journeys expand the public inventory to seven packs.

LM Evaluation Harness and Inspect AI retain four signed transactions.

A five-minute CPU path verifies, receipts, and renders evidence without rerunning a model.

What to noticeThe three inventories demonstrate different parts of the workflow. Their counts do not combine into an approval score.

Version 0.15 retains seven public packs, four evaluator transactions, and one protected consumer while keeping integrity, policy, recipient acceptance, and deployment approval separate. — What to noticeThe three inventories demonstrate different parts of the workflow. Their counts do not combine into an approval score.

Replay retained evidence in five minutes

From an empty directory, the tagged quickstart installs the core wheel and obtains the separate example files. Use Python 3.12 or newer:

python -m venv .venv
. .venv/bin/activate
python -m pip install "invarlock==0.15.0"

curl -fsSLO \
  https://github.com/invarlock/invarlock/archive/refs/tags/v0.15.0.tar.gz
tar -xzf v0.15.0.tar.gz --strip-components=3 \
  invarlock-0.15.0/examples/quickstart \
  invarlock-0.15.0/examples/acceptance-handoff/golden

python run.py --fixture golden

The example prints Decision: pass and writes verification.result.json, verification.receipt.json, and evidence.html inside invarlock-quickstart-output/. It verifies the retained pack against independently supplied anchors before issuing the receipt and rendering HTML. It needs no model execution, container engine, network service, or source-tree import.

Model execution produces the evidence. Verification authenticates those retained records and recomputes the decision. The example files come from the tagged source archive rather than the installed wheel, so the package does not carry its own expected test evidence.

Report v3 can reject a pair below the accuracy floor

A relative non-regression policy has an edge case. A baseline at 18% accuracy and a subject at 19% may satisfy a relative threshold even though both miss a 20% minimum.

Comparison-report v3 adds a bounded response for exact match: minimum_side_accuracy. Policy may set a finite floor from 0 through 1, and both baseline and subject means must meet it. The boundary is inclusive.

The accuracy floor is a separate check from the allowed regression, number of records, and uncertainty limit. Every enabled requirement must pass.

Table 1: Report v3 can reject a pair below the accuracy floor. Data table with columns: Control, Question it answers, Failure means
Control	Question it answers	Failure means
Regression threshold	Does the paired interval rule out a loss beyond the allowed bound?	The evidence does not establish that the loss stays within the allowed bound.
Minimum record count	Is enough paired material retained for this policy?	The evidence set is too small.
Maximum interval width	Is the paired estimate sufficiently precise?	The result is too uncertain.
Minimum side accuracy	Did both baseline and subject clear the declared absolute floor?	At least one side is too weak even if their relative difference is acceptable.

Scroll horizontally to see every column.

The control is intentionally metric-specific. It applies to exact match, not normalized NLL or arbitrary observation content.

Historical evidence keeps its original identity and arithmetic. Version 2 reports remain verifiable without side-accuracy qualification. Version 1 reports continue to use their original exact-match interval method. Verification does not silently reinterpret either format as v3.

Three GGUF journeys join the public inventory

The public evidence index now contains seven strictly verified packs. Three new rows compare an authenticated BF16 checkpoint with its source-derived Q5_K_M GGUF deployment over a balanced 400-record MMLU-Pro exact-match schedule:

Deployment comparison
Qwen3.8 27B
BF16
63.00%
Q5_K_M
64.50%
Paired 95% interval
[−0.74, 3.75] pp
Policy
Pass
Deployment comparison
Qwen3.5 9B
BF16
53.00%
Q5_K_M
54.75%
Paired 95% interval
[−0.83, 4.32] pp
Policy
Pass
Deployment comparison
Ministral 3 8B
BF16
44.75%
Q5_K_M
45.25%
Paired 95% interval
[−1.12, 2.12] pp
Policy
Pass

Table 2: Three GGUF journeys join the public inventory. Data table with columns: Deployment comparison, BF16, Q5_K_M, Paired 95% interval, Policy
Deployment comparison	BF16	Q5_K_M	Paired 95% interval	Policy
Qwen3.8 27B	63.00%	64.50%	[−0.74, 3.75] pp	Pass
Qwen3.5 9B	53.00%	54.75%	[−0.83, 4.32] pp	Pass
Ministral 3 8B	44.75%	45.25%	[−1.12, 2.12] pp	Pass

What to noticeThe three lower endpoints sit to the right of the dashed −2 pp policy line; crossing zero is allowed by this non-regression policy.

All three GGUF paired intervals cross zero but remain above the authenticated −2 pp lower-bound threshold. — What to noticeThe three lower endpoints sit to the right of the dashed −2 pp policy line; crossing zero is allowed by this non-regression policy.

Each journey binds separate runtime identities, conversion provenance, an independently signed receipt, and the declared record-count, interval-width, side-accuracy, and regression controls.

For each named pair, the Q5_K_M version meets the signed policy on the recorded cases relative to its BF16 baseline. That does not establish bit-for-bit equivalence or preserved quality on every task. Planned comparisons remain labelled Evidence not yet created until a qualified pack exists.

Four evaluator journeys show both pass and rejection

The v0.15 evaluator matrix records three independent properties: adapter support, replay authority, and signed-journey maturity.

It contains 19 maintained profiles. Seventeen retain deterministic per-record exact-match replay. MLflow's aggregate cannot reconstruct the records; Garak's retained replay semantics are outside the v0.15 contract. Both remain observation-only.

The shared qualification corpus now uses pinned Qwen3.5 0.8B output with 102 records: 61 matches and 41 mismatches. That corpus demonstrates the adapter and replay boundary for one exact-match surface. It is not a general evaluator benchmark.

LM Evaluation Harness and Inspect AI retain four larger model-running transactions:

Transaction
Qwen3.5 9B · LM Evaluation Harness
Baseline
55.50%
Subject
53.00%
Paired 95% interval
[−6.42, 1.43] pp
Signed outcome
Integrity valid; policy rejected
Transaction
Qwen3.5 9B · Inspect AI
Baseline
55.50%
Subject
53.00%
Paired 95% interval
[−6.42, 1.43] pp
Signed outcome
Integrity valid; policy rejected
Transaction
Gemma 4 12B instruction-to-QAT · LM Evaluation Harness
Baseline
44.00%
Subject
42.50%
Paired 95% interval
[−3.84, 0.84] pp
Signed outcome
Integrity valid; policy rejected
Transaction
Qwen3.5 0.8B deployment · Inspect AI
Baseline
49.25%
Subject
43.50%
Paired 95% interval
[−8.99, −2.47] pp
Signed outcome
Policy passed

Table 3: Four evaluator journeys show both pass and rejection. Data table with columns: Transaction, Baseline, Subject, Paired 95% interval, Signed outcome
Transaction	Baseline	Subject	Paired 95% interval	Signed outcome
Qwen3.5 9B · LM Evaluation Harness	55.50%	53.00%	[−6.42, 1.43] pp	Integrity valid; policy rejected
Qwen3.5 9B · Inspect AI	55.50%	53.00%	[−6.42, 1.43] pp	Integrity valid; policy rejected
Gemma 4 12B instruction-to-QAT · LM Evaluation Harness	44.00%	42.50%	[−3.84, 0.84] pp	Integrity valid; policy rejected
Qwen3.5 0.8B deployment · Inspect AI	49.25%	43.50%	[−8.99, −2.47] pp	Policy passed

The three MMLU-Pro transactions authenticate correctly and then miss their −2 pp lower-bound requirement. Their receipts record valid evidence and valid policy rejection.

The deployment transaction uses a separate pinned 400-record LAMBADA corpus and passes its declared −20 pp lower-bound policy. The subject remained inside that named deployment-regression tolerance; improvement was not required.

For the Qwen3.5 comparison, LM Evaluation Harness and Inspect AI agree across all 400 baseline and all 400 subject record digests and scores. That is cross-evaluator replay-consistency evidence. It does not create an extra verdict.

The protected consumer binds the exact subject

A valid receipt can still be consumed badly. A deployment workflow might check that “some comparison passed” while fetching a different artifact, using a different trust profile, or skipping the recipient's current approval controls.

The standalone CI consumer models a separate repository with release-review and production environments. Release review replays the evidence against independently supplied anchors and requires an explicit policy pass. Production then:

reauthenticates the receipt;
binds the fetched candidate to the exact approved subject identity; and
checks the trust-profile digest or its explicit absence from protected approval inputs.

The example demonstrates how a recipient can consume a technical result without letting that result act as a deployment command. Environment protection, artifact fetching, recipient policy, and organizational approval remain consumer-owned.

Software-supply-chain metadata remains context

The signed package declaration must name the same artifact as the comparison. The CPU-only example checks that binding with an SPDX 3.0.1 declaration, a standard format for software-supply-chain metadata.

The declaration remains authenticated context rather than policy input. The example checks a documented subset, not full SPDX conformance or deployment suitability. The retained integration record carries the JSON-LD, digest, schema status, and internal envelope details.

This is the observation boundary in practice: authenticate the context, preserve it, render it, and keep it outside the verdict unless a future versioned contract explicitly grants it authority.

Release assurance expands around the same claim

Candidate-wheel preflight verifies and renders all seven public packs and all four evaluator transactions without rerunning inference, evaluators, or conversion.

Runtime hardening: bounds commands and evidence loading, rejects symlinked worker inputs, and gives temporary engines immutable identities.
Release hardening: tests a core-only installation and protected consumer, then checks request binding, trust profiles, approvals, and 95% coverage.
Scoped compatibility: adds text-only checkpoint loading and one TensorRT-LLM 1.2.1 Qwen3 fixture without claiming broader image-input or runtime support.

Put the new controls to work

Try the CPU replay first, then set the accuracy, regression and precision requirements that fit your comparison before collecting its results. Use the exact-candidate consumer example when handing the result to a deployment workflow. The retained examples demonstrate their named versions and inputs; your organization still owns the requirements and release decision. Tagged sources below preserve the release-specific setup.

Sources

Website documentation explains the maintained workflow. The changes described here belong to InvarLock v0.15.0; the tagged release record preserves that historical scope.