InvarLock 0.15.0: Replayable Deployment Evidence and Bounded Decisions

InvarLock 0.15 adds replayable deployment evidence, an absolute accuracy floor, four retained evaluator journeys, and a protected consumer example.

August 16, 2026Updated September 6, 20268 min readInvarLock Team

A comparison can pass exactly as designed and still be insufficient to authorize deployment.

Version 0.15 ships comparison-report v3, three full-precision-to-five-bit GGUF deployment comparisons, four retained evaluator journeys, and a standalone protected consumer. GGUF is a format for local inference; Q5_K_M names the five-bit quantization recipe used here.

The release retains seven public evidence packs. Four model-running evaluator transactions exercise two evaluator paths. A separate protected consumer verifies and binds a result before its own deployment approval. These are different demonstrations, not interchangeable counts.

At a glance — v0.15 release snapshot

New evaluations emit comparison-report v3.

Exact match can require a minimum accuracy on both sides.

Three BF16-to-Q5_K_M GGUF journeys expand the public inventory to seven packs.

LM Evaluation Harness and Inspect AI retain four signed transactions.

A five-minute CPU path verifies, receipts, and renders evidence without rerunning a model.

What to noticeThe three inventories demonstrate different parts of the workflow. Their counts do not combine into an approval score.

Version 0.15 retains seven public packs, four evaluator transactions, and one protected consumer while keeping integrity, policy, recipient acceptance, and deployment approval separate. — What to noticeThe three inventories demonstrate different parts of the workflow. Their counts do not combine into an approval score.

Replay retained evidence in five minutes

From an empty directory, the tagged quickstart installs the core wheel and obtains the separate example files. Use Python 3.12 or newer:

python -m venv .venv
. .venv/bin/activate
python -m pip install "invarlock==0.15.0"

curl -fsSLO \
  https://github.com/invarlock/invarlock/archive/refs/tags/v0.15.0.tar.gz
tar -xzf v0.15.0.tar.gz --strip-components=3 \
  invarlock-0.15.0/examples/quickstart \
  invarlock-0.15.0/examples/acceptance-handoff/golden

python run.py --fixture golden

The example prints Decision: pass and writes verification.result.json, verification.receipt.json, and evidence.html inside invarlock-quickstart-output/. It verifies the retained pack against independently supplied anchors before issuing the receipt and rendering HTML. It needs no model execution, container engine, network service, or source-tree import.

Model execution produces the evidence. Verification authenticates those retained records and recomputes the decision. The example files come from the tagged source archive rather than the installed wheel, so the package does not carry its own expected test evidence.

Report v3 can reject a pair below the accuracy floor

A relative non-regression policy has an edge case. A baseline at 18% accuracy and a subject at 19% may satisfy a relative threshold even though both miss a 20% minimum.

Comparison-report v3 adds a bounded response for exact match: minimum_side_accuracy. Policy may set a finite floor from 0 through 1, and both baseline and subject means must meet it. The boundary is inclusive.

This check remains independent of the regression threshold, minimum record count, and maximum interval width. Every enabled control must pass.

Data table with columns: Control, Question it answers, Failure means
Control	Question it answers	Failure means
Regression threshold	Did the subject degrade beyond the allowed paired bound?	The comparison-specific change is too large.
Minimum record count	Is enough paired material retained for this policy?	The evidence set is too small.
Maximum interval width	Is the paired estimate sufficiently precise?	The result is too uncertain.
Minimum side accuracy	Did both baseline and subject clear the declared absolute floor?	At least one side is too weak even if their relative difference is acceptable.

Scroll horizontally to see every column.

The control is intentionally metric-specific. It applies to exact match, not normalized NLL or arbitrary observation content.

Historical evidence keeps its original identity and arithmetic. Version 2 reports remain verifiable without side-accuracy qualification. Version 1 reports continue to use their original exact-match interval method. Verification does not silently reinterpret either format as v3.

Three GGUF journeys join the public inventory

The public evidence index now contains seven strictly verified packs. Three new rows compare an authenticated BF16 checkpoint with its source-derived Q5_K_M GGUF deployment over a balanced 400-record MMLU-Pro exact-match schedule:

Deployment comparison: Qwen3.8 27B
BF16: 63.00%
Q5_K_M: 64.50%
Paired 95% interval: [−0.74, 3.75] pp
Policy: Pass

Deployment comparison: Qwen3.5 9B
BF16: 53.00%
Q5_K_M: 54.75%
Paired 95% interval: [−0.83, 4.32] pp
Policy: Pass

Deployment comparison: Ministral 3 8B
BF16: 44.75%
Q5_K_M: 45.25%
Paired 95% interval: [−1.12, 2.12] pp
Policy: Pass

Data table with columns: Deployment comparison, BF16, Q5_K_M, Paired 95% interval, Policy
Deployment comparison	BF16	Q5_K_M	Paired 95% interval	Policy
Qwen3.8 27B	63.00%	64.50%	[−0.74, 3.75] pp	Pass
Qwen3.5 9B	53.00%	54.75%	[−0.83, 4.32] pp	Pass
Ministral 3 8B	44.75%	45.25%	[−1.12, 2.12] pp	Pass

What to noticeThe three lower endpoints sit to the right of the dashed −2 pp policy line; crossing zero is allowed by this non-regression policy.

All three GGUF paired intervals cross zero but remain above the authenticated −2 pp lower-bound threshold. — What to noticeThe three lower endpoints sit to the right of the dashed −2 pp policy line; crossing zero is allowed by this non-regression policy.

Each journey binds separate runtime identities, conversion provenance, an independently signed receipt, and the declared record-count, interval-width, side-accuracy, and regression controls.

For each named pair, the Q5_K_M subject satisfies the authenticated finite-schedule policy relative to its BF16 baseline. The result is not a claim of bit-for-bit equivalence or universal quality preservation. Planned comparisons remain labelled Evidence not yet created until a qualified pack exists.

Four evaluator journeys show both pass and rejection

The v0.15 evaluator matrix records three independent properties: adapter support, replay authority, and signed-journey maturity.

It contains 19 maintained profiles. Seventeen retain deterministic per-record exact-match replay. MLflow's aggregate cannot reconstruct the records; Garak's retained replay semantics are outside the v0.15 contract. Both remain observation-only.

The shared qualification corpus now uses pinned Qwen3.5 0.8B output with 102 records: 61 matches and 41 mismatches. That corpus demonstrates the adapter and replay boundary for one exact-match surface. It is not a general evaluator benchmark.

LM Evaluation Harness and Inspect AI retain four larger model-running transactions:

Transaction: Qwen3.5 9B · LM Evaluation Harness
Baseline: 55.50%
Subject: 53.00%
Paired 95% interval: [−6.42, 1.43] pp
Signed outcome: Integrity valid; policy rejected

Transaction: Qwen3.5 9B · Inspect AI
Baseline: 55.50%
Subject: 53.00%
Paired 95% interval: [−6.42, 1.43] pp
Signed outcome: Integrity valid; policy rejected

Transaction: Gemma 4 12B instruction-to-QAT · LM Evaluation Harness
Baseline: 44.00%
Subject: 42.50%
Paired 95% interval: [−3.84, 0.84] pp
Signed outcome: Integrity valid; policy rejected

Transaction: Qwen3.5 0.8B deployment · Inspect AI
Baseline: 49.25%
Subject: 43.50%
Paired 95% interval: [−8.99, −2.47] pp
Signed outcome: Policy passed

Data table with columns: Transaction, Baseline, Subject, Paired 95% interval, Signed outcome
Transaction	Baseline	Subject	Paired 95% interval	Signed outcome
Qwen3.5 9B · LM Evaluation Harness	55.50%	53.00%	[−6.42, 1.43] pp	Integrity valid; policy rejected
Qwen3.5 9B · Inspect AI	55.50%	53.00%	[−6.42, 1.43] pp	Integrity valid; policy rejected
Gemma 4 12B instruction-to-QAT · LM Evaluation Harness	44.00%	42.50%	[−3.84, 0.84] pp	Integrity valid; policy rejected
Qwen3.5 0.8B deployment · Inspect AI	49.25%	43.50%	[−8.99, −2.47] pp	Policy passed

The three MMLU-Pro transactions authenticate correctly and then miss their −2 pp lower-bound requirement. Their receipts record valid evidence and valid policy rejection.

The deployment transaction uses a separate pinned 400-record LAMBADA corpus and passes its declared −20 pp lower-bound policy. The subject remained inside that named deployment-regression tolerance; improvement was not required.

For the Qwen3.5 comparison, LM Evaluation Harness and Inspect AI agree across all 400 baseline and all 400 subject record digests and scores. That is cross-evaluator replay-consistency evidence. It does not create an extra verdict.

The protected consumer binds the exact subject

A valid receipt can still be consumed badly. A deployment workflow might check that “some comparison passed” while fetching a different artifact, using a different trust profile, or skipping the recipient's current approval controls.

The standalone CI consumer models a separate repository with release-review and production environments. Release review replays the evidence against independently supplied anchors and requires an explicit policy pass. Production then:

reauthenticates the receipt;
binds the fetched candidate to the exact approved subject identity; and
checks the trust-profile digest or its explicit absence from protected approval inputs.

The example demonstrates how a recipient can consume a technical result without letting that result act as a deployment command. Environment protection, artifact fetching, recipient policy, and organizational approval remain consumer-owned.

Software-supply-chain metadata remains context

The signed package declaration must name the same artifact as the comparison. The CPU-only example checks that binding with an SPDX 3.0.1 declaration, a standard format for software-supply-chain metadata.

The declaration remains authenticated context rather than policy input. The example checks a documented subset, not full SPDX conformance or deployment suitability. The retained integration record carries the JSON-LD, digest, schema status, and internal envelope details.

This is the observation boundary in practice: authenticate the context, preserve it, render it, and keep it outside the verdict unless a future versioned contract explicitly grants it authority.

Release assurance expands around the same claim

Candidate-wheel preflight verifies and renders all seven public packs and all four evaluator transactions without rerunning inference, evaluators, or conversion.

Runtime hardening: bounds commands and evidence loading, rejects symlinked worker inputs, and gives temporary engines immutable identities.
Release hardening: tests a core-only installation and protected consumer, then checks request binding, trust profiles, approvals, and 95% coverage.
Scoped compatibility: adds text-only checkpoint loading and one TensorRT-LLM 1.2.1 Qwen3 fixture without claiming broader image-input or runtime support.

What the release still leaves to people

A policy pass establishes only the declared finite-schedule comparison predicate.
Public evidence packs and evaluator transactions are separate retained surfaces.
Observation-only content cannot affect paired statistics or acceptance.
A verification receipt does not replace recipient review, deployment policy, or organizational approval.
Retained examples demonstrate exact versions and inputs, not open-ended support for every future environment.

For migration details and exact supported contracts, use the tagged release record below. Recipient review still determines whether the authenticated comparison is sufficient for the intended deployment.

Sources

Website documentation explains the maintained workflow. The changes described here belong to InvarLock v0.15.0; the tagged release record preserves that historical scope.