InvarLock · Paired release-regression assurance

Run the evaluation. Verify the evidence. Control acceptance.

Run or import paired release-regression evidence, replay the technical verdict independently, then hand the exact result to the recipient's current policy.

Run the paired workflow Read the v0.14.0 release

Private deployment constraints? Review the design-partner scope

Install the core

Verification and reporting are in the core package. Add invarlock[hf] for the built-in Hugging Face text runtime.

$python -m pip install invarlock

One closed transaction

The evaluation operator publishes evidence. The technical verifier owns replay and signs the resulting technical receipt.

evidence-pack-v1

Evaluate

invarlock evaluate request.yaml

Verify

invarlock verify evidence/

Report

invarlock report evidence/

Core transaction: Evaluate · verify · report
Qualified imports: Per-record replay or observation-only
Current release: v0.14.0

Signed evidence

Verify the decision from lower-level facts

The verifier reconstructs record scores, paired statistics, uncertainty, policy checks, and the final verdict. An evaluation-operator summary cannot override that replay.

Canonical signed evidence: A closed inventory, SHA-256 checksums, and an Ed25519 manifest signature bind the request, schedule, provider records, runtime facts, and comparison report.
Independent replay: The technical verifier supplies its own policy and artifact, schedule, runtime, signer, and scorer anchors before recomputing the verdict.
Explicit boundary: A valid bundle identifies signed bytes and a scoped paired decision. It does not prove representative sampling, broad safety, honest execution, or global model quality.

Canonical comparison report

{
  "format": "invarlock/comparison-report-v2",
  "comparison_id": "comparison-5549b1518637b3b7d4a6b7338f157d3c",
  "metric": "normalized_nll_per_utf8_byte",
  "record_count": 400,
  "baseline": {
    "mean_score": 1.28119384765625
  },
  "subject": {
    "mean_score": 1.27853271484375
  },
  "comparison": {
    "kind": "normalized_nll_ratio",
    "value": 0.997922927262437,
    "maximum": 1.05
  },
  "uncertainty": {
    "method": "paired_percentile_bootstrap_sha256_v1",
    "interval_mass": 0.95,
    "lower": 0.9969656551480706,
    "upper": 0.9987858378380267,
    "replicates": 2048
  },
  "verdict": "pass"
}

Structure and values shown from the retained public v0.13.0 Mistral qualification example. Its original bundle and independent receipt—not this excerpt—carry the historical technical claim.

Evidence and verification

Follow publication, trust inputs, replay, receipts, and offline handoff.

Decision semantics

Inspect exact-match, normalized-NLL, scorer, interval, and policy arithmetic.

Trust model

Separate evidence, receipt, envelope, recipient, and rendering authority.

Recipient acceptance

Bind one technical receipt to one artifact and apply current recipient policy.

Workflow

One evidence transaction, three explicit commands

Execution, independent technical verification, and presentation stay separate without producing competing summaries.

Read the request contract

Command 01

Close the request, then evaluate both sides

The request authenticates baseline and subject artifacts, dataset bytes, one paired schedule, providers, runtimes, metric or scorer, policy, execution mode, and a fresh evidence destination.

invarlock evaluate request.yaml

Command 02

Replay evidence with independent anchors

Verification treats the bundle as untrusted and re-derives record scores, paired statistics, the interval, policy checks, and verdict against verifier-owned trust inputs.

invarlock verify evidence/ --trust-profile trust.json \
  --receipt verification.receipt.json

Command 03

Render the authenticated comparison

Reporting authenticates the bundle and renders its canonical comparison as console text or standalone HTML. The signed verification receipt remains the authoritative technical record.

invarlock report evidence/ --html evidence.html

invarlock demo

The verifier owns the decision

Aggregation: Recompute means and paired record outcomes from authenticated observations.
Uncertainty: Replay the metric-specific paired interval over the fixed schedule.
Policy: Use an independently supplied policy and conservative interval bound.
Verdict: Sign a receipt that binds the exact evidence and trust anchors used.

Scope

One technical verdict across metrics, runtimes, and evaluator imports

Runtime packages can change how authenticated records are produced. They do not get to redefine pairing, aggregation, uncertainty, policy, or the verifier's verdict.

Comparison paths

Exact match: Regression and improvement counts, exact two-sided McNemar probability, and a paired Newcombe 95% effect-size interval.
Normalized NLL: Expected-continuation negative log-likelihood per UTF-8 byte with a deterministic paired schedule-resampling interval.
Deterministic scorer: An explicitly authorized extension can emit per-record scores; core verification still owns aggregation, interval, policy, and verdict.

Perplexity ratios appear only when tokenizer contracts and paired token counts are comparable. They are derived interpretation, not a separate verdict surface.

First-party runtime paths

Hugging Face text: invarlock[hf]; Built-in provider; optional execution dependencies.
GGUF / llama.cpp: invarlock-runtime-gguf; Optional coordinated runtime distribution.
Hugging Face vision-text: invarlock-runtime-hf-vision-text; Optional coordinated runtime distribution.
TensorRT-LLM: invarlock-runtime-tensorrt-llm; Optional coordinated runtime distribution.

LM Evaluation Harness, PEFT LoRA, TorchAO INT8, and offline handoff are maintained integration journeys. Spectral, RMT, and variance diagnostics live in invarlock-diagnostics and remain observation-only.

Evaluator qualification

Normalize outside core. Recompute inside the boundary.

Open-source and proprietary evaluators run outside InvarLock. Their adapters bind a source-pinned profile, independent schedule, normalized export, retained output, runner bundle, and dependency declaration to the same evaluator-neutral contracts.

invarlock-qualify-evaluator qualify \
  profile.json schedule.json export.json upstream-output.json

Read the evaluator qualification boundary

Qualification profile: A maintained row demonstrates one pinned version and entry point. It is not a built-in plugin or blanket support claim.
Verdict-authoritative import: Complete ordered records with bound inputs and outputs, successful status, and deterministic exact-match agreement can cross into runtime-import replay.
Observation-only evidence: Aggregate-only, human-judged, nondeterministic, or unsupported replay evidence keeps its reason and cannot be promoted into verdict authority.

Recipient handoff

Transport the result. Let the recipient decide.

InvarLock v0.14 adds a portable acceptance envelope after technical verification. It carries a bounded result into current recipient policy without changing the evidence transaction or pretending to grant final approval.

Keep the signed technical receipt authoritative

The independently verified receipt remains the replayable result. Full evidence replay stays with invarlock verify.

Wrap one exact artifact

An optional in-toto Statement in a DSSE envelope carries the receipt and binds the exact derived artifact by digest.

Authenticate two trust relationships

The recipient checks the envelope signer and embedded receipt verifier against separate recipient-controlled registries.

Apply current recipient policy

Current contract, signer status, artifact, freshness, and verdict rules decide portable acceptance—not organizational approval.

Exact subject: Supply either the expected digest or the artifact path to hash. Supplying neither or both fails closed.
Separate freshness: Envelope age and receipt-authenticated evidence age use different clocks. A fresh envelope never renews stale evidence.
Policy interoperability: OPA/Rego and CUE consume an authenticated projection. They do not perform raw signature checks or full evidence replay in the maintained example.

Acceptance attestations · OPA/Rego and CUE interoperability · v0.13 compatibility covenant

Next step

Start with the open evidence transaction

The v0.14.0 engine, evaluator-neutral qualification boundary, recipient handoff contracts, runtimes, and examples are public now. Teams with private deployment constraints can scope one real release-review workflow with us.

Open the getting-started guide View on GitHub

Need the review model first? Read the assurance case · Inspect public evidence · Review recipient handoff · Review design-partner scope