InvarLock 0.16.0: Native, Captured and Judge Evidence

A reviewer needs to know what produced a result, what can be checked again, and what still requires judgment.

A team may run a model locally, call a hosted service, or keep results from an existing evaluator. Each path can produce a useful comparison. The review problem is knowing which records must survive and what another person can verify from them.

InvarLock 0.16 brings these paths into the same evaluate → verify → report workflow. Native comparisons retain model and runtime bindings. Captured comparisons use records from an existing evaluation. Bounded judge measurements assess fixed answers against a declared rubric. They share commands, while keeping their verification requirements and limits explicit.

At a glance — v0.16 release snapshot

• Supported evaluator exports can supply paired records through the main commands.
• Exact match, measured likelihood and bounded judging each require their own per-case facts.
• Hosted-service checks distinguish fresh observations from replay of older evidence.
• Redesigned offline reports put the policy result beside the checks that explain it.
• Core, first-party add-ins and example files use matching release versions.

Start with the records you already have

Suppose an evaluation pipeline already retains the baseline and changed model's answer for each case. The captured-results workflow can compare those records without asking the pipeline to move to another evaluator. One request identifies both complete runs, the comparison policy and where to write the evidence.

An export containing only accuracy: 0.71 is not enough. A verifier needs the individual cases to reconstruct the scores and comparison. The required facts depend on the scorer:

• Exact match compares correctness. It needs answers and reference strings for the same cases on both sides.
• Normalized negative log-likelihood (NLL) compares loss on the reference text. It needs measured log probabilities, byte and token counts, and the identities of the inputs and models they belong to.
• Bounded judging compares ratings under a rubric. It needs task inputs, fixed answers, the rubric, a collection plan and retained judge calls.

Likelihood needs actual measurements. InvarLock cannot reconstruct it from generated answer text, token usage alone or a summary score. Captured replay checks the supplied records; it does not prove that the original model execution happened as claimed.

Judge fixed answers under a declared rubric

Judging starts with fixed baseline and candidate answers. The collection plan declares the rubric, judge configuration, sampling choices and call limits before ratings are collected. Keeping the answers fixed lets another reviewer replay the recorded analysis without generating different answers or making new provider calls.

The optional invarlock-inspect-judge package collects the ratings. Core handles offline import, analysis, verification and reporting. Repeated ratings do not create more independent test cases, and successful replay does not establish that the ratings are correct or that the judge is generally accurate. The judge reference describes those statistical and verification boundaries.

There are two distinctions to keep when choosing an example. The starter that compares already-recorded scalar ratings does not collect bounded judge measurements. And an evidence set can require both a deterministic comparison and a judge comparison over the same answers, but each keeps its own statistical meaning. Combining them does not create a joint confidence guarantee.

Recheck a hosted service with fresh observations

A hosted service may expose an endpoint name and revision without giving the caller access to immutable model weights. Captured evidence can record that service configuration, the evaluation harness and the observation window. The service's stated revision remains a claim from the source, rather than independent proof of which weights ran.

For a periodic check or an incident review, collect fresh observations, then compare and verify them. Replaying yesterday's pack checks yesterday's evidence. It does not measure the service again or establish continuous monitoring. The hosted-service guide follows the process from baseline approval through fresh capture and review.

Read the policy result beside the checks that explain it

The redesigned offline report leads with the policy result and its supporting checks. It shows the baseline and candidate identities, recorded differences, metric values, uncertainty and numerical requirements together. Reports with several metrics or evaluation scopes include an overview and navigation. Failed checks remain visible.

HTML reports support mobile layouts, dark mode and printing every metric. Console, Markdown and JUnit outputs serve other review and automation workflows; the report reference covers each format.

A valid signature can accompany a policy failure. A policy pass can still be insufficient for a recipient to accept the result. Rendering either outcome does not supply the recipient's independent trust inputs or approve deployment.

Try the release and update existing workflows

Install the lightweight core with Python 3.12 or newer:

python -m pip install "invarlock==0.16.0"

The tagged five-minute quickstart verifies retained evidence on a regular CPU, issues a fresh signed receipt and renders HTML. It does not run a model. To create a first captured comparison, use the synthetic extraction starter in the captured-results guide. Its invented cases demonstrate how the workflow fits together; they say nothing about a real model's quality.

Keep core, first-party add-ins and example files on the same release. For model execution, follow runtime setup: maintained runtime images and source-checkout dependency groups replace the retired public inference extras, including hf.

Captured workflows now use v2 requests through the main commands and the invarlock.engine Python interface. If migrating affected legacy captured evidence, recreate it from reviewed source records. The recipient must independently approve the new expected digests and signer identities—the values used to check that the supplied evidence is the intended comparison.

Native v1 evidence, native v1/v2 receipts and the v0.13 compatibility promise remain unchanged. Historical evidence retains its original bytes and meaning.

What this release still leaves to review

The shared workflow does not make all evidence interchangeable. A retained example demonstrates its named comparison and integration. It does not establish quality across other models or tasks. In particular, the bounded judge reference does not extend the qualification of models in the separate native evidence campaign.

Verification reconstructs a technical result under its declared rules. The recipient still decides whether that result, its coverage and its limits are sufficient for the proposed release.

Sources

Website documentation explains the maintained workflow. The changes described here belong to InvarLock v0.16.0; the tagged release record preserves that scope. Retained comparisons keep their original evidence identities and qualification limits.

• Captured results
• Judge measurements
• Evidence sets
• Hosted-service requalification
• Reports and receipts
• Runtime setup
• Compatibility covenant
• Release verification
• InvarLock v0.16.0 release record
• Tagged v0.16.0 changelog
• Five-minute signed-evidence check