InvarLock 0.14.0: Evaluator Evidence In, Recipient Policy Out
v0.14 normalizes external evaluator records, then hands a signed technical receipt to recipient-owned trust and acceptance rules.
The evaluator may produce the records. It does not get to decide what the recipient trusts.
Teams rarely use one evaluation tool forever. A research group may run LM Evaluation Harness, a security team may use Garak, and a production group may depend on an internal evaluator that cannot be published. The hard part is not adding another adapter. The hard part is deciding what evidence must survive so that every result can be checked under the same authority model.
InvarLock 0.14 extends the v0.13 transaction at both ends. Evaluator output first crosses a common evidence boundary: the minimum records and identities every adapter must provide. After verification, the technical result enters recipient acceptance, where the receiving organization applies its own trust and policy.
At a glance — v0.14 release snapshot
- 19 evaluator profiles cross the common adapter boundary.
- 17 profiles retain replayable records for one 102-record evaluation.
- Two profiles remain observation-only.
- A Dead Simple Signing Envelope (DSSE) carries the exact technical receipt into recipient-owned policy.
One evidence boundary for many evaluators
An evaluator name is not an evidence contract. It identifies the source, but it does not show whether the result contains the complete ordered records needed for independent recomputation.
Consider an export containing a model name and accuracy: 0.71. The number may be correct and signed. Without its records and schedule, however, a verifier cannot confirm that every sample appears once, both sides use the same order, or the governing contract permits that record shape.
Version 0.14 defines four closed JSON, CLI, and Python contracts at the evaluator-neutral qualification boundary. Maintained runners execute pinned upstream entry points, then normalize their output into those common contracts.
At release time, the v0.14 matrix described three progressively deeper demonstrations:
| v0.14 evidence level | The question it answered | Retained coverage |
|---|---|---|
| Qualification profile | Did a pinned upstream entry point execute and cross the generic contract? | 19 profiles |
| Authoritative import | Were complete per-record results retained and independently recomputed? | 17 profiles over one 102-record evaluation |
| End-to-end transaction | Did a maintained model-running journey cross evaluate → verify → report? | LM Evaluation Harness |
Scroll horizontally to see every column.
The 17 authoritative rows replay one pinned Qwen3-0.6B exact-match evaluation: 102 ordered records, 52 matches, and 50 mismatches. That retained run demonstrates exact-match import compatibility for those profiles. It is not a general ranking of evaluators.
The matrix also retains two outputs as context. MLflow is aggregate-only, so another verifier cannot reconstruct its records. Garak uses replay semantics that the v0.14 contract does not support. Both remain authenticated observations outside the paired verdict.
Historical note: Version 0.14 described evaluator evidence as cumulative “levels.” Version 0.15 replaces that framing with three independent axes: adapter support, replay authority, and signed-journey maturity. See An Evaluator Can Be Supported Without Being Decision-Ready for the newer model.
Why replay authority follows records
The full replay-authority test asks four questions:
- Were all expected samples present exactly once?
- Did the evaluator score the baseline and subject in the same order?
- Can the metric be recomputed without trusting the aggregate?
- Does the governing contract allow that record shape to determine policy?
That is why v0.14 does not equate “adapter maintained” with “decision-ready.” Authority comes from the retained evidence and its contract, not from the popularity or commercial status of the evaluator.
A verified result is still not recipient acceptance
Version 0.13 gave the technical verifier independent replay authority. Version 0.14 adds a separate question: is that authenticated result acceptable now for the exact artifact this recipient is considering?
For example, a receipt can verify artifact A while a deployment job fetches artifact B. The historical comparison remains valid, but the recipient must reject the handoff because the supplied subject no longer matches the receipt.
The distinction matters because technical verification and recipient acceptance operate on different clocks and trust stores. A result can remain historically valid while a current recipient rejects it because the evidence is stale, a signer is no longer trusted, the contract is unsupported, or the supplied artifact does not match the approved subject.
The portable handoff works in five steps:
- The evaluation operator identifies the exact baseline and subject artifacts.
- Per-record results are recomputed and published in a signed evidence pack.
- An independent technical verifier replays the pack and signs a receipt.
- An envelope signer places the exact receipt bytes in an in-toto Statement—an explicit software-supply-chain claim—then carries that statement in a Dead Simple Signing Envelope (DSSE).
- The recipient authenticates the envelope, binds it to the artifact bytes, and applies current policy.
From signed receipt to recipient policy
- Authenticate
- Check the outer envelope signature and the inner technical receipt against separate trust registries.
- Bind the subject
- Match the supplied artifact to the exact subject named in the authenticated receipt.
- Apply current policy
- OPA/Rego or CUE receives authenticated fields. Evidence age and envelope age remain separate.
Full evidence replay remains the job of invarlock verify. Acceptance is not deployment approval.
The outer envelope signer authenticates transport and statement bytes. The inner receipt verifier authenticates the technical result and evidence identity. Trust in one never substitutes for trust in the other.
Freshness remains separate too. Envelope age comes from the attestation timestamp. Evidence age comes from the timestamp authenticated by the technical receipt. Re-signing an old result for transport does not make the evidence young again.
Policy engines receive an authenticated projection
The retained fixtures reject a handoff when its artifact, signer, freshness, contract, or technical verdict is wrong. They also retain one accepted handoff, so reviewers can see each boundary close for a specific reason.
The release includes these policy-engine fixtures for Open Policy Agent's Rego language and for CUE configuration rules. Authentication comes first. Policy evaluation comes second.
A local verifier checks the signed handoff and exact subject. It then emits an authenticated projection: a small verified set of fields that Rego or CUE can evaluate safely.
The receipt-envelope reference shows the nested statement and exact receipt fields. The process above keeps transport authentication, technical verification, and recipient policy separate.
| Layer | Responsibility |
|---|---|
| Local verifier | Canonicalization, Ed25519 signatures, receipt authentication, subject binding, and signed projection agreement |
| OPA/Rego or CUE | Current recipient rules over the authenticated projection |
invarlock verify | Full semantic replay of the complete evidence pack |
The policy engine never receives unverified claims directly. It receives the verifier's projection of the authenticated result.
Historical verification, current acceptance
Version 0.14 makes a long-term compatibility promise explicit. Existing v0.13 evidence and receipts remain verifiable under their original parsing, binding, replay, and decision semantics. Their format identifiers and exact receipt bytes are preserved instead of being relabelled as v0.14 evidence.
Recipient acceptance is intentionally different. It is a current decision. A recipient can apply today's signer registry, freshness window, supported-contract set, and exact-artifact policy to an older technical result without altering what that historical result meant.
This gives the two layers a useful independence:
- Verification asks: what did the authenticated evidence establish under its original contract?
- Acceptance asks: does our organization accept that exact result for this exact subject now?
The compatibility covenant defines the permanent verification side of that relationship.
What changed from 0.13
| v0.13 foundation | v0.14 extension |
|---|---|
| One closed request and canonical evidence pack | A common qualification boundary for external evaluator records |
| Independent verifier replay | Explicit replay-authority requirements for imported per-record evidence |
| Separate signed verification receipt | Portable in-toto/DSSE transport of the exact receipt bytes |
| Technical policy result | Recipient-controlled trust, freshness, contract, and artifact acceptance |
| Stable v0.13 semantics | A permanent verification promise plus current acceptance evaluation |
Version 0.13 remains the technical foundation. Version 0.14 does not replace its decision. It makes the input boundary evaluator-neutral and the output boundary portable.
What this release does not collapse
- The 17 replayable profiles cover exact match over one retained 102-record Qwen3-0.6B evaluation, not every task, model, judge, or evaluator feature.
- At v0.14, LM Evaluation Harness is the sole retained model-running end-to-end evaluator journey.
- Aggregate-only and unsupported-replay outputs remain observations, even when authenticated.
- Recipient acceptance covers the signed technical handoff. Deployment policy, organizational approval, and operational risk remain outside that predicate.
For the exact historical contract and retained accepted/rejected handoffs, use the release record below.
Sources
Website documentation explains the maintained workflow. The changes described here belong to InvarLock v0.14.0; the tagged release record preserves that historical scope.
More in Release
Explore nearby related posts.
Release note
InvarLock 0.15.0: Replayable Deployment Evidence and Bounded Decisions
InvarLock 0.15 adds replayable deployment evidence, an absolute accuracy floor, four retained evaluator journeys, and a protected consumer example.
Release note
InvarLock 0.13.0: One Request, One Signed Decision Trail
One YAML request now binds artifacts, schedule, metric, and policy. One signed evidence pack carries that comparison through independent verification and reporting.
Release note
InvarLock 0.16.0: Native, Captured and Judge Evidence
InvarLock 0.16 brings captured evaluator results and bounded judge measurements into the shared review workflow, with clearer reports and explicit verification limits.