A Signed Score Is Not Yet Decision Evidence

Integrity tells you which bytes were authenticated. Authority tells you whether those bytes may determine the verdict.

InvarLock can authenticate the aggregate retained by its MLflow integration even when that aggregate lacks the ordered records needed to recompute the score. Authenticating the export does not make it usable for paired policy.

A signature proves that someone authenticated a particular set of bytes. It does not prove that another verifier can reconstruct the score or that the governing comparison contract lets it affect policy.

In InvarLock, that difference is the line between decision evidence and an authenticated observation. Decision evidence is replayable material that an authorized contract may use. An authenticated observation is preserved context that cannot vote on the verdict.

The evidence model examined here applies this replay-authority boundary to evaluator imports and attached diagnostic context.

The fork is independent replay

For the retained exact-match qualification surface, replay authority requires complete ordered records. Each record is bound to the schedule, input, output, reference, and status. InvarLock recomputes the score instead of trusting the evaluator's aggregate claim.

Diagram

Authenticated evaluator output enters paired policy only when complete ordered records can be independently replayed under an authorized contract; other output remains observation-only context.

Which output may affect policy?

Authenticate the output

Check the signature or independently pinned digest and the bound identities.

Complete, authorized records

Independently recompute the score and interval. The declared contract may then use the result in paired policy.

Otherwise: retain context

Preserve the observation for review. Missing records or unsupported replay semantics cannot determine the verdict.

Observation-only describes authority under this contract, not whether the output is useful.

Authentication happens before the fork. Only complete records that can be independently recomputed under an authorized contract enter the decision path.

When complete deterministic replay is unavailable, the result remains useful context. InvarLock can retain it, hash it, sign it, render it, and bind it to the comparison. What it cannot do is manufacture the missing records or let the observation determine policy.

MLflow and Garak stop for different reasons

MLflow stops because its retained aggregate cannot reconstruct the ordered records. Garak stops for a different reason: the qualification contract examined here does not support its retained replay semantics.

Both integration paths are maintained, and both outputs remain useful for provenance and review. “Observation-only” is an authority classification, not a failure label. The full support, replay, and signed-journey matrix is covered in An Evaluator Can Be Supported Without Being Decision-Ready.

There is no command-line switch that promotes an observation into authoritative records. When a caller requires replay authority, InvarLock rejects the observation-only import.

Authenticated diagnostics stay diagnostic

The standalone diagnostics package makes the same separation explicit for numerical context. Spectral, random-matrix, and variance summaries can be emitted as canonical JSON and attached to a signed evidence transaction.

Strict verification can then establish that the observation is canonical, bound to the comparison, and unchanged. A report can display it next to the policy result. Those are meaningful properties.

They still do not turn the diagnostic into a calibrated threshold.

If a diagnostic suggests a useful future rule, the team must define its semantics, calibrate its threshold, and review its failure modes. The approved rule then belongs in a new versioned policy and transaction; it cannot change the meaning of an old receipt.

The SPDX observation example applies the same rule to software-supply-chain metadata: it checks the declaration’s artifact binding while recording full schema and semantic validation as not_evaluated. The declaration remains context.

Four questions before an output may affect policy

When integrating an evaluator or analysis tool, ask these questions in order:

The first three answers must all be yes before the output can enter paired policy. The fourth protects the decisions that come afterward.

This test is intentionally stricter than “is the result signed?” A signature is necessary for authenticating the retained bytes. It is not a substitute for records, replay semantics, or policy authorization.

Preserve context without granting a verdict

An observation-only lane lets reviewers retain useful provenance and diagnostics without allowing an aggregate score or an uncalibrated diagnostic to decide policy. To compare integration status across tools, use the separate evaluator evidence matrix.

Limitations

• The retained matrix demonstrates one deterministic exact-match surface, not every task or evaluator feature.
• Other replay semantics, including non-deterministic judges and human scoring, need separately defined evidence and authority contracts.
• Authentication does not establish correctness, representativeness, safety, compliance, or deployment suitability.

Sources

Website documentation explains the maintained workflow. The evaluator classifications and diagnostic behavior described here use InvarLock v0.15.0; the tagged sources preserve that evidence basis. Later contracts may authorize other record shapes.

• Evaluator qualification
• Optional diagnostics
• InvarLock 0.15.0: Replayable Deployment Evidence and Bounded Decisions
• Tagged v0.15.0 qualification matrix
• SPDX 3.0.1 AI observation