Fail-Closed Verification for Weight-Edit Evaluation

Historical context: This article preserves the product model and examples described at its original publication date. The v0.13.0 transition replaced that operator workflow. Documentation links lead to maintained guidance; the retired commands and contracts below are not present-day instructions.

A clean result should require a complete bundle

Highlights

• invarlock verify covers five surfaces: schema, pairing, ratio math, measurement contracts, runtime provenance.
• In CI and Release, pairing (E001), digest mismatches (E002, E006), and verification failures (E601) are hard aborts, not warnings.
• Exit code 3 is reserved for hard aborts; a softer-looking pass is not available when required evidence is missing.

A verification story can look stronger than it is when incomplete evidence degrades gracefully. A report is missing a sidecar. A baseline artifact cannot be found. Pairing is weaker than expected. Instead of rejecting the claim, the system simply keeps going and leaves a cleaner-looking result than the evidence deserves.

That is the behavior a serious evaluation boundary should avoid.

Fail-closed verification keeps the claim boundary narrow. If the evidence bundle is incomplete or inconsistent, the stronger claim stops there rather than turning into a reassuring summary.

What “Fail Closed” Means Here

In this context, fail closed does not mean “crash on everything.” It means that verification should refuse to validate a report bundle when the conditions required for the public claim are not present.

The CLI reference is explicit about this posture. invarlock verify is not described as a convenience formatter. It verifies report JSONs against schema, pairing math, profile gates, and the required runtime provenance for container-backed evaluation outputs. In CI and Release profiles, that path is intentionally strict.

That strictness is the point. If a result depends on paired evaluation, deterministic provenance, and container-backed execution, then those conditions should not become optional at the last step.

What The Verifier Actually Checks

The reports reference gives a compact view of the verification path:

• schema
• pairing
• ratio math
• measurement contracts
• runtime provenance

That list matters because it separates “the file parsed” from “the evidence is valid enough to support the claim.”

The assurance overview reinforces the same idea from the claim side. It ties public claims to runtime enforcement and observability, not to prose alone. Pairing, bootstrap sanity, deterministic evaluation, and primary-metric contracts are all written as enforceable conditions rather than advisory notes.

Why The Runtime Manifest And Baseline Report Matter

Fail-closed verification is not only about the main report file.

The public docs repeatedly say that container-backed evaluations emit runtime.manifest.json next to evaluation.report.json, and that they should be archived and verified together. The artifact-layout reference makes the same point operational: baseline and subject reports, the evaluation report, and the runtime manifest belong together because verification depends on that evidence surface staying intact.

The report and runtime manifest carry different checks:

{
  "reports/eval/evaluation.report.json": ["schema", "pairing", "ratio_math"],
  "reports/eval/runtime.manifest.json": ["runtime_provenance", "report_digest_binding"]
}

This keeps the evidence bundle intact for later verification. If the report travels without the runtime provenance sidecar or without the baseline material needed for pairing checks, the system should not quietly pretend the bundle is still complete.

How Hard-Abort Errors Protect The Claim Boundary

The troubleshooting guide states the fail-closed posture in operational terms.

Pairing errors like E001, digest mismatches like E002 or E006, and verification failures such as E601 are not described as minor quality warnings in CI and Release. They are hard-abort conditions. The CLI reference says the same thing at the command level: exit code 3 is reserved for hard aborts when evidence would be invalid.

That detail matters. A verification command that can distinguish “this report parsed” from “this report is valid enough to gate a release” is enforcing a meaningful boundary. A command that normalizes missing or contradictory evidence into a softer pass is mostly doing presentation work.

What Fail-Closed Verification Still Does Not Do

The verifier's job is still narrow.

Fail-closed verification does not prove the model is good. It does not replace human review. It does not guarantee deployment security or broader governance hygiene. What it does is narrower: it protects the evidence boundary around the specific public claim InvarLock is making.

That is enough. A verifier does not need to solve the whole problem to be useful. It needs to stop weak evidence from being mistaken for strong evidence.

Claim Map

The practical structure is:

• evaluation emits evaluation.report.json
• container-backed paths also emit runtime.manifest.json
• verification checks schema, pairing, math, and contract consistency
• CI and Release profiles hard-abort when required evidence is missing or mismatched

That is what “fail closed” should mean here: stronger conclusions require complete evidence.

Limitations

• The profile-specific hard stops describe the historical verifier. Present-day commands and authority rules require the maintained documentation.
• Rejecting incomplete evidence does not establish the quality or safety of a model whose evidence clears the checks.
• A valid comparison remains conditional on its baseline, records, metric, and policy.

Sources

Website documentation explains the maintained workflow. The examples described here preserve the historical workflow at the article’s original publication date. These maintained guides explain the related concepts; they do not reinstate the retired commands or contracts.

• Documentation
• Assurance case
• Command-line interface
• Reports and receipts
• Evidence artifacts
• Troubleshooting