What Evidence Packs Still Do Not Prove

Historical context: This article preserves the product model and examples described at its original publication date. The v0.13.0 transition replaced that operator workflow. Documentation links lead to maintained guidance; the retired commands and contracts below are not present-day instructions.

Stronger packaging should narrow claims, not invite overclaim

Highlights

• Evidence packs are strong at portability, integrity, and package-level re-verification.
• They are not universal proof of representativeness, safety, or deployment readiness.
• Stating that boundary plainly makes the claim stronger, not weaker.

On June 15, Evidence Packs, Not Screenshots argued for portable evidence over screenshot-style presentation. The archive synthesis then showed how that portability fits into a re-checkable model-edit decision.

That portability argument still stands.

Strong packaging can create a different risk: readers start attributing guarantees to the pack that the pack never actually made.

The public InvarLock docs make a narrower promise.

What Evidence Packs Secure

The evidence-pack docs define a useful promise. A pack can carry reports, a reviewer summary, manifest, checksums, a package-native signature bundle, and a package-native verification path. The historical package-native verification path requires a signed manifest.signature.json, validates checksum binding and digest-backed references, checks checksums.sha256, and runs nested report verification. On the recommended strict path, --strict adds no-extra-files discipline, while --report-assurance strict requires every bundled clean report to satisfy strict report assurance.

That matters because the evidence remains portable, structured, and checkable after the original run environment is gone.

The limitations only make sense if those strengths stay visible.

What Evidence Packs Do Not Prove

The assurance case overview gives the clearest boundary.

InvarLock's scope is intentionally about regression risk from weight edits under a chosen baseline, configuration, and documented tiers. It is explicitly not a universal claim about content safety, alignment, prompt-level attacks, deployment hardening, or arbitrary generalization outside the documented support surface.

An evidence pack does not widen that scope. It packages and verifies evidence inside that scope. It does not silently answer questions that the underlying assurance case declares out of scope.

Why Underlying Report Quality Still Matters

Even a strong pack depends on what it contains.

If the bundled reports are narrow, underpowered, or mismatched to the question a reviewer cares about, packaging them well does not change that fact. A strict evidence pack can show that the bundle is intact and re-verifiable. It cannot turn a weak underlying evaluation into a strong scientific result.

This is the central conceptual distinction: package integrity is not the same thing as claim sufficiency.

Why This Distinction Improves Credibility

Stating the limit plainly has a practical benefit.

If a team acts as though evidence packs prove everything, the next skeptical reader will find the boundary anyway and trust the rest less. If the team says up front that evidence packs secure portability, integrity, and re-verification inside a narrow scope, the claim becomes more durable.

Here, restraint is a direct credibility gain.

Practical Reading Rule

When you receive an evidence pack, ask two separate questions:

• what does the pack let me verify about integrity, provenance, and report structure?
• what does the underlying evaluation still leave unproven?

Those questions belong together, but they are not the same question.

Limitations

• The packaging and command examples describe the historical evidence-pack format identified above.
• Authentication and retention do not establish genuine execution, representative evaluation, general safety, or deployment approval.
• Missing or weak underlying evidence remains missing or weak after packaging.

Sources

Website documentation explains the maintained workflow. The examples described here preserve the historical workflow at the article’s original publication date. These maintained guides explain the related concepts; they do not reinstate the retired commands or contracts.

• Evidence and verification
• Assurance case
• Reports and receipts
• Evidence artifacts
• Evidence Packs, Not Screenshots
• How to Archive a Model-Edit Decision So Someone Else Can Recheck It