Back to blog

Evaluation reports, strict evidence packs, and Transformers v5

A terminology reset (report/evaluate), stricter evidence-pack verification, and a clean upgrade path for Hugging Face Transformers v5.

2 min readInvarLock Team
A loose imprint recedes beside a clear measured surface, emphasizing reporting evidence rather than a broad certification impression.

Release: InvarLock 0.3.8 - Report/evaluate terminology and stricter pack verification

Highlights

  • “Certificate” is now “report” everywhere (and evaluate replaces certify).
  • Evidence packs can now verify in strict mode (fail-closed when signatures/contents don’t match expectations).
  • Runtime summaries, version flags, Transformers v5 loading, and report bundle filenames are aligned.

This release is mostly about making the story of an InvarLock run easier to understand for both operators and reviewers. The breaking cleanup is broad: “certificate” becomes report across artifacts, docs, scripts, notebooks, and Python API surfaces, while CLI terminology converges on evaluate instead of certify.

Evidence packs get stricter evidence handling. verify_pack.sh --strict and PACK_STRICT_MODE=1 can fail closed on missing or invalid GPG signatures, unexpected pack contents, checksum tampering, and extra files. Manifests now bind the checksum digest and can record the signing key fingerprint, which makes “what exactly did I verify?” a more answerable question.

On the ecosystem side, this release adds --version / -V, includes runtime and confidence-interval information in evaluate summaries, updates bundle filenames to evaluation.report.json and evaluation_report.md, rejects legacy Transformers v4 load keys, and migrates loading contracts for Transformers v5. It also reduces noisy Hugging Face output in CI/release profiles and keeps calibrate import-safe for docs/example validation without torch installed.

Sources

Website documentation explains the maintained workflow. The changes described here belong to InvarLock v0.3.8; the tagged release record preserves that historical scope.

More in Release

Explore nearby related posts.