Where Verification Ends and Governance Begins
The candidate missed 219 questions. Only four were new misses. What does a passing benchmark tell a training team deciding whether learners can see unchecked answers?
InvarLock Team
Ideas, evidence, and release notes
Engineering notes on model evaluation, independent verification and the choices behind InvarLock.
62 posts across 26 topics. Latest: November 2, 2026
Read post
7 min read
Research note
When Two Evaluators Say “Exact Match”
Two evaluators can agree on every saved score and still treat another answer differently. InvarLock checks the scoring boundary before admitting those records.
Read more10 min read
Research note
Recheck the Evidence, or Rerun the Model?
A new report, a new policy, and a changed service call for different work. Start by asking which facts the review is missing.
Read more7 min read
Release note
InvarLock 0.17.0: Plan the Comparison, Explain the Result
Check judge-evaluation precision before collection, explore exact-match sensitivity, and rehearse reassessment while preserving earlier evidence.
Read more12 min read
Research note
GPT-6.1 Sol vs GPT-6 Sol: fewer tokens on successful fixes
Across 200 paired coding tasks, GPT-6.1 gained 15 fixes and lost 7. Explore why token savings, net scores and first-attempt outcomes each tell only part of the story.
Read more7 min read
Research note
The Dataset Is Part of the Signed Decision
A dataset name does not identify the cases behind a result. A signed comparison needs the exact inputs, expected answers, selection, and order—and a separate argument for why those cases matter.
Read more6 min read
Research note
What Signed Case Studies Add Beyond Conformance Fixtures
A small test with known answers checks the software. A signed case study records an actual model comparison. A reviewer needs to know which question each answers.
Read more6 min read
Research note
Five Recipient Checks Before a Technical Result Can Be Accepted
A historical policy pass can remain valid while a recipient rejects the handoff. Follow one retained example through identity, trust, requirements, missing evidence and the final decision.
Read more6 min read
Release note
InvarLock 0.16.0: Native, Captured and Judge Evidence
Compare saved evaluator answers through the main commands, collect judge ratings under a declared rubric, and read the result beside the checks that explain it.
Read more
Browse 53 older posts
9 min read
Research note
Which Answers Changed Behind the Score?
All ten lost successes select valid but wrong choices; two other answers become empty without changing their score. Inspect the original records and reproduce the checks.
Read more10 min read
Research note
Three GGUF Comparisons: Different Regressions, Shared Errors
Five different losses follow a shared correct starting point, while 30 questions keep one wrong answer across all six outputs. Reproduce the findings from unchanged public records.
Read more5 min read
Research note
A Passing Release Test Can Contain Policy Rejections
In the retained preflight example, all eleven references must reproduce their expected outcomes: eight policy passes and three integrity-valid policy rejections.
Read more5 min read
Research note
The Proof Stops at the Signed Schedule
A passing 400-question comparison can support its declared requirement while leaving improvement and performance on your workload unresolved. Those are separate claims.
Read more5 min read
Research note
A Signed Score Is Not Yet Decision Evidence
A missing answer record, unsupported scoring and an uncalibrated diagnostic need different repairs. Re-signing the output solves none of them.
Read more8 min read
Release note
InvarLock 0.15.0: Replayable Deployment Evidence and Bounded Decisions
InvarLock 0.15 adds replayable deployment evidence, an absolute accuracy floor, four retained evaluator journeys, and a protected consumer example.
Read more6 min read
Research note
Four Evidence Packs, Four Narrow Answers
Two engines scored zero and still passed their non-regression rule. Four retained comparisons show why a pass needs its metric, uncertainty and requirements beside it.
Read more6 min read
Research note
An Evaluator Can Be Supported Without Being Decision-Ready
Before choosing an evaluator integration, ask which work has actually been demonstrated. A maintained adapter, replayable scores and a completed model comparison answer different questions.
Read more7 min read
Release note
InvarLock 0.14.0: Evaluator Evidence In, Recipient Policy Out
v0.14 normalizes external evaluator records, then hands a signed technical receipt to recipient-owned trust and acceptance rules.
Read more9 min read
Release note
InvarLock 0.13.0: One Request, One Signed Decision Trail
One YAML request now binds artifacts, schedule, metric, and policy. One signed evidence pack carries that comparison through independent verification and reporting.
Read more8 min read
Paper note
A Valid Evaluation Artifact Is Not Yet a Release Decision
A 340-run study separates three release gates: artifact readiness, evidence saturation, and host parity. Each answers a question that a valid evaluation report leaves open.
Read more3 min read
Research Note
Invariants Are Necessary but Not Sufficient
Structural checks matter because they prevent obviously broken edits from reaching evaluation. They do not, by themselves, show that an edit preserved quality.
Read more2 min read
Release
Guard-value contracts and stock-clean attention control
InvarLock 0.12.1 tightens the Mistral guard-value evidence contract, promotes targeted spectral/RMT/VE probes to required detections, and replaces the attention control with a stock-cap-clean 1.05x lane.
Read more2 min read
Release
Self-edit evidence, backend compatibility, and larger model lanes
InvarLock 0.12.0 adds knowledge/self-edit workflow metadata, widens BYOE edit-lane evidence, and reorganizes public evidence around backend compatibility and larger model findings.
Read more4 min read
Evidence Note
What Evidence Packs Still Do Not Prove
Even when an evidence pack is strict, signed, and marked PASS, it is still a portable verification bundle. It does not answer every scientific, safety, or deployment question a reader might ask.
Read more5 min read
Research Note
How to Archive a Model-Edit Decision So Someone Else Can Recheck It
Archiving a model-edit decision is not about saving more files. It is about preserving the exact bundle another reviewer would need to re-check the result later.
Read more3 min read
Release
Report outlines, guard warnings, and wider public evidence
InvarLock 0.11.0 makes human reports more consistent, separates guard warnings from hard failures, and expands the public evidence surface across model families.
Read more5 min read
Research Note
Evidence Packs, Not Screenshots
A screenshot can communicate a result. An evidence pack can be inspected, checked, and re-verified later. That is the difference between presentation and portable evidence.
Read more4 min read
Research Note
Runtime Manifests and Why Provenance Must Travel With the Result
A strong evaluation result should carry its runtime provenance with it. In InvarLock, that means the runtime manifest travels next to the report and is rechecked by invarlock verify.
Read more2 min read
Release
Evidence packs, authenticity, and quantized-adapter validation
InvarLock 0.10.0 makes public evidence packs more explicit, adds signer-authenticity checks, and expands optional quantized-subject adapter validation.
Read more5 min read
Research Note
What Belongs in evaluation.report.json
An evaluation report is strongest when it is treated as a stable evidence contract: a small required core, meaningful optional blocks, and a clear boundary around what still lives outside the JSON.
Read more5 min read
Research Note
Calibration Is the Product Surface, Not a Side Utility
Calibration is not just analysis around the product. It changes how thresholds are derived, when correction paths may turn on, and which policy values later govern reports.
Read more2 min read
Release
Strict assurance and runtime provenance
InvarLock 0.9.0 adds strict assurance mode, fail-closed verifier checks, runtime provenance guidance, and maintainer evidence gates for release review.
Read more5 min read
Research Note
From Sweep Outputs to Tier Policy
Calibration becomes operational when sweep artifacts end in reviewable YAML patches that later appear as resolved runtime policy in reports.
Read more5 min read
Research Note
Variance Enablement Should Be Evidence-Gated
Variance equalization is stronger when it must earn enablement through predictive evidence, explicit tier knobs, and report-visible provenance.
Read more6 min read
Research Note
Null Sweeps as Threshold Derivation, Not Tuning Folklore
Thresholds are stronger when they come from measured null behavior and end in a policy patch, not from knob-tuning folklore.
Read more5 min read
Research Note
The Minimum Evidence Surface for Trustworthy Weight-Edit Results
A trustworthy weight-edit result needs more than a benchmark delta. It needs a bounded claim, an exactly paired comparison, and verification that rejects incomplete evidence.
Read more2 min read
Release
Evidence packs and explicit runtime provenance
InvarLock 0.8.0 moves the public bundle surface to evidence packs, pins docs to versioned release paths, and makes container-vs-host runtime provenance explicit across evaluate and verify.
Read more5 min read
Research Note
Fail-Closed Verification for Weight-Edit Evaluation
A verifier is only useful if it rejects incomplete evidence. InvarLock's verification path is designed to stop stronger claims when the evidence bundle is missing or inconsistent.
Read more2 min read
Release
Tag-based publishing with slimmer release verification
InvarLock 0.7.2 simplifies the public release surface around immutable source tags plus the PyPI wheel and sdist, with docs and verification gates aligned around that path.
Read more5 min read
Research Note
Why Paired Evaluation Beats Before/After Benchmarks
A model-edit benchmark number is only as strong as the comparison behind it. Pairing makes the comparison inspectable.
Read more2 min read
Release
Standalone contract bundles with tighter release gates
InvarLock 0.7.1 makes wheel-only verify/report workflows first-class, ships a public contract bundle, and tightens supply-chain and release-validation gates.
Read more2 min read
Release
GPT-OSS pilots with CUDA-ready attested lanes
InvarLock 0.7.0 adds first-class GPT-OSS support, pilot Ministral 3 8B/14B presets, and a CUDA-capable attested runtime path for GPU hosts.
Read more6 min read
Research Note
What InvarLock Actually Claims
A narrow claim can be stronger than a broad one. InvarLock is about auditable regression risk from weight edits, not general model safety.
Read more2 min read
Release
Gemma 4 pilot lanes with a clearer assurance contract
InvarLock 0.6.0 adds a shipped Gemma 4 E2B text lane, phase-1 multimodal evaluation, and a unified `--assurance attested|trusted-local` workflow.
Read more2 min read
Release
Attested smoke lanes with package-native evidence pack signing
InvarLock 0.5.1 adds a push-gated tiny attested smoke lane, a scheduled GPT-2 canary lane, and package-native Ed25519 evidence pack signatures.
Read more2 min read
Release
Offline release verification with a slimmer public CLI
InvarLock 0.5.0 adds offline release-verification bundles, package-native evidence-pack verification, and a simplified public CLI centered on evaluate, verify, and report.
Read more2 min read
Release
Stable public contracts with stricter fail-closed verification
InvarLock 0.4.0 stabilizes contracts around policies, evidence packs, and evaluation provenance while tightening verification, CI, and coverage enforcement.
Read more2 min read
Release
Coverage floors and fail-closed CLI/reporting paths
Split-module coverage thresholds now protect critical CLI/reporting paths while config, plugin, report, overhead, and observability edge cases fail closed more reliably.
Read more2 min read
Release
Quantization, spectral, and report-schema hardening
A focused hardening release: safer AWQ plugin discovery, stronger quantization clipping behavior, and broader report-schema acceptance for edge payloads.
Read more2 min read
Release
Evidence pack showcase coverage and reproducible CI
Evidence packs add new showcase and evidence artifacts, while CI and release flows become more deterministic and easier to validate repeatedly.
Read more2 min read
Release
Report rename cleanup and offline evidence-pack hardening
A stability-focused release: cleaner report output, safer offline evidence-pack flows, and CI/test hardening after the report rename.
Read more2 min read
Release
Evaluation reports, strict evidence packs, and Transformers v5
A terminology reset (report/evaluate), stricter evidence-pack verification, and a clean upgrade path for Hugging Face Transformers v5.
Read more2 min read
Release
Proof packs v2 and role-based adapter routing
Adapters move to role-based routing, proof packs become easier to inspect (v2 layout), and reporting output gets a readability upgrade.
Read more2 min read
Release
Measurement contracts for CI and release verification
Certificates now record and enforce estimator measurement contracts under CI/release profiles, and B200 validation suites can cleanly split calibration vs execution.
Read more2 min read
Release
Deterministic evidence packs and safer perplexity runs
Evidence packs gain a deterministic bash test suite and better runtime helpers, window selection becomes stable/offline, and perplexity runs get safer around bad token IDs.
Read more2 min read
Release
Fail-closed baseline pairing with lower-memory retries
CI/release baseline pairing is fail-closed (pairing evidence is required), and adapters reduce peak memory during retries via chunked snapshot/restore.
Read more1 min read
Release
Token-weighted paired statistics and stricter release gates
Token-weighted paired bootstrap lands across the pipeline, strictness toggles expand, and CI/release pairing expectations become explicit and enforceable.
Read more1 min read
Release
Calibration, determinism, and regression protection
`invarlock calibrate` arrives, determinism utilities mature, and regression harness + golden tracking help prevent silent policy drift.
Read more2 min read
Release
Large-model reload stability and B200 controls
Fixes a GPU memory leak during reload fallback, hardens B200 scripts, and adds practical controls for acceptance ranges and overhead measurement.
Read more1 min read
Release
Quantization-aware adapters and safe device movement
First-class quantization metadata, safer device movement across quantized models, auto-routing based on checkpoint info, and major test coverage expansion.
Read more1 min read
Release
Public compare/certify pipeline and Safety Certificate schema v1
The initial public release on GitHub and PyPI: core compare/certify pipeline, guard chain, Safety Certificate schema v1, and the first docs/CLI surface.
Read more3 min read
Announcement
Welcome to InvarLock
The original InvarLock introduction, with a current starting point for native comparisons, captured results and bounded judging.
Read more