Where Verification Ends and Governance Begins

The smaller Ministral model passed its benchmark policy. It also answered 219 of the 400 questions incorrectly. If you owned a training service, would you let it write quiz answers?

The comparison gives you a precise answer to a narrower question. On the same questions, the candidate lost four answers the baseline had right and gained six the baseline had missed. That was enough to meet the recorded tolerance for accuracy loss. Whether staff can catch a convincing wrong answer before it reaches a learner is a different question.

The client review example puts these decisions on the same page. Its measurements come from a real comparison. Its training team and its decision to allow a supervised trial are fictional. Reading the two together shows what a verifier can settle and what a release owner still has to decide.

What the receipt preserves

The comparison changed both the model artifact and its runtime, from BF16 weights in Transformers to Q5_K_M GGUF in llama.cpp. On 400 paired MMLU-Pro questions, exact-match accuracy went from 44.75% to 45.25%. The estimated change was +0.50 percentage points, with a paired 95% interval from −1.12 to +2.12 points. The lower bound cleared the allowed loss of two points. The interval includes zero, so the result leaves the direction of the underlying change uncertain.

A verifier can check the signed evidence against independently approved model, runtime, schedule and policy identities. It can recalculate the scores and the verdict from the saved answers. The retained signed receipt records which comparison the verifier checked and the policy pass it found. A future reviewer can inspect that result even if the organization later changes its mind about using the model.

Diagram

The verifier preserves the benchmark result. The recipient checks the handoff, the deployment job binds the candidate, and the release owner records permitted use.

One receipt, three decisions about its use

Technical verifier

What did the signed comparison establish?

Recipient

Does this handoff meet our trust policy?

Deployment job

Are these the approved candidate bytes?

Release owner

Who may use it, and under what conditions?

Each decision refers back to the original comparison. Changing the permitted use leaves the receipt intact.

What has to travel with the model

Suppose the training team receives the candidate and a copy of the passing report. The recipient needs a way to tell whether the files it received are the ones the verifier checked. It also needs to know whether its own rules authorize the signer and accept evidence of that age.

A portable acceptance handoff binds the artifact to signed evidence that the recipient can check under its own policy. In the released handoff example, the intended submission is accepted. Supplying the wrong artifact or applying a stricter recipient policy causes rejection, while the historical verification still records a pass. These are controlled examples of the handoff rules. The team in the quiz scenario would have to perform its own check.

There is one more place to lose the connection. A deployment job might receive the right receipt and then fetch a model called latest. If that label moved, the job could install a candidate nobody reviewed. The public native-pack workflow shows how to close the gap. The production job reauthenticates the verifier’s receipt using inputs controlled by production. Its deployment adapter must match the artifact it actually fetches to the approved candidate identity before invoking deployment. A successful upstream job alone cannot identify the bytes at that last step.

What a release owner can authorize

In the fictional review, Maya Chen authorizes two weeks of internal use by five staff reviewers, with an answer-key check on every answer. Independent recipient verification is an entry condition. The next evidence she asks for is performance on 200 internal practice questions. The example records a decision and a plan. It reports no trial outcomes.

Why write that down separately? Consider a request to send model answers directly to learners the next morning. The verifier could reproduce exactly the same benchmark pass. Yet the proposed use removes the human check on which Maya’s decision depended. The review needs a new decision from the owner, even if the model file, receipt and all 400 answers remain identical.

This is the work a private review platform adds. It keeps the original evidence together with the people, requirements, exceptions and decisions that govern its use. If the owner limits a rollout or later withdraws approval, the platform can record who changed the decision, when and why. The original verifier’s verdict continues to mean what it meant on the day it was issued.

When handing a release to the next person, make two things easy to find. Give them the signed result and the independent inputs needed to check it. Then give them the owner’s decision, with the permitted use, conditions and next review. Someone asked to remove supervision can then see exactly whose decision must change and which evidence that person still needs.

Limitations

• The benchmark covers one model-and-runtime change on 400 fixed multiple-choice questions. It did not measure quiz-writing quality, error severity or staff review performance.
• The recipient scenarios and protected deployment workflow illustrate separate public contracts. Neither records a production deployment of this Ministral candidate.
• The client review’s trial authorization is illustrative. The retained technical receipt carries no customer approval or evidence of a completed trial.