Evidence artifacts

An invarlock/evidence-pack-v1 bundle is a closed, signed evidence directory. Its paths are fixed by the manifest schema; additional files are rejected in strict verification.

Reference

• Surface: invarlock/evidence-pack-v1 directory, manifest, payload inventory, and external outputs
• Stability: Versioned public artifact contract; fixed paths and closed inventory are verification requirements
• Use this page when: Inspecting a bundle, implementing artifact storage, or determining which bytes carry a particular claim

Diagram

Evidence dependency map

Directory layout

Diagram

The closed evidence directory assigns one fixed path to the manifest, signature, checksum inventory, inputs, schedule, paired records, canonical report, per-side runs, and provider material.

inputs/*.json are identity records, not copies of model weights, the dataset, or the policy. Each carries a material digest and may carry a stable locator or media type. The exact normalized request, schedule, provider material, paired records, and canonical report are separate payloads referenced by the manifest.

Manifest

manifest.json contains exactly:

• format and deterministic comparison_id;
• six input references: baseline, subject, dataset, both runtimes, and policy;
• fifteen evidence-role references at their fixed paths;
• the paired-record path, digest, and count;
• optional typed observation references with fixed paths, digests, kinds, and scopes;
• the checksum-file path and digest; and
• the evidence-signing key fingerprint.

Every input reference has a digest of the identity file and a distinct material_digest for the authenticated input. Every evidence reference has a digest of the exact file bytes.

The manifest structure is equivalent to this abbreviated, non-copyable view:

{
  "format": "invarlock/evidence-pack-v1",
  "comparison_id": "...",
  "inputs": {
    "baseline": {
      "path": "inputs/baseline.json",
      "digest": "sha256:...",
      "material_digest": "sha256:..."
    },
    "subject": { ... },
    "dataset": { ... },
    "baseline_runtime": { ... },
    "subject_runtime": { ... },
    "policy": { ... }
  },
  "evidence": {
    "request": {"path": "request.json", "digest": "sha256:..."},
    "schedule": {"path": "schedule/runtime-behavioral-schedule.json", "digest": "sha256:..."},
    "evaluation_report": {"path": "reports/evaluation.report.json", "digest": "sha256:..."},
    "...": "twelve additional fixed evidence roles"
  },
  "paired_records": {
    "path": "records/paired-records.json",
    "digest": "sha256:...",
    "count": 20
  },
  "checksums_sha256": "checksums.sha256",
  "checksums_sha256_digest": "...bare sha256...",
  "signing_key_fingerprint": "sha256:..."
}

The real object has no comments, ellipses, or abbreviated roles. It has exactly six input references and fifteen evidence references at schema-fixed paths.

manifest.signature.json embeds the Ed25519 public key, its fingerprint, and a base64 signature over the exact canonical manifest.json bytes. The public key is evidence of who signed only after its fingerprint is matched to an independent expected signer.

Checksums and closed inventory

checksums.sha256 covers every payload below the manifest, but not the manifest or its signature. The manifest separately binds the checksum file. Verification checks both directions:

• every manifest payload must be covered; and
• no checksum or directory entry may introduce an undeclared payload.

Regular-file, no-symbolic-link, size, and safe-relative-path rules apply while reading. A valid checksum does not make a file semantically valid; cross-file replay follows the integrity checks.

The manifest is outside the checksum payload because it binds the checksum file. The signature is outside both because it authenticates the canonical manifest. This avoids a self-referential digest while still closing every pack byte into one directed dependency graph.

Paired records

records/paired-records.json has exactly:

{
  "format": "invarlock/paired-records-v1",
  "metric": "exact_match",
  "schedule_sha256": "...",
  "records": [
    {
      "record_id": "example-1",
      "input_sha256": "...",
      "baseline": {
        "observation_record_digest": "sha256:...",
        "score": 1.0
      },
      "subject": {
        "observation_record_digest": "sha256:...",
        "score": 1.0
      }
    }
  ]
}

The engine derives this file from the canonical schedule and both scoring observations. It rejects failed observation records, order changes, input-digest mismatches, output-text/digest mismatches, and schedules without expected outputs. Import mode must supply byte-canonical paired records equal to that fresh derivation; imported scores are never accepted as authority.

For exact_match, each score is 1.0 when output text exactly equals expected output and 0.0 otherwise. For normalized_nll_per_utf8_byte, each score is -logprob_sum / utf8_byte_count; required counts must be positive and the result finite and non-negative.

When a scorer extension is selected, the paired-record object also binds the exact scorer configuration and each side's replay result. The explicitly authorized scorer receives only authenticated expected_output, output_text, and output_sha256 facts and returns one higher-is-better value in [0, 1] per record. The core verifies result order and digests, computes the arithmetic mean and paired percentage-point delta, and owns the interval and verdict.

Normalized NLL is teacher-forced expected-continuation likelihood regression, not a general model-quality measure. The paired-record object may also carry a verifier-derived perplexity interpretation when the authenticated tokenizer contracts and every pair's positive target-token counts match. That object has no policy, interval, or verdict authority; an unavailable interpretation does not invalidate the byte-normalized NLL comparison.

The canonical report adds the metric-specific paired interval to the two side means and point comparison. New reports use invarlock/comparison-report-v2; the verifier also replays existing invarlock/comparison-report-v1 packs under their original arithmetic. Exact match records paired regression and improvement counts, an exact two-sided McNemar probability, and the paired Newcombe 95% effect-size interval whose lower bound controls policy. Normalized NLL uses the upper bound of its authenticated-schedule resampling interval. An authorized scorer extension uses the lower bound of the same fixed paired-resampling method against metrics.scorer_extension.delta_min_pp.

When the policy supplies the coupled sample controls, the v2 report also contains sample_qualification. It records the minimum and observed paired count, maximum and observed interval width, units, each check's result, and the combined result. The final verdict is the conjunction of that result and the metric-bound result.

Provider sidecars

Each comparison side includes:

The evidence transaction copies these exact bytes. It does not regenerate sidecars in a way that could break their original runtime-manifest bindings.

Publication and immutability

Evaluation writes a staging directory, flushes files to disk, signs the manifest, and atomically renames the directory into place. It never replaces an existing destination. Published files are read-only and directories are non-writable.

Filesystem permissions are a local hardening measure, not the integrity model. Any later byte change, missing file, or extra file is detected by strict verification. Verification receipts and rendered HTML must remain outside the bundle so the bundle can stay byte-identical.

External outputs

Multiple verifiers can issue separate receipts for the same immutable manifest. They can use distinct verifier identities and keys. Each must independently supply the expected artifact, schedule, policy, runtime, and evidence-signer anchors. The policy bytes must match the policy material already bound by the pack; a different policy requires a newly evaluated evidence bundle.

Related documentation

• Public contracts defines the manifest schema, canonical JSON, and digest rules.
• Reports and receipts defines the report and external receipt carried by or associated with a bundle.
• Evaluation lifecycle explains atomic publication and failure boundaries.
• Architecture places the artifact inventory within the evidence-signer-verifier trust model.