Public contracts

The workflow uses closed, versioned JSON contracts. Source mirrors live in contracts/, and byte-identical package-owned copies ship in the core wheel. Verification always loads the package-owned copies: a working directory or environment variable cannot substitute a different schema.

Reference

Surface: Versioned request, evidence, provider, runtime, report, and receipt contracts

Stability: Closed public interchange formats; incompatible shape or meaning changes require a new format identifier

Use this page when: Authoring contract objects, validating canonical bytes, or reviewing cross-file digest and signature bindings

Schema-backed contracts

Scroll horizontally to see every column.

The acceptance predicate and recipient policy are described in Acceptance attestations. The detailed InvarLock receipt remains the authoritative replayable result. Native pack v1 and receipt v1/v2 retain their published meanings. Captured pack v2 and receipt v3 are the deterministic captured evidence family; captured metric: judge uses the separate judge envelope and receipt. Legacy monolithic comparison evidence is not accepted. A captured receipt's verification_scope: captured_comparison does not qualify native execution, acceptance attestations, ModelKit acceptance, or deployment approval. See Captured records for run, case-set, policy and request normalization contracts; these schema identifiers are not product-release pins.

Evidence sets compose existing deterministic and judge components over the same original runs. Their combined decision does not add a joint confidence guarantee or change either component's acceptance scope.

Evaluator qualification has two stable wire classifications:

These fields do not express adapter maintenance or signed-journey maturity. Those independent axes belong to the examples-layer qualification catalog, so an export cannot promote itself by claiming support or demonstration status.

Identity and evaluation-context boundaries

A model name is a display label. It does not select the evaluated artifact, package, runtime, scorer or complete execution context. Independent verification requires recipient-selected expectations and checks the relationships between the authenticated objects, in addition to validating their schemas.

Scroll horizontally to see every column.

The independently selected complete-request digest can require exact declared settings and observation contents without introducing another model hash. It does not independently confirm a hosted service's reported revision or an observation's scientific conclusion. Do not obtain an expected digest from incoming evidence and describe the resulting equality as independent trust.

Observation payloads may use a versioned profile for method, configuration, probe-set identity, assumptions, result and uncertainty. A consumer must implement that profile's semantic checks before claiming to understand or validate it. The current recipient policy does not implement a generic required lineage or context-profile result. Its optional receipt trust-profile digest identifies the selected verifier configuration; it does not create missing validation logic. Unknown policy fields reject, while opaque observation payloads retain only observation authority.

Declared transformation history, empirical similarity and exact artifact identity remain separate. No supported behavioral fingerprint proves ancestry merely because its payload is signed. A future profile must avoid including the complete normalized-request digest inside a payload already hashed by that request; bind component identities first and let the envelope add comparison bindings after normalization.

The evaluation-record contracts bind complete captured runs and policy bytes, including source versions and per-record context. Their independent run digests include outputs, so they are not pre-execution context identities. Replaying signed captured evidence verifies the comparison; it does not prove that an untrusted capture worker executed the declared model. The captured rehearsal independently pins the protocol and capture before reconstructing those runs.

Provider contracts

The provider ABI uses these schema-backed documents:

Scroll horizontally to see every column.

Python callers can load these exact packaged objects with functions from invarlock.public_contracts. Those loaders return schema dictionaries; they do not validate semantic cross-bindings by themselves.

from invarlock.public_contracts import (
    load_evaluation_request_schema,
    load_evidence_observation_schema,
    load_evidence_pack_schema,
    load_trust_inputs_schema,
    load_acceptance_predicate_schema,
    load_recipient_acceptance_policy_schema,
    load_model_artifact_identity_schema,
    load_runtime_behavioral_schedule_schema,
    load_runtime_manifest_schema,
    load_runtime_provider_capabilities_schema,
    load_runtime_provider_receipt_schema,
    load_runtime_scoring_observation_schema,
)

Each loader returns a new dictionary decoded from the package-owned contract. ContractLoadError identifies a missing, malformed, or non-object packaged contract. Format constants such as EVALUATION_REQUEST_FORMAT_VERSION, EVIDENCE_PACK_FORMAT_VERSION, TRUST_INPUTS_FORMAT_VERSION, and RUNTIME_PROVIDER_ABI_VERSION are exported from the same module for exact comparisons.

The schemas use JSON Schema Draft 2020-12. Every contract object is closed: fields not named by its schema or exact code-enforced shape are rejected rather than ignored.

Independent trust-input profile

invarlock/trust-inputs-v1 is the portable caller-owned input to independent verification. It contains the policy path, baseline and subject artifact digests, schedule digest, both runtime digests, evidence-signer fingerprint, verifier identity, verifier signing-key path, and installed-scorer authorization. The object and all nested objects are closed.

Policy and key paths are safe relative paths resolved from the profile's directory. Absolute paths, traversal, symlinks, duplicate JSON members, unknown fields, and missing files are rejected. Formatting does not affect the profile digest: the loader hashes canonical JSON and the verifier records that digest in its signed receipt. The profile never contains private-key bytes.

Evaluation request fields

The request root contains four required fields and one optional field:

Scroll horizontally to see every column.

Each comparison side has the same closed shape:

For metric: judge, comparison.judge additionally requires a private workspace and signer_identity, and comparison.policy names the closed invarlock/native-judge-policy-v1 recipe. Its code-enforced plan and analysis bindings are described in judge measurements.

comparison.policy is always a safe relative path. comparison.dataset is mode-specific:

• run mode requires a closed local-dataset object with path, bare lowercase sha256, format: jsonl, name, split, input_field, and expected_output_field, plus optional id_field, an all-or-none content role and field mapping, and limit;
• import mode requires a safe relative path to canonical invarlock/runtime-behavioral-schedule-v1 bytes.

The canonical comparison.task binds the request, schedule, provider capabilities, evaluation batches, and provider receipts. Built-in identifiers are text_causal, masked_language, text_seq2seq, and vision_text_generation; a provider must explicitly declare execution support. The request selects exactly one built-in metric or one complete scorer_extension binding. Built-in scorers are exact_match, normalized_nll_per_utf8_byte and judge. Providers declare the required collection metric: judge and deterministic scorer extensions use exact_match to retain complete text outputs; normalized NLL requires its own likelihood facts. A scorer extension uses that authenticated collection so that expected and observed text are authenticated for verifier replay. The built-in hf_transformers provider declares exact-match and normalized-NLL collection. The first-party llama_cpp, tensorrt_llm, and hf_vision_text add-ins currently declare exact match for their tasks.

Deterministic scorer extension

comparison.scorer_extension is a closed invarlock/scorer-extension-binding-v1 object. It binds scorer ABI 1, a dotted scorer ID, semantic version, descriptor digest, configuration object, and canonical configuration digest. The descriptor fixes supported tasks and input/output kinds, the configuration-schema digest, and these v1 semantics:

• replay reads exactly authenticated expected_output, output_text, and output_sha256 facts for every ordered record;
• each result is a finite higher-is-better value in [0, 1];
• the core computes the arithmetic mean, subject-minus-baseline percentage- point delta, and fixed 2,048-replicate paired interval; and
• network access, external models and externally assigned ratings are forbidden in a deterministic extension scorer.

The independently supplied policy must contain resolved_policy.metrics.scorer_extension with the same scorer_id, scorer_version, descriptor_sha256, and configuration_sha256, plus delta_min_pp. Evaluation and verification require an explicitly authorized ScorerExtensionRegistry; the request and evidence cannot authorize scorer code. The verifier runs the scorer twice, requires identical canonical results, then independently reconstructs the core-owned aggregate, paired interval, threshold comparison, and verdict.

Core ships invarlock.normalized_match, invarlock.numeric_tolerance, invarlock.json_fields, invarlock.json_exact and invarlock.token_f1 through this boundary. The CLI enables them without --allow-installed-scorers; SDK callers supply ScorerExtensionRegistry(allow_installed=False). Their exact version, descriptor and configuration bindings remain mandatory. Additional implementations, such as a VQA normalization scorer, require separate installation or caller injection and explicit authorization. SQL or code execution, model-based semantic similarity, network services, externally supplied ratings, and LLM judges require different trust contracts. The bounded frozen-answer judge formats provide one such contract for their declared text profile; other judge outputs can be attached as authenticated observations without technical-verdict authority.

Evaluator input boundary

Native provider import mode is an extension boundary for measurements produced by an external evaluator. Under that contract, output is admissible for an acceptance decision only when InvarLock can authenticate the ordered per-record inputs and outputs, bind them to the exact schedule, artifacts, runtime, and source, and deterministically recompute the decision-contract metric or authorized scorer.

Captured evaluator integrations use the separate evaluation-record contracts. They bind supplied runs and scorer-specific facts without requiring native provider sidecars or asserting native execution. Captured judge requests use the bounded judge evidence and recipient-policy contracts.

An adapter alone does not establish evaluator neutrality. The generic qualification boundary binds the profile, independent schedule, normalized export, retained upstream output, runner bundle, and dependency declaration. For a deterministic exact-match profile, every ordered input and output must be present and InvarLock independently recomputes every score. Aggregate-only results, missing or reordered record facts, and external-judge outputs outside the bounded judge-measurement profile remain observation-only and expose no runtime-import records.

The maintained evaluator qualification matrix executes representative upstream tools through example-owned runners. Every deterministic profile also scores and replays the complete 102-record output of a pinned real model evaluation through the runtime-import boundary. The matrix separately records profiles that demonstrate the deeper model-running, signed transaction journey. These levels record evidence maturity rather than a permanent support hierarchy and can advance without changing the generic boundary. Evaluator names and native parsers remain outside the engine; a private evaluator crosses the same JSON, CLI, or Python SDK boundary.

Run request

format_version: invarlock/evaluation-request-v1
comparison:
  baseline:
    artifact:
      model_id: acme/baseline
      locator: registry://acme/baseline@immutable-revision
      path: models/baseline
    runtime:
      provider: hf_transformers
      settings:
        immutable_revision: 0123456789abcdef0123456789abcdef01234567
        checkpoint_tree_sha256: "1111111111111111111111111111111111111111111111111111111111111111"
        tokenizer_metadata_sha256: "3333333333333333333333333333333333333333333333333333333333333333"
        batch_size: 1
        context_length: 2048
        max_output_tokens: 64
        offline: true
        seed: 7
        timeout_seconds: 120
  subject:
    artifact:
      model_id: acme/subject
      locator: registry://acme/subject@immutable-revision
      path: models/subject
    runtime:
      provider: hf_transformers
      settings:
        immutable_revision: fedcba9876543210fedcba9876543210fedcba98
        checkpoint_tree_sha256: "2222222222222222222222222222222222222222222222222222222222222222"
        tokenizer_metadata_sha256: "3333333333333333333333333333333333333333333333333333333333333333"
        batch_size: 1
        context_length: 2048
        max_output_tokens: 64
        offline: true
        seed: 7
        timeout_seconds: 120
  dataset:
    path: inputs/release-regression.jsonl
    sha256: "4444444444444444444444444444444444444444444444444444444444444444"
    format: jsonl
    name: release-regression
    split: validation
    input_field: prompt
    expected_output_field: expected
    id_field: case_id
    limit: 400
  policy: policy/acceptance.json
  task: text_causal
  metric: normalized_nll_per_utf8_byte
execution:
  mode: run
output:
  evidence: artifacts/evidence

The digest values are illustrative. A real request must bind the actual source, artifact, and tokenizer bytes. Provider identity digests in runtime.settings and comparison.dataset.sha256 use bare lowercase 64-character values; general bundle and runtime identities use the sha256: prefix where their contracts require it.

For run mode, the transaction verifies the JSONL digest, preserves source order, maps the declared top-level text fields, selects the exact prefix named by limit, derives stable IDs or deterministic position IDs, and builds the canonical schedule. Blank lines, invalid JSON objects, missing mapped text, duplicate IDs, digest mismatch, or a limit larger than the source fail closed.

Import request

Import mode replaces provider execution with authenticated sidecars. In addition to mode, it requires records, schedule, and these six references for both baseline and subject:

execution:
  mode: import
  records: import/paired-records.json
  schedule: inputs/schedule.json
  baseline:
    identity: import/baseline/model-artifact.identity.json
    receipt: import/baseline/runtime-provider.receipt.json
    observation: import/baseline/runtime-scoring.observation.json
    run_report: import/baseline/report.json
    runtime_manifest: import/baseline/runtime.manifest.json
    runtime_config: import/baseline/run.yaml
  subject:
    identity: import/subject/model-artifact.identity.json
    receipt: import/subject/runtime-provider.receipt.json
    observation: import/subject/runtime-scoring.observation.json
    run_report: import/subject/report.json
    runtime_manifest: import/subject/runtime.manifest.json
    runtime_config: import/subject/run.yaml

This fragment is inserted under the same request root as the comparison and output objects. In import mode, comparison.dataset and execution.schedule both name the canonical schedule. Evaluation re-derives artifact identities and paired records; supplying a valid-looking aggregate is not sufficient.

Parser and path limits

These are native v1 request limits. Captured comparisons use the separate capacity limits; judge recipes and measurements use judge operational bounds.

The YAML loader accepts only a JSON-compatible mapping. It rejects aliases, anchors, tags, directives, merge keys, duplicate keys, include-like keys, unsafe scalar types, and non-canonical scalar spellings. File reads repeat component-by-component no-follow checks at use time, so a path that passed the first parse cannot be replaced with a symbolic link unnoticed.

The built-in deterministic native metrics are exact_match and normalized_nll_per_utf8_byte; native judge uses the separately described judge analysis policy. Exact-match reports include paired outcome counts, an exact two-sided McNemar probability, and a versioned paired Newcombe 95% interval whose lower bound controls policy. Current v3 reports and historical v2 reports use the continuity-corrected method; strict verification preserves the original method for legacy v1 reports. Normalized-NLL reports include the fixed 2,048-replicate paired_percentile_bootstrap_sha256_v1 interval over the authenticated schedule and apply the policy ceiling to its upper bound. A scorer-extension comparison uses the same fixed paired-resampling method over unit-interval record values and applies delta_min_pp to the lower bound. Decision semantics defines the exact arithmetic.

Every metric policy contains its threshold and may also contain the coupled minimum_record_count and maximum-width fields. Exact match and scorer extensions use maximum_interval_width_pp; normalized NLL uses maximum_interval_width_ratio. Exact match may independently add minimum_side_accuracy, a finite value from 0 through 1 that both side means must meet. A v3 report passes only when the metric-bound check and every configured sample-qualification and side-accuracy check pass. Historical v2 reports have no side-accuracy section and retain their original semantics.

Provider document field map

The schemas remain authoritative for types, patterns, nullability, and nested conditional rules. This map makes every top-level contract field discoverable:

Nested provider-receipt groups are closed:

Each scoring record requires record_id, input_sha256, and status. An ok record contains output text plus its digest, log-likelihood facts, or both. An error record contains only a canonical error_code and no measured facts. Log-likelihood facts are finite logprob_sum, positive token_count, and positive utf8_byte_count. Byte-normalized NLL verifies the byte count against the scheduled target. Matching authenticated tokenizer metadata and equal positive paired token counts allow a verifier-derived perplexity interpretation without adding a metric or policy surface.

The runtime manifest fixes execution_mode to container. Its outer-container object binds the image reference/digest and observed execution switches; its runtime-provider object binds the provider name, ABI, and sibling identity, observation, and receipt files. report and config bind their sibling files by digest. Unknown fields at any of these levels are rejected.

Closed formats without standalone schemas

Several exact shapes are enforced by code and verifier replay rather than a separate JSON Schema file:

These are documented for inspection and interchange, not as permission to construct partial objects. Use evaluate to produce bundles, verify to produce receipts, and the stable Python verify_signed_verification_receipt facade to validate a received receipt.

Canonical JSON and digests

Canonical JSON uses UTF-8, sorted object keys, compact separators, and finite numbers. Newline behavior is contract-specific: most bundle JSON documents and signed receipt statements use one trailing line feed, while canonical schedule, artifact-identity, scoring-observation, and embedded substructure hash inputs use the same representation without a final line feed. Writers and readers must use the contract serializer rather than normalize whitespace themselves.

The base JSON syntax follows RFC 8259. InvarLock deliberately narrows it with unique keys, finite numbers, closed schemas, exact encodings, and semantic cross-replay.

Bundle-level digests use lowercase sha256:<64 hex>. Some provider ABI fields and the checksum-file digest use a bare lowercase 64-character SHA-256 value; their schemas identify that distinction. Signatures use Ed25519. A key fingerprint is sha256: followed by the SHA-256 of the raw Ed25519 public key. Ed25519 is specified by RFC 8032.

Digest and signature dependency chain

Diagram

Contract bindings connect request fields to canonical artifacts, signature domains, and the verifier checks that consume them.

A later layer authenticates references to earlier layers; it does not replace their semantic checks. In particular, a valid evidence signature proves that one key signed the manifest, not that artifact identities, scores, policy, or runtime declarations are true. Independent verification supplies and checks those acceptance anchors.

Semantic validation

Schema validity is only the first layer. The verifier also recomputes:

• fixed path and closed-inventory rules;
• canonical bytes and file digests;
• artifact, schedule, runtime, observation, and request cross-bindings;
• record order, input digests, and observation-record digests;
• exact-match or normalized-NLL scores, or explicitly authorized scorer- extension results replayed twice from authenticated text facts;
• paired exact-match counts, exact McNemar probability, and the report-version-specific Newcombe interval, or deterministic paired replicates and interval endpoints for normalized NLL and scorer-extension deltas;
• derived perplexity facts when tokenizer and paired token counts are comparable;
• comparison means, threshold arithmetic, optional count/interval-width and exact-match side-accuracy qualification, and policy verdict; and
• evidence signer and verifier signature bindings.

A custom reader that performs schema validation alone is not equivalent to invarlock verify.

Version and change discipline

A format identifier names one exact shape and interpretation. Additive fields are not silently accepted by closed objects. A breaking artifact change requires a new format identifier and explicit reader behavior. Runtime providers must also match ABI 1 exactly.

The v3 comparison report is the current writer format. Strict verification continues to reconstruct v2 and v1 reports under their original shapes and arithmetic when signed historical packs identify those formats; it never relabels a reconstructed report as another version.

Receipt v1 remains valid for existing evidence whose runtime does not require a request-level executable binding. A supplied request digest selects receipt v2. GGUF verification requires that external request anchor because the normalized request authorizes the exact llama.cpp binary, source, version, execution settings, and GGUF identity reconciled with provider evidence.

Core and first-party add-ins are released at the same package version. Provider add-ins declare the exact coordinated core release, while the provider ABI remains the runtime compatibility gate. See Release verification.

Related documentation

• Evidence artifacts maps contract roles to fixed bundle paths.
• Runtime providers defines the typed ABI that produces provider contract objects.
• Reports and receipts explains the decision and independent verification objects.
• Release verification separates package versioning from evidence-format and provider-ABI compatibility.