Command-line interface

The public command line supports one evaluation, verification and reporting journey for native execution, authenticated imports, and captured results:

Reference

Surface: invarlock evaluate, invarlock verify, and invarlock report

Stability: Stable public CLI; command help is authoritative for installed options

Use this page when: Automating a transaction, selecting flags or environment fallbacks, or interpreting outputs and exit status

invarlock --help
invarlock evaluate request.yaml
invarlock verify evidence/
invarlock report evidence/

Use invarlock --version for the installed version and invarlock --help for the authoritative option list. The core commands have the same transaction boundaries as the Python facade.

The deterministic captured-results flow compares existing records without inference while retaining an explicit captured-evidence and verifier-receipt scope. See the captured-results guide.

External evaluator qualification uses a separate companion executable so the three-command release transaction remains unchanged:

invarlock-qualify-evaluator qualify PROFILE SCHEDULE EXPORT RAW_OUTPUT \
  [--output RESULT] [--require-verdict-authority] [--json]

This command is evaluator-neutral: it never executes or dispatches to a named evaluator. It authenticates one already-normalized export and independently recomputes the supported deterministic metric declared by its profile. See Evaluator qualification.

Diagram

The evaluate command creates signed evidence, verify independently replays it and signs a technical receipt, and report renders authenticated evidence without changing either authority record.

Root command

Shell completion is deliberately not installed by the CLI. The root command also applies the process security defaults used by every transaction before a command implementation imports a runtime backend.

evaluate

invarlock evaluate REQUEST \
  --signing-key PATH \
  [--allow-installed-scorers] \
  [--runtime-profile FILE] \
  [--runtime-image IMAGE] \
  [--runtime-image-digest sha256:...] \
  [--baseline-runtime-image IMAGE] \
  [--baseline-runtime-image-digest sha256:...] \
  [--subject-runtime-image IMAGE] \
  [--subject-runtime-image-digest sha256:...] \
  [--container-engine docker|podman] \
  [--runtime-device cpu|cuda|cuda:<index>] \
  [--baseline-runtime-device DEVICE] \
  [--subject-runtime-device DEVICE] \
  [--runtime-entrypoint auto|python|nvidia] \
  [--baseline-runtime-entrypoint PROFILE] \
  [--subject-runtime-entrypoint PROFILE] \
  [--runtime-cpus DECIMAL] \
  [--runtime-memory-mib INTEGER] \
  [--runtime-user UID:GID] \
  [--preflight] \
  [--json]

For native exact-match/NLL and deterministic-extension run/import requests, evaluate loads one closed YAML request and runs the complete execution-free preflight before any worker starts. It then prepares or validates the canonical schedule, executes or imports paired runtime records, derives the selected metric and its paired interval, applies the policy to the conservative bound, applies any coupled count and width controls and exact-match side-accuracy floor, and atomically publishes the evidence directory named by the request.

--preflight returns after that mandatory validation instead of continuing to execution and publication. It validates the request and every referenced path, prepares or authenticates the schedule, checks the policy shape, reproduces both declared artifact identities, checks provider task/metric capabilities, loads the evidence-signing key, and checks the output destination without creating it. In run mode it also confirms that the selected Docker or Podman executable and both digest-pinned images are already available locally. It resolves the same frozen, caller-owned per-side provider resources used by execution. Providers with the optional input hook then authenticate every selected schedule-bound external object before a worker or model starts. The vision-text add-in validates the content ID, regular no-follow file, length, SHA-256, media type, safe decode, frame count, dimensions, and aggregate media limits. When a scorer extension is selected, preflight loads the explicitly authorized scorer descriptor and configuration schema, then checks the exact ABI, version, descriptor, configuration, task, input-kind, and output-kind binding without replaying records. It does not pull an image, start a container, load a processor or model, initialize CUDA, run inference, create an output directory, or sign evidence. A successful preflight cannot prevent later file replacement, so each worker reopens and authenticates the same content before model preparation and the provider checks it again when scoring.

The machine-readable success and failure format is invarlock/evaluation-preflight-v2. If the policy supplies coupled sample qualification fields, preflight records the observed schedule count and checks its minimum. It records the maximum interval width with status pending_execution, because no interval exists before paired execution.

For a valid run request, the host prepares the canonical schedule and launches one independently configured Docker or Podman worker for each comparison side. Each worker loads only its side's artifact and support resources, scores the same authenticated schedule, and returns a closed six-file side result. The host validates both results, derives the comparison, signs the manifest, and publishes the evidence directory. Import requests do not launch workers.

Runtime image, device, entrypoint and resource controls apply only to run-mode requests. Import evidence already records its runtime identity, so evaluate rejects explicit run controls for an import request instead of silently ignoring them. The command also fails if the request's execution mode changes between mode detection and full loading.

The five shipped deterministic scorer IDs are available without --allow-installed-scorers; see scorer extensions. The flag enables discovery of separately installed scorer code, which must still match the request and policy bindings.

Native judge scoring

A native request can select comparison.metric: judge with its own rubric, analysis policy and private collection workspace. evaluate captures or imports the native answers and calls the optional installed collector, then publishes judge evidence. Preflight makes no calls; interrupted judging retains its checkpoint and leaves the final output absent for continuation. Native runtime resources and artifact authentication still apply. verify replays the native capture and judgments offline using a judge recipient policy; report displays model, runtime, rubric, judge identity, outcomes and uncertainty. See the judge reference for the policy and supported profile.

Captured evaluation controls

Captured requests use invarlock/evaluation-request-v2 with execution.mode: captured. Use --signing-key (or INVARLOCK_SIGNING_KEY) for handoff, or explicit --unsigned for local reporting. The latter rejects an explicit key. --baseline-run, --subject-run, and --output are caller-relative path overrides confined to the request root; they do not change source identity pins. --max-bootstrap-draws controls the caller-owned captured work allowance (default 102,400,000). Runtime, container, and scorer-extension flags are rejected for this mode. Native resource limits are unchanged.

--init DIRECTORY --example classification|extraction|judge|native-judge, --keygen DIRECTORY, and --freeze-cases FILE are mutually exclusive setup actions on evaluate, without a request argument. --case-set-output FILE optionally writes the canonical case set. These actions emit invarlock/evaluation-setup-v1 with --json and do not evaluate or establish assurance.

invarlock evaluate --freeze-cases cases.json --case-set-output frozen-cases.json --json

When comparison.metric: judge is selected, captured requests use the judge recipe, private workspace and optional retained measurements. Preflight and evaluation return judge result envelopes; --max-bootstrap-draws does not apply. Without retained measurements, judging uses the installed collector under the recipe's explicit budgets. Verification uses a judge recipient policy.

Deterministic captured preflight emits invarlock/evaluation-preflight-v3; captured publication emits invarlock/evaluation-result-v2. The captured-results guide gives the request, complete trust profile, and output fields. --fail-on-policy applies after either native or captured publication, retaining its success JSON: passing decisions exit 0, adverse decisions exit 7, and unknown/unavailable decisions exit 2. It cannot be combined with preflight or setup actions.

Reusable runtime profiles

Use --runtime-profile runtime.json to reuse local runtime settings without putting host resources into the signed request schema. Profiles are explicit: there is no automatic file discovery. Import requests reject this option.

{
  "format": "invarlock/runtime-profile-v1",
  "runtime": {
    "engine": "docker",
    "image": "registry.example/runtime@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
    "device": "cpu",
    "entrypoint": "auto",
    "cpus": "4",
    "memory_mib": 65536,
    "user": "65532:65532"
  },
  "subject": {
    "device": "cuda:0"
  }
}

format and the runtime object are required. baseline and subject are optional objects accepting only image, image_digest, device, and entrypoint. The common runtime object also accepts engine, cpus, memory_mib, and user. Every value is a nonempty string except memory_mib, which must be a positive JSON integer; cpus is a decimal string. Existing OCI validation still enforces pinned local images, supported devices and entrypoints, resource limits, and a non-root user. Profiles cannot authorize network access, signing keys, or installed scorers.

For each side's field, profile mode uses this precedence:

1. Explicit side-specific command-line option.
2. Explicit common command-line option.
3. Side-specific profile value.
4. Common profile value.
5. Side-specific environment variable.
6. Common environment variable.
7. The existing default.

Engine and resource fields use the applicable common steps. Without a profile, existing option and environment resolution stays unchanged. Image and digest are resolved independently: if an overridden image embeds a digest that conflicts with the selected separate digest, the command fails and asks you to update the matching value. It never silently repairs the mismatch.

Profiles must be regular, non-symlink files of at most 16 KiB. This is a configuration-file limit. Unknown fields, duplicate JSON keys, non-finite numbers, and invalid value types are rejected. Run evaluate REQUEST --runtime-profile runtime.json --preflight to validate the complete effective launch before execution. Preflight text output shows the profile SHA-256, resolved side settings, resource limits and each setting's origin. Existing preflight JSON and signed request/evidence formats do not change.

The signing key must be a real regular file. The request and every referenced input must remain beneath the request root. Keep the signing key in a separate, caller-controlled location. It remains in the host process and is never mounted into either worker. The output destination must not already exist.

Run-mode comparison.dataset is a digest-pinned local JSONL object. The transaction verifies the source bytes and deterministically prepares the canonical ordered schedule. Import-mode comparison.dataset is the canonical schedule path and must match execution.schedule.

Each OCI worker uses its selected local image only (--pull=never), disables networking, uses a read-only container root, drops capabilities, enables no-new-privileges, bounds process count, and receives a temporary filesystem. The launcher also applies the caller-owned CPU and memory ceilings and numeric non-root identity above. Submitted request data cannot relax these host controls. The job description, canonical schedule, side artifact, and closed support resources are mounted read-only. Only an isolated side-output directory is writable. The worker never receives the other side's artifact or the signing key.

Each provider's validated timeout_seconds setting remains a per-record deadline. The host derives an additional worker-process deadline from that value and the authenticated schedule size, including two timeout intervals for startup and cleanup, with a final 24-hour cap. On expiry it stops the container by its engine-issued ID, escalates to a kill when necessary, and reports a failed side instead of waiting indefinitely.

CPU workers add no GPU mapping. --runtime-device cuda or a per-side CUDA override exposes the selected GPU to that worker; it does not add CUDA support to a CPU-only image. Use an image built from runtime/Dockerfile.cuda for the canonical x86_64 CUDA Hugging Face runtime. Two CPU workers may run in parallel. Two workers on distinct explicit indexes such as cuda:0 and cuda:1 may also run in parallel. Generic CUDA selection, a shared explicit index, or a CPU/CUDA pair runs sequentially so two workers do not contend for one GPU.

The auto entrypoint profile selects nvidia for TensorRT-LLM and python for the other first-party providers. Explicit per-side profiles are available when an authenticated image requires one of those known launch forms.

For native pack-v1 evaluation, text success output identifies the published directory. JSON success output contains format_version, ok, comparison_id, evidence, and the immutable pack_manifest_digest calculated from the canonical manifest bytes before publication. JSON failures use the same format version with ok: false and an errors array.

{
  "comparison_id": "cmp-...",
  "evidence": "artifacts/evidence",
  "format_version": "invarlock/evaluation-result-v1",
  "ok": true,
  "pack_manifest_digest": "sha256:..."
}

The displayed evidence path is the host-side destination from the request. Treat it as an opaque location. The manifest digest, rather than the path name, binds automation to the exact pack produced by this evaluation; the signed manifest carries the comparison identity.

Preflight success reports the mode, relative output destination, schedule and policy digests, record count, providers, completed check classes, and run-mode runtime-image digests. runtime_resources appears in checks when both sides were resolved against the inspected image digests; optional provider hooks also authenticate schedule-bound resources. Preflight removes predictable configuration failures; it cannot prove that a backend will initialize, that inference will finish, or that the comparison will pass policy. Those claims require the real evaluation and independently verified evidence.

verify

invarlock verify EVIDENCE \
  --trust-profile PATH \
  --receipt verification.receipt.json \
  [--json]

Native exact-match/NLL and deterministic-extension evidence uses one closed invarlock/trust-inputs-v1 object. Native judge evidence instead requires the judge recipient policy:

{
  "allow_installed_scorers": false,
  "anchors": {
    "baseline_artifact_digest": "sha256:...",
    "baseline_runtime_digest": "sha256:...",
    "evidence_signer_fingerprint": "sha256:...",
    "request_digest": "sha256:...",
    "schedule_digest": "sha256:...",
    "subject_artifact_digest": "sha256:...",
    "subject_runtime_digest": "sha256:..."
  },
  "format": "invarlock/trust-inputs-v1",
  "policy": {"path": "acceptance.json"},
  "verifier": {
    "identity": "release-verifier",
    "signing_key_path": "verifier.pem"
  }
}

Policy and key paths resolve relative to the profile. The loader rejects absolute paths, traversal, symlinks, duplicate members, unknown fields, missing files, and malformed anchors. Environment variables do not override a profile. Supplying any explicit trust-anchor option with --trust-profile is an error. The profile, its policy, and its verifier key must remain outside the submitted evidence directory. The canonical profile digest is included in the signed receipt.

Captured evidence instead requires a kind: captured invarlock/trust-inputs-v2 profile with baseline_run_digest, subject_run_digest, request_digest, and evidence_signer_fingerprint anchors, plus policy and verifier fields. It has no runtime anchors or installed-scorer authorization. See the complete captured profile. Without a profile, use --expected-baseline-run and --expected-subject-run (environment alternatives INVARLOCK_EXPECTED_BASELINE_RUN and INVARLOCK_EXPECTED_SUBJECT_RUN) along with policy, request, signer and verifier options. The request digest is mandatory for captured evidence. Native and captured anchor families cannot be mixed. A recipient may independently set --max-bootstrap-draws, also with a profile; it is local work control, not trust.

For native evidence, request_digest is optional for existing non-GGUF evidence and required when either request side selects llama_cpp. Record it from the execution-free evaluate --preflight --json result after reviewing the normalized request.

For systems that already keep each anchor separately, the equivalent explicit form remains available:

invarlock verify EVIDENCE \
  --policy PATH \
  --expected-baseline-artifact sha256:... \
  --expected-subject-artifact sha256:... \
  --expected-schedule sha256:... \
  --expected-baseline-runtime sha256:... \
  --expected-subject-runtime sha256:... \
  --expected-signer sha256:... \
  [--expected-request-digest sha256:...] \
  --receipt verification.receipt.json \
  --verifier-signing-key verifier.pem \
  --verifier-identity release-verifier \
  [--allow-installed-scorers] \
  [--json]

Native and captured directory-pack verification treats the bundle as untrusted, requires all acceptance anchors from the caller, and writes a signed receipt outside the bundle. Judge verification has an optional signed receipt, as described under Frozen-answer judge evidence.

The following table describes native pack-v1 verification. With a trust profile, the profile supplies the trust fields; the explicit alternatives are not also required. Judge and evidence-set inputs use their separate policies.

Use either --trust-profile or all explicit trust-anchor options, never both. --receipt has no environment alternative. The destination must be outside the evidence directory and must not already exist.

Verification checks the closed manifest, evidence signature, checksums and inventory; caller-approved artifact and schedule identities; all input and evidence references; the normalized request and schedule; runtime manifests and provider sidecars; for GGUF, the caller-approved request digest and its artifact/backend/execution bindings; record ordering and input digests; derived paired scores; the canonical comparison report; and the policy verdict. A policy verdict includes any configured record-count, interval-width, and exact-match side-accuracy qualification. A verdict of fail is a valid, integrity-checked result but is not command success.

An installed scorer is executable code. The flag does not authorize an arbitrary scorer: discovery still requires the exact scorer ID, ABI, version, descriptor digest, configuration digest, task, and policy pins bound by the transaction. Evaluation and independent verification must authorize and load the same scorer identity separately.

For native evidence, --json emits invarlock/evidence-pack-verify-v1, the signed-receipt path, verifier identity, and pack_manifest_digest. That digest is the same immutable manifest identity signed into the receipt. Exit status 0 means the evidence passed both integrity and policy verification. Any nonzero status must be treated as rejection.

The result distinguishes integrity from acceptance. For example, a correctly signed and internally consistent pack can have integrity_ok: true, policy_verdict: "fail", and ok: false. See Reports and receipts for the complete field matrix.

report

invarlock report EVIDENCE [--html report.html] [--markdown report.md] [--junit results.xml] [--explain] [--case-id ID]... [--json]

For native and captured directory packs, report checks the closed inventory, checksums, reference digests, canonical JSON and embedded signature when signed, then renders reports/evaluation.report.json without a full comparison replay. Judge and evidence-set reports replay their retained judge measurements under their own contracts. None of these paths performs recipient acceptance.

Scroll horizontally to see every column.

Every output option refuses to overwrite an existing file. By default, report emits the text view to standard output and prints the written path when HTML is requested. With --json, it instead emits one compact invarlock/evidence-report-v1 object containing ok, the pack-manifest digest, and html (a path or null) for native default/HTML-only calls. Captured calls, and native calls requesting Markdown or JUnit, emit invarlock/evidence-report-v2 with kind, ok, pack_manifest_digest, requested_outputs, written_outputs, failed_output, and errors. Judge reports emit invarlock/judge-evidence-report-v1 with kind: judge and evidence_digest; evidence-set reports emit invarlock/evidence-set-report-v1 with kind: evidence_set and index_sha256. Both include output maps and their scoped report facts. These formats do not reuse the native manifest identity field.

All destinations are checked up front; a later write failure leaves earlier completed outputs accurately listed. There is no automatic receipt discovery. Unsigned captured reports remain explicitly local, without independent assurance.

Judge evidence and evidence sets show the first 50 case IDs by default. Repeat --case-id to render any specific retained cases in HTML or in the expanded text/Markdown view produced by --explain. Selection changes presentation only; the report still replays the complete retained measurement set. Other evidence formats reject this option.

Captured text, HTML and Markdown reports include the baseline and subject run IDs, complete-run digests, attributed artifact digests and evaluator source identities in their technical details. The comparison context also shows the recorded model, workflow, dataset and prompt metadata when supplied. These details identify what was compared without changing the report's assurance.

Hosted runs use execution.mode: captured, artifact_digest: null and explicit service identity. Their complete-run pins cover the service configuration, harness and observation window. They do not use native artifact or runtime anchors. Captured evaluate consumes supplied service records; it does not execute a hosted service. Offline verification does not refresh that observation. See hosted requalification.

Reporting does not accept independent artifact, schedule, policy, runtime, or signer anchors and does not issue a verification receipt. It is therefore a safe renderer, not a substitute for invarlock verify.

Text and JSON output

Core text output separates operation completion, recorded policy result and independent verification. evaluate prints Evidence created and the recorded policy result. A published policy failure exits 0 unless --fail-on-policy requests a nonzero policy gate; publication does not establish recipient acceptance. Preflight shows the mode, paired record count, destination and number of validated checks, without execution or publication.

verify distinguishes an authentic policy rejection from evidence integrity or input failures. On policy rejection, it shows failed checks with observed and required values and the signed receipt location. Diagnostic text is rendered literally, including brackets in signer fingerprints and paths.

report renders Markdown as terminal content and can write self-contained HTML. Its --json mode emits a rendering result object on success or an application error object with ok: false and errors on failure. Native evaluate and verify JSON contracts are unchanged; captured evaluation and verification use v2 result formats. CLI syntax, option and path-validation failures that occur before the command handler still use usage diagnostics on stderr and exit 2; --json does not convert those parser errors into result objects.

Runtime resources and explicit trust anchors are grouped separately in command help. Grouping changes presentation only; flag defaults and environment-variable resolution remain unchanged.

Exit and write behavior

All three commands fail closed. A nonzero exit indicates that the requested transaction did not succeed. Evaluation does not publish a partial destination; verification does not write inside the pack; and reporting does not mutate pack bytes. Output files and directories are no-clobber by design.

Do not build automation that accepts a particular nonzero value. The lower-level EvidencePackStatus enum has finer internal categories, while the public transaction may normalize a precondition failure to 2. Automation should require status 0, the expected output, and ok: true where JSON is available.

Automation pattern

Use a fresh output name for every immutable transaction and capture JSON on a dedicated stream:

invarlock evaluate request.yaml \
  --signing-key evidence-signer.pem \
  --runtime-image registry.example/invarlock-runtime-cuda@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa \
  --runtime-image-digest sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa \
  --container-engine docker \
  --runtime-device cuda \
  --json
invarlock verify artifacts/evidence \
  --policy policy/acceptance.json \
  --expected-baseline-artifact sha256:... \
  --expected-subject-artifact sha256:... \
  --expected-schedule sha256:... \
  --expected-baseline-runtime sha256:... \
  --expected-subject-runtime sha256:... \
  --expected-signer sha256:... \
  --receipt receipts/verification.json \
  --verifier-signing-key verifier.pem \
  --verifier-identity release-verifier \
  --json

The aaaaaaaa... value above is illustrative and must be replaced. Ellipses in the verification command are placeholders, not valid digests. Fingerprints and all verifier anchors use sha256: plus 64 lowercase hexadecimal characters. Artifact anchors identify the canonical provider artifact-identity bytes, not a path or mutable model name. The schedule anchor identifies the exact canonical schedule bytes. Obtain both through a verifier-controlled handoff, not from the submitted evidence pack. Provider settings and local JSONL source hashes use bare lowercase 64-character values where their schemas require them. Keep the receipt and optional HTML report outside the immutable evidence directory.

See Environment variables for the complete small CLI and runtime inventory.

Related documentation

• Environment variables lists supported secret, anchor, and runtime inputs.
• Python API exposes the same transactions to embedding applications.
• Evaluation lifecycle explains write boundaries and retry behavior.
• Reports and receipts defines JSON results and formatted reports.

Frozen-answer judge evidence

The versioned judge request uses the same evaluate, verify and report commands. See judge measurements for import preflight, bounded reports and independent recipient policy verification. Judge verification returns an unsigned local result. --receipt requires both --verifier-signing-key and --verifier-identity and writes a separately signed judge receipt outside the evidence. Its JSON ok field is true only when accepted is true.

Deterministic and judge evidence sets

Use the existing evaluate requests for each component, then index the two packs and run verify with an independent composition recipient policy. report shows their metrics together. Both components must bind the same original runs and case set; their statistical methods retain separate meanings. See evidence sets for commands, policies and exit codes.