CLI Guide

📌 The exhaustive command/flag reference is generated from the code: cli-reference.md — rendered from the clap definitions (the same source as --help), so it cannot drift and needs no manual verification. This page is the guide: the same commands with worked examples, output samples, and the why. For the guaranteed-current flag list of any command, trust the generated reference (or run rivet <command> --help).

Machine-readable output. Most commands emit JSON via a boolean --json flag (run, check, doctor, metrics, state …); validate (and the reconcile / plan report writers) instead take --format json. This split is a known inconsistency to be unified in a future release — until then, pass --help to confirm which idiom a given command uses.

Global

rivet [--json-errors] [COMMAND] [OPTIONS]

rivet --version       # print version
rivet --help          # show help

rivet --json-errors run --config rivet.yaml
rivet run --config rivet.yaml --json-errors   # global flag accepted in any position

rivet run

Run export jobs defined in a config file.

rivet run --config <PATH> [OPTIONS]

Examples

# Basic run
rivet run -c my_export.yaml

# Run with validation and reconciliation
rivet run -c my_export.yaml --validate --reconcile

# Run a single export
rivet run -c my_export.yaml -e orders_daily

# Resume interrupted chunked export
rivet run -c my_export.yaml -e big_table --resume

# Parameterized query
rivet run -c my_export.yaml -p region=us-east -p year=2026

# Parallel exports (all at once)
rivet run -c my_export.yaml --parallel-exports

# Parallel exports — one OS process per export, parent-side cards UI
rivet run -c my_export.yaml --parallel-export-processes

--parallel-export-processes — one card per export

--parallel-exports runs the exports in the same Rivet process on up to 16 worker threads. That keeps logs simple, but every export shares the same source connection pool / global allocator. A panic in one export is caught and reported as that export’s failure while the others finish (release builds unwind; the release-min profile aborts instead).

--parallel-export-processes instead spawns one rivet child process per export — full memory and connection isolation, no shared allocator. The parent process owns the screen and renders one card per export with a live progress bar, ETA, row count, and elapsed time. When a child finishes, the progress bar is replaced in place with the export’s final metrics, so the on-screen card becomes a self-contained per-export summary; below the cards a single aggregated Run summary block prints once for the whole run.

▸ orders            chunked   11/20 chunks   1.1M rows  8.7K r/s   2m 06.0s  ETA 1m 43.1s

One card line per export (▸ running, ✓ finished), redrawn in place; the children’s verbose per-export output goes to a timestamped log file beside the config.

Children emit structured NDJSON events (Started, ProgressInit, Progress, Finished) on stdout via the RIVET_IPC_EVENTS=1 env var; the parent multiplexes them into the cards UI. If a child crashes without a Finished event, its card is marked failed with a synthetic warning so a silent crash never leaves the run looking healthy.

rivet cdc

Stream log-based change data capture directly (without a config). The engine is chosen from the URL scheme — mysql:// (binlog) / postgresql:// (logical slot) / sqlserver:// (change tables) / mongodb:// (change stream). Emits NDJSON to stdout by default, or typed Parquet/CSV with --output (--output requires exactly one --table — the schema is resolved from the source); --checkpoint persists a resume position.

rivet cdc --source-env DATABASE_URL --table orders                 # NDJSON to stdout
rivet cdc --source-env DATABASE_URL --table orders --output ./cdc --format parquet --checkpoint ./o.ckpt

The full reference — per-engine prerequisites, --slot / --capture-instance / --server-id, --stream (opt into continuous; bounded is the default), the config-driven rivet run + mode: cdc path (the fuller path, all four engines incl. MongoDB), and the failure/recovery playbook — is in cdc.md.

rivet plan

Generate a sealed execution plan artifact — no data is exported.

rivet plan runs preflight analysis (row estimate, index check, sparsity), computes chunk boundaries for chunked exports, snapshots the current cursor for incremental exports, and writes everything to a PlanArtifact JSON file. The artifact can be reviewed, committed, stored as a CI artifact, or passed to rivet apply.

rivet plan --config <PATH> [OPTIONS]

Examples

# Human-readable summary (no file written)
rivet plan -c rivet.yaml

# Write full JSON artifact to a file
rivet plan -c rivet.yaml --format json --output plan.json

# Plan a single export
rivet plan -c rivet.yaml -e orders --format json -o orders_plan.json

Pretty output (example)

  Plan ID  : a1b2c3d4e5f6...
  Created  : 2026-04-14 10:00:00 UTC
  Expires  : 2026-04-15 10:00:00 UTC
  Export   : orders
  Strategy : chunked
  Chunks   : 42
  Row est. : ~2,100,000
  Verdict  : Acceptable
  Profile  : balanced
  Warnings :
    • sparse id range: ~12% fill
  Resources:
    Batch size   :  10,000 rows
    Batch memory : ~2 MB (narrow) – ~95 MB (wide)
    RSS guard    : 4,096 MB
    Throttle     : 50 ms between batches
  Output   : local → ./out
  Format   : parquet + zstd

The Resources section shows:

Memory estimate methodology — advisory only

The memory estimate in rivet plan is a heuristic, not a guarantee. Treat it as a planning signal, not a hard prediction.

rivet plan does not sample the table. It computes the batch memory range using two fixed assumptions:

• Narrow bound — 200 B per row (all INTEGER / BIGINT / TIMESTAMPTZ columns)
• Wide bound — 10 KB per row (all TEXT / JSONB / BYTEA columns)

For most real tables the actual per-row size falls between these two bounds. The narrow bound is a reliable floor for numeric-heavy schemas; the wide bound is a reliable ceiling for text-heavy schemas.

What the estimate does not capture:

How to get a precise number: run rivet run once with RUST_LOG=info against a representative sample, then check the peak_rss in the logged summary or in rivet metrics. That measured value from your actual data is more reliable than any pre-run estimate.

Planned enhancement: a future rivet plan --sample N flag will query up to N rows to compute a data-driven row-width estimate. This will narrow the uncertainty for variable-width schemas without a full table scan.

Plan artifact structure

The JSON artifact (--format json) contains:

{
  "rivet_version": "0.18.0",
  "plan_id": "a1b2c3d4...",
  "created_at": "2026-04-14T10:00:00Z",
  "expires_at": "2026-04-15T10:00:00Z",
  "export_name": "orders",
  "strategy": "chunked",
  "plan_fingerprint": "0123456789abcdef",
  "resolved_plan": { ... },
  "computed": {
    "chunk_ranges": [[1, 50000], [50001, 100000], "..."],
    "chunk_count": 42,
    "cursor_snapshot": null,
    "row_estimate": 2100000
  },
  "diagnostics": {
    "verdict": "Acceptable",
    "warnings": ["sparse id range: ~12% fill"],
    "recommended_profile": "balanced"
  }
}

Security note: resolved_plan embeds the full source connection config including credentials. Treat plan files with the same care as your rivet config file.

rivet apply

Execute a sealed plan artifact, or run a config’s exports wave-by-wave. The mode is chosen by the path’s extension:

• .json → a sealed PlanArtifact: deserialize, validate staleness + cursor integrity, then execute the single export using the artifact’s pre-computed chunk boundaries — no SELECT min/max queries against the source.
• .yaml / .yml → a config: run every export wave by wave in ascending wave: order (the wave each export was assigned by rivet plan). See Wave-ordered execution below.

rivet apply <PLAN_FILE | CONFIG> [OPTIONS]

Staleness rules

Cursor drift (Incremental exports)

If another rivet run completed after the plan was generated, the cursor will have advanced. rivet apply detects this and rejects the artifact to prevent re-exporting already-exported rows. Regenerate with rivet plan.

Examples

# Apply the plan
rivet apply plan.json

# Apply an old plan (override staleness check)
rivet apply plan.json --force

What apply does NOT do

• Does not re-read the config file
• Does not re-run preflight queries
• Does not recompute chunk boundaries (uses pre-computed ranges from the artifact)
• Does not enforce preflight verdict (diagnostics are advisory — see ADR-0005)

State location

rivet apply opens .rivet_state.db next to the config file recorded inside the plan artifact (artifact.config_path), so apply shares the same state (cursors, manifests, schema history) as rivet run. It falls back to the plan file’s own directory — with a warning — only when the recorded config directory no longer exists or the artifact was generated before 0.7.5. Plan files do not need to sit beside the config.

Wave-ordered execution (YAML config)

rivet apply <config>.yaml runs every export in the config wave by wave, lowest wave: first, with a barrier between waves — every export in wave 1 finishes before wave 2 starts. Exports with no wave: run last. rivet plan --annotate-waves writes the wave: and parallel_safe: fields onto each export (you can hand-edit them; apply respects your order). Plain rivet plan is read-only and leaves the config untouched.

Within-wave parallelism. With parallel_export_processes: true in the config (or rivet apply --parallel-export-processes), the cheap exports within a wave — those rivet plan marked parallel_safe: true (cost class Low, < ~100K rows) — run concurrently as separate processes. A heavier export already chunk-parallelizes its own ranges internally, so it runs alone in its wave; two large tables at once would multiply load on the source. Each child still self-throttles via the adaptive governor. Without the flag, every export runs sequentially. parallel_safe also respects the campaign’s isolate_on_source — a cheap export on a contended shared source still runs alone.

# plan assigns waves → you review/edit → apply executes them, lowest wave first
rivet plan  -c rivet.yaml                   # review the schedule (read-only)
rivet plan  -c rivet.yaml --annotate-waves  # write wave:/parallel_safe: into the config
rivet apply rivet.yaml

A failing export does not stop its wave-mates: failures are collected and the run exits non-zero with the most stop-worthy error (data-integrity > internal > refusal > schema-drift > retryable > generic).

Resuming after a partial failure. Re-run with rivet apply <config>.yaml --resume: exports a prior run already completed (their destination carries a _SUCCESS marker) are skipped, and an incomplete chunked export continues from its checkpoint — so recovering a run that failed mid-way does not redo the tables that already succeeded. Without --resume, a re-run re-exports everything.

partition_by exports are not expanded in this path yet — use rivet run for those.

rivet validate

Re-run manifest-aware verification against an existing destination — no extraction.

rivet validate --config <PATH> [OPTIONS]

The same M5/M6 checks rivet run --validate performs at end-of-run, exposed as a standalone command for between-run polling and triage. Reads manifest.json + _SUCCESS at the destination and head-checks every committed part for presence and recorded size_bytes. The source is not queried (use rivet reconcile for that). See ADR-0013 §“Subcommand carveouts” and ADR-0012 M5/M6.

By default validate resolves the destination prefix the same way run does ({date} becomes today’s UTC date). Use --date, --run-id, or --prefix to point at a prior run instead.

Exits non-zero when the manifest references a part that is missing or whose size does not match, and when the manifest records its last run as anything but success (RIVET_VERIFY_RUN_NOT_SUCCESSFUL, exit 1: a failed, interrupted or still-running export is not a completed dataset). A legacy prefix (no manifest) falls back to the M6 reduced-guarantee path and is labelled legacy_run: true.

Examples

# Verify today's run at the configured destination
rivet validate -c my_export.yaml

# Verify a prior run by id, JSON report to a file
rivet validate -c my_export.yaml --run-id orders_20260521T120000 --format json -o verdict.json   # -o is ignored unless --format json is set

rivet reconcile

Partition/window reconciliation — re-runs per-chunk COUNT(*) on the source and compares with the stored per-chunk row counts from the last run. Surfaces matches, mismatches, and repair candidates without re-exporting data (Epic F).

rivet reconcile --config <PATH> --export <NAME> [OPTIONS]

Scope (v1)

• Chunked exports — supported. Requires a previous run with chunk_checkpoint: true so per-chunk ranges and row counts are persisted in .rivet_state.db.
• Time-window — returns an error (“use chunked with chunk_by_days” for partition reconcile).
• Snapshot / Incremental — no natural partitions; use rivet run --reconcile for a whole-export count check.

What it does

For each completed chunk task from the latest chunk run:

1. Rebuilds the exact chunk query the pipeline used (same WHERE predicate, same dense/range shape — build_chunk_query_sql).
2. Runs SELECT COUNT(*) FROM (<chunk_query>) AS _rc.
3. Compares the source count with the stored rows_written for that chunk.

Each partition is classified as:

• match — source and exported counts are equal.
• mismatch — counts differ; partition is a repair candidate (note includes diff).
• unknown — one of the counts is unavailable (chunk never completed, unparseable chunk keys); also a repair candidate.

Examples

# Human-readable summary
rivet reconcile -c my_export.yaml -e orders

# JSON report to file
rivet reconcile -c my_export.yaml -e orders --format json -o reconcile.json

Reports never re-export on their own — they surface what needs repair. They are not merely advisory though: a detected mismatch exits non-zero with the data-integrity class (exit 3), so CI can gate on it.

Verification strategy tradeoffs

Rivet has three verification mechanisms at different cost/precision tradeoffs:

Rule of thumb:

• Use --reconcile always for snapshot/incremental exports — cost is negligible.
• Use rivet reconcile for chunked exports if data correctness is critical or the source is volatile.
• Use rivet repair only when rivet reconcile surfaces mismatches — it re-exports only the flagged chunks.

rivet repair

Targeted repair of chunks flagged by reconcile. Prints a RepairPlan by default; with --execute, re-exports only the flagged chunk ranges (Epic H, ADR-0009 RR1–RR8).

rivet repair --config <PATH> --export <NAME> [OPTIONS]

Examples

# Dry run from the latest reconcile — prints the plan, nothing executes
rivet repair -c my_export.yaml -e orders

# Dry run from a saved reconcile report
rivet repair -c my_export.yaml -e orders --report reconcile.json

# Execute — re-runs only the flagged chunks
rivet repair -c my_export.yaml -e orders --report reconcile.json --execute

What --execute does and does not do

• Re-runs only the flagged chunk ranges via ChunkSource::Precomputed — same SQL shape as extraction and reconcile (RR3).
• Writes new files alongside originals named <export>_<ts>_chunk<idx>_<nonce>.<ext>, where <nonce> is a random 16-hex-digit value (RR5) — the nonce, not the second-granularity timestamp, is what guarantees a repair landing in the same second as the original can never clobber it. Rivet does not delete or overwrite prior files. Downstream deduplication (or a versioned output prefix) is the operator’s responsibility.
• Leaves last_committed_* untouched (RR4) — repair is corrective, not commitment. last_verified_* re-advances only if a subsequent clean rivet reconcile runs.

rivet check

Preflight analysis: diagnose source health, estimate row counts, check indexes, recommend tuning. With --type-report, also introspects column types and validates them against a target warehouse.

rivet check --config <PATH> [OPTIONS]

Examples

# Standard preflight check
rivet check -c my_export.yaml

# Type fidelity report (human-readable table)
rivet check -c my_export.yaml --type-report

# Type report with BigQuery compatibility column
rivet check -c my_export.yaml --type-report --target bigquery

# Type report as JSON — pipe-friendly, one object per export
rivet check -c my_export.yaml --type-report --json

# Strict mode — exits 1 if any lossy or unsupported mapping exists
rivet check -c my_export.yaml --type-report --strict

Type report output

Export: orders  [target: bigquery]

  Column        Source type        Rivet type       Arrow type            Fidelity        Target type   Status
  ----------    ----------------   ---------------  --------------------  --------------  -----------   ------
  id            int4               int4             Int32                 exact           INT64         ok
  amount        numeric(15,4)      decimal(15,4)    Decimal128(15, 4)     exact           NUMERIC       ok
  created_at    timestamptz        timestamp_tz     Timestamp(us, UTC)    exact           TIMESTAMP     ok
  metadata      jsonb              json             Utf8                  logical_string  STRING        ok ~
  tags          text[]             list<text>       List(Utf8)            exact           REPEATED…     ok

Fidelity levels:

Output includes: table existence, estimated row count, index analysis, tuning recommendation.

rivet doctor

Verify source and destination connectivity/auth before running exports.

rivet doctor --config <PATH>

Example

rivet doctor -c my_export.yaml

Output:

rivet doctor: verifying auth for config 'my_export.yaml'

[OK]  Config parsed successfully
[OK]  Source auth (Postgres)
[OK]  Destination Local(./output)

All checks passed.

When tls: is omitted from source: and the host is loopback, nothing is printed (local dev is exempt). On a remote host the [WARN] source: TLS is not enforced… line appears and the source check fails (TLS required — refusing to connect to a remote (non-loopback) host without TLS): fix it with tls: { mode: verify-full }, or explicitly opt into remote plaintext with tls: { mode: disable } on an already-trusted network path — see reference/config.md § TLS.

rivet init

Generate a YAML config scaffold (or a machine-readable discovery artifact) by connecting to PostgreSQL, MySQL, SQL Server, or MongoDB and introspecting tables/collections (read-only). Does not run an export. YAML scaffolds include meta_columns (exported_at / row_hash on by default); scaffolds with heuristic mode: chunked also include chunk_checkpoint: true — see init.md.

rivet init (--source <URL> | --source-env <ENV_VAR> | --source-file <PATH>)
           [--table <NAME>] [--schema <NAME>] [-o <PATH>] [--discover]

Exactly one of --source, --source-env, --source-file must be provided (enforced by the argument group).

Examples

# One table → one export block
rivet init --source-env DATABASE_URL --table orders -o rivet.yaml

# PostgreSQL: entire schema (default public)
rivet init --source-env DATABASE_URL --schema public -o all_public.yaml

# MySQL: entire database from URL path
rivet init --source-file /run/secrets/mysql_url -o all_mydb.yaml

# JSON discovery artifact — ranked cursor/chunk candidates per table
rivet init --source-env DATABASE_URL --schema public --discover -o discovery.json

Narrative guide, heuristics, and Docker Compose examples: init.md.

rivet metrics

Show export run history (duration, row count, file size, status).

rivet metrics --config <PATH> [OPTIONS]

Example

rivet metrics -c my_export.yaml --last 10
rivet metrics -c my_export.yaml -e orders_daily

rivet journal

Inspect the structured run journal for an export — per-run event log with status, file/row/byte summary, retries, quality issues, schema changes, and the first error line.

rivet journal --config <PATH> --export <NAME> [OPTIONS]

Examples

# Last 5 runs for the orders export
rivet journal -c my_export.yaml -e orders

# Last 10 runs
rivet journal -c my_export.yaml -e orders --last 10

# Single run by ID
rivet journal -c my_export.yaml -e orders --run-id orders_20260513T120000.123

Output

Each run is shown as a block:

✓ orders  success  12.3s
  run_id: orders_20260513T120000.123
  files:  3  rows: 150000  size: 4.2 MB

✓ = succeeded · ✗ = failed · • = partial / unknown. Retries, quality issues, schema changes, and first-line error text are appended when present.

Journal entries are persisted to .rivet_state.db (SQLite, migration v7) at the end of every run. An empty result means the export has not run yet in this state DB, or --run-id does not match any stored run.

rivet state

Manage export state (cursors, file manifests, chunk checkpoints).

rivet state show

Show current cursor state for all incremental exports.

rivet state show --config <PATH>

rivet state reset

Reset the cursor for a specific export (next run will re-export all rows).

rivet state reset --config <PATH> --export <NAME>

rivet state files

List files produced by exports.

rivet state files --config <PATH> [--export <NAME>] [--last <N>]

rivet state chunks

Show chunk checkpoint status for a chunked export.

rivet state chunks --config <PATH> --export <NAME>

rivet state reset-chunks

Clear persisted chunk checkpoint rows (chunk_run / chunk_task) so the next chunked run starts a fresh plan.

One export — same as targeting a single table name:

rivet state reset-chunks --config <PATH> --export <NAME>

Every “stuck” export in this config — resets checkpoints only when chunk_run.status is still 'in_progress' (process killed mid-run, concurrent worker left state behind, etc.). Exports whose chunk run already finished normally (completed) are skipped. Names that appear in state but were removed from the YAML are skipped with a printed note.

rivet state reset-chunks --config <PATH> --stuck-checkpoints

Alias (same semantics — checkpoint stuck, not “last metric row failed”):

rivet state reset-chunks --config <PATH> --failed

Then run rivet run --config <PATH> --resume (or a normal run without --resume) as needed.

rivet state progression

Show explicit committed and verified export boundaries (Epic G / ADR-0008).

rivet state progression --config <PATH> [--export <NAME>]

The progression table is advisory: it does not gate rivet run, rivet apply, or rivet reconcile. Consumers are operators and external monitoring.

rivet completions

Generate shell completion scripts.

rivet completions <SHELL>

rivet schema

Emit machine-readable schemas for Rivet’s data contracts.

rivet schema config

Today rivet schema config prints the JSON Schema for the rivet.yaml config to stdout. The schema is generated from the running binary’s Rust types, so it always matches the config grammar this version accepts. Pipe it to a file and reference it via a # yaml-language-server: $schema=… header so VS Code / Neovim’s YAML language server highlights invalid keys, suggests enum values, and surfaces required fields as you edit:

rivet schema config > rivet.schema.json
# then, at the top of rivet.yaml:
# yaml-language-server: $schema=./rivet.schema.json

State backend

By default Rivet keeps all run state (cursors, metrics, manifests, chunk checkpoints, schema drift, run journal, progression) in a SQLite file — .rivet_state.db — placed next to the config file. This works for local and single-node deployments.

For stateless containers / Kubernetes where the rivet pod is ephemeral or replicated, set RIVET_STATE_URL to a PostgreSQL connection string:

export RIVET_STATE_URL=postgresql://rivet:rivet@localhost:5433/rivet_state
rivet run --config rivet.yaml

Rivet creates all state tables automatically on first connect, running the full migration ladder up to the current schema version (the same schema-version sequence as SQLite). No manual DDL required.

Docker Compose (local dev)

docker-compose.yaml includes a dedicated postgres-state service on port 5433 (separate from the source postgres service on port 5432 so data and state never mix):

docker compose up -d postgres-state
export RIVET_STATE_URL=postgresql://rivet:rivet@localhost:5433/rivet_state
rivet run --config pilot.yaml

Security

• Passwords are redacted from all log and error messages: postgresql://user:***@host/db.
• A WARN is emitted when connecting to a non-localhost host without TLS. For production use a sslmode=require URL:

export RIVET_STATE_URL="postgresql://rivet:secret@db.internal/rivet_state?sslmode=require"

• The RIVET_STATE_URL value is not embedded in plan artifacts or config files. It is resolved from the environment at runtime.

Environment variables

Example: verbose logging

RUST_LOG=debug rivet run -c my_export.yaml

Example: PostgreSQL state backend

export RIVET_STATE_URL=postgresql://rivet:rivet@localhost:5433/rivet_state
RUST_LOG=info rivet run -c my_export.yaml

Exit codes

Every coded error and the exit its kind maps to: errors.md.