Export modes — which one do I pick?
Every export declares a mode:. Start from the decision shortcut, then open the
guide for the mode you land on.
Decision shortcut
- Small-to-medium table, want a fresh snapshot every run →
full - Append-only or has an
updated_at, want only new/changed rows →incremental - Millions+ of rows, want speed and crash-resume →
chunked - Only the last N days matter (event/log table) →
time_window - Continuous low-latency replication from the WAL / binlog / oplog →
cdc
When in doubt, rivet init inspects the table and picks a sensible default for
you (small → full, large with an integer key → chunked).
At a glance
| Mode | Use when | Keeps state? | Parallel? | First run |
|---|---|---|---|---|
| full | complete snapshot each run | no (stateless) | no | exports everything |
| incremental | only rows past the saved cursor | yes (cursor) | no | exports everything, then deltas |
| chunked | tables too large for one full scan | yes (checkpoint, --resume) | yes | full, split into ranges |
| time_window | rolling N-day window | no (recomputed each run) | no | the window only |
| cdc | continuous log-based change capture | yes (resume checkpoint) | yes (multiplexed streams) | optional initial snapshot, then stream |
Notes worth knowing before you run:
incrementalfirst run = full export. With no cursor yet, every matching row is exported, so the first run behaves likefull— sizebatch_sizeaccordingly (incremental.md).chunkedclean re-runs are NOT idempotent. A crash +--resumeis at-least-once: a re-run chunk can be written twice (byte-identical), so de-duplicate downstream if you re-run (chunked.md).- Composite cursors are an
incrementalvariant, not a separate mode — see incremental-coalesce.md when one timestamp column isn’t enough. - Source engine matters. The SQL sources (PostgreSQL, MySQL, SQL Server)
support every mode. MongoDB, a document store, supports only
full(with keyset / parallel / resume paging on_id) andcdc— see ../reference/mongodb.md.
Full configuration reference: ../reference/.