ADR-0021: Chunked Schema-Drift Detection Runs Pre-Chunk

Status: Accepted Date: 2026-06-18

Context

rivet detects column-level schema drift (added / removed / retyped columns) against a per-export baseline in export_schema (state::detect_schema_change + store_schema), driven by on_schema_drift: warn|continue|fail. Until now this ran only in single (non-chunked) mode (pipeline::single); the call site’s own comment noted the state store “is only populated by the drift-detect path below, and not at all in chunked mode.”

A pilot re-ran a 655k-row table 4× over two days in chunked mode and got no column-level snapshot at all — export_schema stayed empty, rivet state showed nothing, and drift could never be detected on later runs. Chunked is the default for large tables, so the most-exposed exports were exactly the ones without drift coverage.

Naively replicating the single-mode flow in chunked is wrong for two reasons:

1. Timing / fail semantics. Single detects drift from the sink’s resolved schema and fail aborts before writing. In chunked, by the time a chunk sink resolves a schema the chunk is already written (and parallel modes run many chunks at once) — a post-write fail cannot prevent the corrupt-shaped output it exists to stop.
2. Statelessness. The non-checkpoint executors (chunked::exec) are deliberately stateless (no StateStore), so they cannot store_schema from inside the worker loop at all.

Decision

Detect drift once, pre-chunk — in each chunked run function right after chunk boundaries are computed and before any chunk executes — from a schema resolved via Source::type_mappings (a metadata-only query; no data scan). type_mappings → build_arrow_field → arrow_schema_to_columns yields the same canonical SchemaColumn format single mode derives from the sink, so the baselines are comparable.

fail then aborts before the first chunk writes — matching single’s intent; warn / continue store-or-update the baseline. The logic lives in pipeline::schema_drift as one deep core (check_and_persist: detect → policy → store) behind two thin column-source adapters — check_from_sink_schema (post-write, sink schema: used by the single, keyset, and parallel-Mongo runners) and check_from_type_mappings (chunked, pre-chunk, type_mappings schema). The four chunked Detect arms reach the chunked adapter through one shared preamble, prepare_chunk_plan (compute chunk ranges → run the pre-chunk drift check), rather than re-implementing detect-then-check per runner. This makes schema_drift the third runner-write facade alongside commit::record_part and RunStore, superseding ADR-0018’s note that drift “does not generalize across modes” (see that ADR’s 2026-06-18 update).

Consequences

• All chunked modes (sequential / parallel, checkpoint / non-checkpoint) gain the column-level drift parity single has; export_schema and rivet state are populated for chunked exports.
• One extra metadata round-trip per chunked Detect run (zero-row type_mappings) — negligible against the chunk scans, and itself source-friendly.
• Cross-mode caveat. A baseline stored by single (data-derived sink schema) and one stored by chunked (type_mappings schema) can differ for types rivet infers from data rather than the catalog (e.g. a decimal scale that is a placeholder in type_mappings until a value is observed). Within one mode this is consistent; switching an export between single and chunked may log a one-time drift that self-heals on the next run. Acceptable — documented here so a future contributor does not chase a phantom.
• Resume / Precomputed chunk sources skip the pre-check (drift was already evaluated on the original Detect run that planned the chunks).

Amendment 2026-09-26: resume and apply run the pre-chunk check too

Superseded in part: resume and precomputed (rivet apply) chunk sources now run the pre-chunk drift check as well (check_drift_only, check_drift_only_fresh), because the drift this guards against happens between the plan or the crash and the resume or the apply. The only skip is a precomputed plan with no ranges.