Cross-tool, cross-engine benchmark

How rivet compares to six other extraction tools — duckdb, clickhouse-local, sling, ingestr, dlt, odbc2parquet — reading Postgres / MySQL / SQL Server / MongoDB → Parquet on the same fixture, and how to reproduce it.

One source of truth, one runner, one report:

What it measures — three matrices, one run per tool

1. Benchmark — wall, rows/s, peak RSS, output MB, output files, type-drift count.
2. Harm to the source — the point of the bench. Universal axes (a co-running OLTP p99 probe, longest query / txn, held locks, connections, cache footprint, source CPU) plus each engine’s native counters (pg pg_stat_database, mysql SHOW GLOBAL STATUS + data_locks, mssql DMVs, mongo serverStatus).
3. Type fidelity — every source column vs each tool’s Parquet type family (jsonb→text, naive-ts→timestamptz, bool→int drift). N/A for MongoDB (JSON-blob).

Headline

rivet wins peak memory (15–60× lower — it streams via a server-side cursor, never buffering the result set) and type fidelity (0 drift on every engine, while also checksumming every value). It is not the throughput leader — ingestr and clickhouse beat its rows/s, and the report says so. Full numbers and the honest tradeoffs are in report.html.

Engine coverage: Postgres (8 tools) · MySQL (8) · SQL Server (6 — duckdb/clickhouse have no native reader) · MongoDB (3 — rivet/sling/ingestr only).

Reproducing

Prereqs — docker compose up -d (the project’s postgres/mysql/mssql/mongo containers), then run with the system python (it has PyYAML + dlt; the homebrew pythons ship a broken pyexpat):

# postgres, one table, all tools:
/usr/bin/python3 dev/bench/smoke.py --engine postgres --table content_items
# another engine / table:
/usr/bin/python3 dev/bench/smoke.py --engine mysql --table content_items
/usr/bin/python3 dev/bench/smoke.py --engine mssql --table orders
/usr/bin/python3 dev/bench/smoke.py --engine mongo --table content_items --tools rivet,ingestr,sling

Fixtures are seeded into a dedicated rivet_bench database per engine so the live-test rivet fixtures are never touched. Seed via the Rust tool (cargo run --release --bin seed --features dev-seed -- --target <engine> \ --mssql-url/--mysql-url ...); sizes live in matrix.yaml’s seed: block. MongoDB is seeded via the mongo driver.

Per-tool drivers (documented in each tool’s matrix.yaml note): odbc2parquet needs a vendor ODBC driver per engine (psqlodbc / MariaDB Unicode / msodbcsql18); dlt needs psycopg2 / pymysql / pyodbc; clickhouse’s mysql() needs 127.0.0.1 (not localhost); the rivet MongoDB source needs the worktree build (RIVET_BIN prefers target/release/rivet).

Notes on fidelity of the numbers

• The OLTP p99 probe is cleanest on Postgres (server-side \timing). On mysql/mssql it is a per-query docker exec, whose ~230 ms overhead compresses the signal toward 1×; on MongoDB it is skipped (mongosh cold-start). Read the harm story primarily from longq / memory / native counters on non-PG engines.
• Steelman applied to everyone — each tool runs its lowest-memory config that still completes; memory caps flatter competitors (lower RSS), and a self-audit removed a stray clickhouse thread cap. rivet’s numbers include its always-on per-value checksum (~7%) that no competitor performs.
• SQL Server heavy NVARCHAR(MAX) is ~130× slower to stream over TDS for all tools (a protocol characteristic, not rivet) — the mssql matrix runs on orders.

MongoDB CDC and a folded-in CDC-churn dimension (cdc_churn.sql) are tracked for a future revision.