The operational cost of keeping a correctable history

In one sentence: keeping a full, correctable history costs essentially nothing until something is corrected, after which storage and query time grow with how often corrections happen — and the costs that actually decide a deployment are not those, but a per-process warm-up of roughly half a minute and a memory floor that has to be budgeted.

Decide this first

Before any numbers, the four questions that determine whether this design fits, in the order they matter:

The idea in plain language

When a fact turns out to have been recorded wrongly, an ordinary store overwrites it and the earlier belief is gone. A store that must answer "what did we think last March?" cannot overwrite: it keeps the superseded version and records when the system believed it. That second clock is transaction time, and the cost of the design is the cost of those retained versions.

Axis 1 — correction density sets storage

At zero corrections we could not measure a penalty. The widest latency gap was 3.4%, its sign varying query to query — the full store was faster on 6 of 11 — with bytes differing by 0.002% and resident memory by 0.2% (peak memory by 0.02%), all far inside the ±20% band a single cell reproduces to across days. So: no measurable overhead in the tested zero-correction configuration, not "zero overhead".

Axis 2 — latency is a separate bill

Storage and speed do not move together, so "free until the first correction" is too broad for both. Latency steps almost immediately: at 0.01% density — about a hundred corrections in a million events — most scan queries are 33–47% slower and one neighbourhood query is 3.4× slower. After that, growth is roughly linear in each tenfold increase of density rather than in density itself.

Scale sharpens it. At ten million events, a correction density of only 0.1% makes the interval-join query 4.0× slower — that is the one density the 10M sweep was run at. Back at a million events, where the full density sweep lives, the 20% cell has one full-store query refused outright by the cost guardrail — the limit that declines work predicted to be too expensive — where the stripped store answers in 35.6 ms. Retention can cost answerability, not just time.

Axis 3 — resident memory, and a floor you must budget

At ten million events every process needed about 5.93 GB regardless of the query: a 0.6 ms point lookup and a four-second diff had identical peaks. Under a 4 GB cap — imposed with a container memory limit — the suite did not slow down but was OOM-killed: when a process exceeds its memory limit the operating system terminates it outright rather than degrading it.

After the fix described in the sidebar, the same suite peaks at 1.76 GB under a 2 GB cap, and still answers under a 256 MiB cache budget at 1.61 GB — paying 32,615 cache evictions and running 1.8–27× slower on scans, every answer still hash-identical. Capped configurations are not uniformly faster or slower than uncapped: individual queries land within about 10% either way.

Axis 4 — service lifecycle and concurrency

The cost most likely to disqualify this design is not correction-related at all — and it shrank 7–9× since this post first ran. On a fresh process at ten million events, the first query takes 3-8 seconds — a per-process warm-up rather than a disk-reading story, since evicting the page cache barely changes it. A short-lived client gets its first answer from DuckDB 1.2–8.8× sooner at 10M, and at 1M the two are at parity. The engine still rewards a resident process, and the steady-state numbers here assume one.

Concurrency is the happier axis, with one clarification this post previously owed its readers. Sixteen readers deliver 13.1× the aggregate throughput of one reader at a million events — throughput, not latency: per-query latency worsens under load, by up to 53% on the slowest query. At ten million events the same measurement gives 10.2× with two of sixteen readers OOM-killed. Concurrency is bounded by memory, not locks.

Axis 5 — recovery

Rebuilding state by replaying the event log was superlinear in correction density. At 20% it took 2,856.8 seconds — about 47.6 minutes — before a per-generation close-version cache and a faster version lookup brought the same replay to 361.1 seconds, or 6.0 minutes: 7.9× better.

Two qualifications. There is no stated recovery objective, so six minutes is an improvement rather than a pass — nothing defines what recovery time this workload requires, and that gap is worth closing before anyone depends on it. And this is log replay during dataset construction, not a crash-and-restart test; no process was killed. Reading it as a recovery figure is an inference.

What this result does not show

These are single observations on one host. Storage, memory and replay are one run each, and the replay pair is a before-and-after across two commits rather than a repeated measurement. Latency medians reproduce to about ±20% between days — treat smaller differences as ties.

Correction density is synthetic. We generate corrections at a chosen rate with a fixed mix of replacements, carves and retractions. Real corrections may cluster in time or on particular entities, which these numbers would not capture.

It is not a cross-system cost comparison. Only cold start is measured against another engine. We have not found a directly comparable public cost breakdown for a bi-temporal store to set these beside — which is why they are published in this shape, not a claim that none exists.

What this means in practice: four profiles

Takeaway

Price a durability promise per axis rather than as one number. The bi-temporal contract is cheap where people expect it to be expensive — a store that is never corrected pays nothing we can measure — and expensive where they do not look: a half-minute warm-up per process, and a memory floor that kills rather than slows. Those two, not storage, decide whether it fits.

Continue reading