Post #7 · Evidence & capability

Where TGMS Is Fast — and Where Specialists Still Win

A correctness-gated comparison with DuckDB, PostgreSQL, ClickHouse, Neo4j and Memgraph.
In one sentence: TGMS is fastest on operations that combine history with graph structure, and dedicated systems remain better at large whole-window aggregation — so the useful question is not which system is faster but which operation shape you have.

Why this matters

"Is it fast?" is the wrong question to ask about a specialised store, and answering it with one number is how benchmarks mislead. A system can be twenty times faster on the operation you run hourly and four times slower on the one you run nightly. What a reader deciding whether to adopt TGMS needs is a map: which shapes of work does it do well, which shapes should stay with PostgreSQL or ClickHouse, and where is the boundary.

So we built one registry of 13 queries covering the shapes a temporal graph workload actually contains, and answered every one of them on 6 systems.

The rule that makes the comparison worth reading

No cell is timed until it returns the same answer. Every system replays the same event log, so the underlying rows are identical; each query is implemented natively per system — SQL for PostgreSQL and ClickHouse, one Cypher statement for both graph engines — and its result is reduced to a fingerprint over rows, order and types. A cell that disagrees is a defect to fix, not a number to publish. Nothing in the tables below was timed before its fingerprint matched.

That rule cost real work and paid for itself: it caught bugs in our baselines (a recursive query that enumerated a state space it should have pruned, a missing index that made one engine look ten times worse than it is) and two genuine defects in TGMS. Tuning is part of what is measured, so each baseline got covering indexes, raised working memory, and a checked query plan — the details are in the report.

The workload map

Every registry query combines some of four things: time (a window over when facts held), belief (what the database knew at a past moment), graph structure (following relationships), and grouping. Which of them a query touches predicts who wins it far better than any notion of a system being "fast".

ShapeWhat it combinesFastest here
Point lookup by identitybeliefTGMS
Snapshot, diff, neighbourhoodtime + structureTGMS
Traversal: reachability, k pathstime + belief + structureTGMS
Temporal motiftime + structure + orderingTGMS
Interval jointime + structureTGMS
Whole-window aggregationtime + groupingClickHouse at scale

What we measured

Evidence. Question: which system answers each operation shape fastest, given identical answers? Workload: 13 registry queries on a synthetic bi-temporal event log containing corrections, at 200,000 / 1,000,000 / 10,000,000 events, plus a frozen real messaging dataset. Systems: TGMS native, TGMS-on-DuckDB, PostgreSQL 16.14, ClickHouse 26.8, Neo4j 5.26, Memgraph 3.12 — all on one 40-core host, tuned, warm. Metric: median latency in milliseconds, lower is better, 30 repetitions for sub-second queries. Held constant: the event log, so identifiers and transaction times are identical everywhere. Single latency cells reproduce to about ±20% between days, so differences smaller than that are ties. Raw records: benchmarks/results-v1/.

What we found

Three results, in decreasing order of how much they should affect a decision.

1. On traversal, the specialists lose by about two orders of magnitude — on the query family that is supposedly their home ground. Temporal reachability at 200k events costs TGMS 14.7 ms; the two graph engines take 3.9-7.3 seconds. The closed-triangle motif costs 28.7 ms against 2.1-5.5 seconds. This is not a claim that Neo4j and Memgraph are slow at graph traversal — they are not. It is that every hop of a bi-temporal walk re-filters by validity and belief on relationship properties, and no index in either system accelerates that predicate, so the traversal degenerates into a filtered scan per hop. Cypher expressed the motif query more elegantly than any SQL in the study and still ran it two orders of magnitude slower. Query elegance and execution speed are independent axes.

2. On whole-window aggregation ClickHouse still wins above a few hundred thousand events, by a factor of two. It answers the bucketed count at 10M in 37.9 ms against our 84.72.2×, and the same ratio at a million, and a tie at 200k — a constant of the query shape rather than something that widens as data grows.

That constancy is itself a result, and it took three rounds of profiling to see. The gap at 10M measured 12× before any of that work, and each round of improvement to our own scan and aggregation path shrank it — to 8.7×, then 4.6×, then 2.2× — while the ratio at smaller scales barely moved. What had looked like ClickHouse pulling away with scale was our implementation falling behind with scale. The residual factor of two has not moved across any of it, and we do not currently know how to move it. If aggregating whole windows is the job, a column store is still the right tool.

3. On point lookups the picture reversed, and it is worth saying why. PostgreSQL held this shape for most of the campaign — a warm B-tree is a hard floor. At 200k and 1M, TGMS now answers in 0.1 ms against PostgreSQL's 0.3 at 200k and 0.4 at a million, because a profiling pass removed a segment rebuild from a path that an index had already resolved. At 10M events PostgreSQL is still ahead. The honest summary is that the two are close on this shape and the ordering depends on scale.

What this means in practice

If your workload is…Use
Reconstructing past belief states, auditing correctionsTGMS
Time-respecting traversal, temporal motifs, interval joinsTGMS
Large whole-window aggregation and rollupsClickHouse, by about 2×
Grouped aggregation over event historyEither — comparable at every scale measured
High-rate indexed point reads over current statePostgreSQL, or TGMS — measure at your scale
Ordinary graph traversal with no temporal filteringNot established by this study

A realistic deployment may well be TGMS beside a column store: the temporal-graph composition in one, the nightly rollups in the other. That is a normal outcome for a specialised system, and it is more useful to a reader than a claim of universal superiority.

What this result does not show

Takeaway

The comparison did not produce a winner; it produced a boundary. TGMS owns the shapes that combine history with structure, by margins large enough to survive any reasonable measurement noise. Dedicated analytical engines own large-scale aggregation. Knowing which of those your workload is made of is worth more than any single benchmark number.

The baselines audited us back

Building honest baselines fixed our engine, which is the part of this exercise we did not anticipate. PostgreSQL's sub-millisecond lookups exposed point reads that rebuilt whole segments an index had already located — 76 ms became 1. ClickHouse answered a motif query our own cost model had refused: a guardrail false positive, since repriced. A second such false positive, on the k-paths query, was found and closed the same way. These are defects that TGMS-only benchmarking could never surface, because both our backends were slow in the same places and agreement reads as health.

Storage, one accounting

systemwhole store, B/rownote
TGMS native25.1compressed, after generation collection
ClickHouse78.4lz4 MergeTree — the only other compressed form
DuckDB187.7uncompressed columnar
PostgreSQL549.7two thirds is the indexes its sub-millisecond lookups are made of
What these storage numbers count. TGMS counts segments, manifests, correction records and its dictionary, but its query indexes live in memory and are not persisted — so its figure buys less query readiness than the others'. ClickHouse is lz4-compressed with no secondary indexes. PostgreSQL includes the eight covering indexes its lookup speed is made of. DuckDB is a single uncompressed columnar file. The comparison is of stored bytes for the same logical content, not of equal query readiness.
Reproduce it. One harness command regenerates every table; raw records ship in the repo; the run exits nonzero on any hash mismatch, so the correctness gate and the benchmark are the same command. docs/eval has the report, capability matrix, and step-by-step reproduction.