vortex-rdflib · bench SPARQL over rdflib stores — comparative benchmark

A run of bench/run_bench.py: vortex-rdflib's VortexRdflibStore against itself and:

Using a synthetic SPARQL set that includes lookups and scans, star and chain joins, FILTER / DISTINCT / ORDER BY / GROUP BY, and patterns over named graphs. The Vortex configurations use all a Dictionary layout, varying its residency (in-memory or file-backed) and secondary index (none, by-copy, by-reference).

rdflib's engine evaluates every store (except for pyoxigraph), to isolate the impact of every store overquad patterns reads. pyoxigraph is Oxigraph's own Python bindings used directly (its own parser, engine and terms, with no rdflib dependency). The labels specify where data comes from: in-mem stores live fully in memory, file stores perform file access reads per query.

The test dataset is composed of quads. Every statement about a subject goes into one graph, so the union of the graphs forms a triple set over which queries that name no graph operate. Each store loads it as an rdflib Dataset whose default graph is that union. pyoxigraph loads into its own store, queried with that union as its default graph. HDT and COTTAS cannot serve named graphs through rdflib — HDT's format has none, and pycottas' store does not expose the proper store interface for it. These stores use flattened N-Triples and do not report measurements in the Named graphs panel.

The BSBM tab of every section runs the same stores on the Berlin SPARQL Benchmark's Explore use case: data from the official generator, and the query streams the unmodified official test driver sends, captured once. The number in its name is the generator's scale, in thousands of products. Each store answers the warm-up mixes, then every measured query instance once, with the parameters the driver drew for it. A query over the per-query timeout is aborted and counted; a store over its time budget finishes its mix and stops, marked partial.

LOAD

Source file to queryable store

Cold start: from the shared source file — .nq for the stores with named graphs, the flattened .nt for HDT and COTTAS — to a queryable store. The same statements, but not the same bytes: dropping the graph column makes the .nt about 18% smaller, so those two libraries have that much less to parse. Most libraries build their own format first — serialize_rdf (vortex-rdflib), rdf2cottas, hdt convert — then open it; rdflib and oxrdflib parse directly, and pyoxigraph bulk-loads the file with its own parser. Median of repeated full rebuilds. rdflib-hdt is read-only: it cannot write HDT, so its file is built by the Rust hdt crate's CLI and only opened by the library measured here.

QUERY

SPARQL execution, per query

Median wall-clock time to execute and fully consume each query, reported as a pair of figures taken from one run: exec only is the evaluation of an already-translated algebra — the part a store's speed can move; full adds rdflib's parse and algebra translation. Green marks the fastest cell in each column, and the grey multiplier is every other cell's slowdown against it.

Two marks on the column headers. • is a heavy query — it touches the whole dataset. ‡ is a query whose answer is a cardinality (e.g., ASK or COUNT(*)).

rdflib-hdt's own optimize_sparql() fast path is switched off: it answers two of the queries wrongly — chain-2 with 26 results and optional with 152, against the 18 and 1 that rdflib and rdflib-hdt itself return without it. That row therefore runs on rdflib's engine, as every other row does.

MEMORY

Peak memory

Each store answers its queries in a process of its own. A store built into a file (vortex-rdflib, COTTAS, HDT) is built in one process and queried in a fresh one that opens the file, as an application opens a built store; the others load in the process that queries them. The figures are the querying process's. Peak RSS is the kernel high-water mark (VmHWM), which counts file pages a store maps and the kernel may drop again; Peak anon is the highest anonymous memory (RssAnon) sampled after the open or load and after every query: the memory the process itself holds. Lower is better.

SET

The query set

Anchor terms and thresholds come from the dataset generator's modular arithmetic, so every query matches at least one row. heavy queries touch the whole dataset and run with a lower iteration budget.

DATA

Every measurement

Every (query, store, mode) measurement from this run. Click a column to sort. An id ending in ::exec is the prepared-algebra run; the plain id is the full one, including rdflib's parse and algebra translation.