Benchmarks

Referenced from CLAUDE.md §6.2 (“regressions block merges”) and from docs/rfcs/0001-template-miner.md §8. Flat-file, living document, parallel to docs/hazards.md. Updated with measured results as they come in.

This document is an honesty contract with ourselves. The thesis (CLAUDE.md §2) claims that Parquet + Drain-derived template mining + DataFusion beats the naive alternative of byte-level compression over flat text. That claim is falsifiable. This file lists the measurements that would falsify it.

The thresholds were pinned before any number was measured; if we miss them on representative corpora, the thesis is wrong and a pillar changes. As of 2026-06-14 the four gating thesis-gates B1, B2, C1, C2 all pass on the §1 hardware baseline (§9.4/§9.6). A1 fails but no longer gates — RFC 0011 (accepted) reclassified the compression-vs-zstd ratio as a recorded diagnostic (its failure is structural; see §2 / the §7 table).

0. How to read this document

Every goal below carries two labels.

• Scope — thesis-gate, tuning-goal, or diagnostic.
  • A thesis-gate failing on representative corpora means a pillar (CLAUDE.md §2) is wrong. The response is an RFC, not a sprint.
  • A tuning-goal failing means the design is sound but the implementation needs work. The response is a PR.
  • A diagnostic is measured and recorded but gates nothing — it characterises a property or guards against regression. A1 was reclassified here by RFC 0011 (accepted); see §2.
• Bar — must-win, should-win, stretch, or informational.
  • must-win — shipping without it is shipping a broken claim.
  • should-win — expected on representative corpora; explained when missed.
  • stretch — aspirational; missing is not a bug.
  • informational — a diagnostic’s bar: the number is recorded for insight, never blocks.

A goal with scope thesis-gate and bar must-win is load-bearing for the whole project. Four of those below are gating — B1, B2, C1, C2, each marked [THESIS]. A1 keeps the [THESIS] tag (a thesis-relevant measurement) but RFC 0011 (accepted) set its scope to diagnostic: it is recorded, not gating (see its section below and the §7 table).

1. Corpora and methodology

Before any goal is meaningful, the corpora and methodology must be pinned — otherwise we will argue about numbers instead of about architecture.

• Public: LogPAI corpora (HDFS, BGL, Spark, Apache, OpenSSH, Windows) — the same corpora the Drain paper reports on. Lets us reproduce published claims as a sanity floor.
  • LogHub HDFS_v1 is the first of these wired in, as a bench-time-fetched corpus for the query gates: .github/workflows/query-bench.yml downloads HDFS_v1.zip from the official Zenodo record (record 8196385, DOI 10.5281/zenodo.8196385, md5-pinned in the workflow), uses the extracted HDFS.log (~1.47 GiB plain text — above §8’s ≥ 1 GiB canonical minimum) in-job, and discards it with the runner. It is never redistributed: not committed (the testdata/corpus/README.md anonymisation gate — LogHub data is explicitly not sanitised), not attached to a release, not uploaded as an artifact; only aggregate numbers leave the job. LogHub’s license notice, included here as it requires: “The datasets are freely available for research or academic work. For any usage or distribution of the datasets, please refer to the loghub repository URL (https://github.com/logpai/loghub) and cite the loghub paper: Jieming Zhu, Shilin He, Pinjia He, Jinyang Liu, Michael R. Lyu. Loghub: A Large Collection of System Log Datasets for AI-driven Log Analytics. In IEEE International Symposium on Software Reliability Engineering (ISSRE), 2023. The above license notice shall be included in all copies.”
• Self-collected (deferred): at least one anonymised corpus per target deployment archetype. Proposed set:
  1. Structured Java/Spring service (well-templated, low entropy).
  2. Go service under Kubernetes (heterogeneous, mid entropy).
  3. Heterogeneous k8s aggregate across many services (high entropy, mixed formats).
• Hardware baseline: a commodity 8 vCPU / 32 GiB RAM host with gp3-class SSD. All must-win numbers are quoted against this baseline; scaling to larger hardware is a separate question. The realised baseline (the baseline-8vcpu-32gib hardware tag, first used for the §9.4 authoritative run) is a dedicated host with 8 dedicated vCPU, 32 GiB RAM, and a local NVMe-class SSD — at or above the spec on every axis, so numbers quoted against the tag satisfy this baseline. It is identified only by the tag.
• Reference system: zstdcat <file.zst> | grep <pattern>. The “naive alternative” the thesis beats or does not beat. Everything is quoted relative to this, not in absolute terms.

Goals quoted below assume this setup. When a goal is measured on a different setup, the measurement is annotated.

2. Compression goals (Category A)

The core claim that template mining does useful work before byte codecs run.

A1 [THESIS] — End-to-end compression ratio vs. zstd-alone

Demoted to a diagnostic (RFC 0011, accepted). A1 is refuted on every corpus class — including the maximally-templated one — for structural reasons, so it no longer gates any RFC’s validated. It is still measured and recorded (§7 table / §9 series) as the columnar queryability premium and a codec-regression guard. The scope, bar, target, and falsifier below are retained as the diagnostic’s reference line — now informational, not gating.

• Scope: diagnostic (RFC 0011; originally thesis-gate).
• Bar: informational (RFC 0011; originally must-win).
• Metric: bytes(raw_corpus) / bytes(ourios_parquet_directory) compared to bytes(raw_corpus) / bytes(zstd_compressed_corpus).
• Target: Ourios ratio ≥ 3× the zstd-alone ratio, on every corpus in §1. Best-case corpora (well-templated services) should show ≥ 10×.
• Falsifier: if any representative corpus yields ≤ 2× improvement over zstd-alone, the template-mining pillar is not pulling its weight on that class of logs. Open an RFC.
• Why recorded (diagnostic, not a bar): CLAUDE.md §2 pillar #2 describes a logical 50–200× reduction (lines → (template_id, params)) whose payoff is query pruning (B1/B2), not on-disk bytes vs a byte codec. A1 tracks the on-disk ratio as the columnar queryability premium + a codec-regression guard; RFC 0011 (accepted) demoted it from a gate to this diagnostic.

A2 — Bytes per line, amortised

• Scope: tuning-goal.
• Bar: should-win.
• Metric: total Parquet bytes for tenant / line count for tenant.
• Target:
  • Structured service logs: ≤ 30 B/line.
  • Heterogeneous k8s: ≤ 100 B/line.
  • Stretch: ≤ 15 B/line on high-repetition corpora.
• Why: makes A1 legible to operators, who think in bytes-per-line, not ratios.

3. Query performance goals (Category B)

Why not zstdcat | grep? Because the query layer is supposed to exploit structure the tree extracted.

B1 [THESIS] — Predicate-pushdown queries

• Scope: thesis-gate.
• Bar: must-win.
• Query shape: count events WHERE tenant=X AND ts BETWEEN t1 AND t2 AND level='ERROR'.
• Reference: zstdcat files_in_range.zst | grep ERROR | wc -l on the same corpus, same time window.
• Target: Ourios ≥ 10× faster at 1 GiB corpus, widening to ≥ 100× at 100 GiB.
• Falsifier: if Ourios is not materially faster than the zstdcat pipeline on predicate queries, DataFusion + Parquet statistics are not delivering on the “skip row groups via footer reads” pillar (CLAUDE.md §2.1). Open an RFC.
• Instruments: B1 is proven structurally (deterministically) by ourios-querier’s rfc0007_1_* tests. The criterion bench crates/ourios-bench/benches/b1.rs adds the wall-clock ratio: a b1/synthetic group (controlled pruning instrument vs. an in-process zstdcat | grep reference) and a b1/real-corpus group (set OURIOS_B1_CORPUS_DIRS to a comma-separated list of corpus dirs; skipped when unset). The real arm runs OTLP corpora only (corpus/otel-demo-v*, which carry real per-record severity): B1’s predicate filters on severity, and the RFC 0006 §3.3 plain-text loader assigns every line a fixed severity (9 / INFO), so a severity predicate over a plain-text corpus has no selectivity and such dirs are skipped with a note. CI runs land via .github/workflows/query-bench.yml on ci-runner — indicative only. The authoritative numbers are the baseline-8vcpu-32gib run of 2026-06-12 (§9.4): PASS at 34.2× / 25.4× on the two ~1 GB OTel-Demo corpora, with exact row-count agreement against the reference pipeline. Open quality improvement (non-blocking): the measured error bands are ultra-thin (11 / 28 rows), which flatters pruning — a denser error band is the remaining methodological wish.

B2 [THESIS] — Template-exact queries

• Scope: thesis-gate.
• Bar: must-win.
• Query shape: SELECT * WHERE template_id = X AND ts BETWEEN ….
• Target: latency proportional to result cardinality, not to corpus size, above a corpus size of ~10 GiB. Concretely: median latency ≤ 200 ms for a query returning 10 000 rows, regardless of whether the corpus is 10 GiB or 10 TiB.
• Falsifier: if template-exact queries scan proportionally to corpus size, template mining is buying compression but not query locality — the inverted-index collapse thesis (CLAUDE.md §2) is wrong in practice. Open an RFC.
• Instruments: B2 is proven structurally (deterministically) by ourios-querier’s rfc0007_2_* test — for a fixed result the scanned row groups + bytes stay flat as the corpus grows. The criterion bench crates/ourios-bench/benches/b2.rs adds the wall-clock view: a b2/synthetic group (result held constant, corpus scaled 1×/10×/50×) and a b2/real-corpus group over real corpora (set OURIOS_B2_CORPUS_DIRS to a comma-separated list of corpus dirs; skipped when unset, since the corpora aren’t committed). Both loader formats feed it: the OTLP/JSON corpus/otel-demo-v* releases and the bench-time-fetched plain- text LogHub HDFS_v1 (§1). Run with cargo bench -p ourios-bench --bench b2. CI runs land via .github/workflows/query-bench.yml on ci-runner — indicative only. The authoritative numbers are the baseline-8vcpu-32gib run of 2026-06-12 (§9.4): PASS — the windowed template-exact scan stays at 1 row group with a flat ~4.2–5.9 ms latency band across every corpus, including the first reading from a second corpus family (LogHub HDFS_v1, 11.2 M rows: 1/14 row groups, 5.92 ms), while the full-span variant grows with corpus size. The formal target speaks above ~10 GiB, which remains a future scale extension; the flat shape holding at 11.2 M rows across two corpus families is the operative evidence.

B3 — Substring queries (the hard case)

• Scope: tuning-goal.
• Bar: must-match; stretch: beat.
• Query shape: SELECT * WHERE body LIKE '%<substring>%' or equivalent.
• Target: not slower than the reference system. Stretch: faster on well-templated corpora by searching the template text rather than every line.
• Why this is only tuning-goal, not thesis-gate: substring search is the case where the tree does not help directly. We are allowed to match the reference system here; losing against it is a bug but not a pillar failure.

4. Miner correctness goals (Category C)

Correctness is not a performance goal, but it belongs here because these are the properties the benchmark harness actually measures on every run.

C1 [THESIS] — Bit-identical reconstruction rate

• Scope: thesis-gate.
• Bar: must-win.
• Metric: of all non-lossy-flagged rows, fraction whose reconstruct(template, params) equals the ingested bytes exactly.
• Target: 100.000%.
• Falsifier: a single row that reconstructs wrong without a lossy flag is a violation of CLAUDE.md §3.3 and a blocker, not a benchmark regression. Accompanied by: the lossy-flagged fraction should be ≤ 5% on structured corpora, ≤ 20% on heterogeneous ones, as a quality signal (not a gate).
• Why this is a thesis-gate: if we cannot promise reconstruction, the honesty contract (lecture §6) collapses.

C2 [THESIS] — Template count convergence

• Scope: thesis-gate.
• Bar: must-win.
• Metric: template count as a function of lines ingested, on a corpus from a single stable service.
• Grain (amended for #444, 2026-07-10): because the metric is defined per stable service, the gate is evaluated per service.name on a multi-service corpus, not on the whole corpus. A corpus passes iff every service with ≥ 1 M lines converges; a single-service (or plain-text <unknown>) corpus is gated on that one service’s exact-millionth-line ratio, reproducing the pre-amendment verdict for historical converged corpora. The whole-corpus ratio is retained as a diagnostic. See RFC 0006 §3.4.3.
• Target: template count grows sub-linearly and plateaus within 2× of its steady-state value by 1 M lines. Steady-state value is corpus-specific but is on the order of 10²–10⁴ templates for a normal service.
• Falsifier: if template count grows linearly with corpus size, Drain has failed to abstract — we are storing one template per line, which means the tree is providing compression only accidentally. That is the inverse of the thesis. Open an RFC.

C3 — Merge rate

• Scope: tuning-goal.
• Bar: should-win.
• Metric: merges_total / lines_ingested.
• Target: ≤ 1 merge per 10⁵ lines on stable corpora, with every merge carrying an audit event. Spikes above this rate are investigated; they usually indicate a new service version.
• Why only tuning-goal: merge rate depends on corpus stability more than on algorithm quality. The auditing is the invariant (§3.1); the rate is a signal.

C4 — Parameter overflow rate

• Scope: tuning-goal.
• Bar: must-win.
• Metric: fraction of rows where any params slot hit the 256 B limit.
• Target: ≤ 1% on representative corpora, per CLAUDE.md §3.2.
• Falsifier (tuning sense): if >1% on a common archetype, either the limit is too tight for that workload or a masking rule is missing. The response is tuning, not an RFC.

5. Ingest goals (Category D)

The hot path must keep up with real deployments; otherwise none of the above matters.

D1 — OTLP → WAL throughput

Recast per-node (RFC 0034; enacted 2026-07-21). D1’s original metric was lines/second/core — an axis the architecture deliberately serializes twice (sequential per-tenant mining — the CLAUDE.md §3.7-scoped trees assign ids first-seen, which must match WAL-order replay, RFC 0001 §3.5.3; and the single durable WAL stream, §3.4) and one that contradicted D1’s own per-node falsifier (§9.19–§9.21). The must-win below is per-node on the §1 baseline class; the old per-core target and the per-tenant single-stream ceiling are retained as recorded diagnostics — informational, gating nothing (the RFC 0011 A1 pattern). Asserting run: §9.23 (PASS).

• Scope: tuning-goal.

• Bar: must-win.

• Metric: lines/second sustained per node on baseline-8vcpu-32gib, multi-tenant load (soak --tenants N with N = cores) through one shared WAL/commit stream, with WAL fsync batched at 100 ms (the CLAUDE.md §3.4 default).

• Target: the asserting run offers exactly 100 000 lines/s per node and must achieve ≥ 99% of offered (the single acceptance rule — pacing loss up to 1% is within the bar), with p99 ingest-ack latency ≤ 200 ms over that same run. Achieved ≈ offered is also the below-saturation proof: a saturated pipeline cannot keep pace with the paced load, and queue-bound latencies at over-offered load are a different regime that does not count (§9.20’s reading).

• Diagnostics (informational, still recorded — RFC 0034):

  • the original per-core target (≥ 100 000 lines/s/core) is retained as the diagnostic’s reference line; §9.23 records 12,490 lines/s/core at the asserting run.
  • the per-tenant single-stream ceiling — the most one service can push into one tenant (≈ 86k lines/s under the §9.20 probe configuration; §9.20/§9.21) — guards the mining path against regression.

  Neither gates any RFC’s validated.

• Falsifier (tuning sense): below this we cannot ingest a meaningful share of production traffic per node, which makes the operational story uninteresting.

D2 — WAL → Parquet compaction keeps up

• Scope: tuning-goal.
• Bar: must-win.
• Metric: WAL backlog (bytes, segments) as a function of time under sustained ingest at D1’s rate.
• Target: bounded; backlog returns to zero during any one-hour window of sustained load.
• Falsifier (tuning sense): a growing backlog under steady-state load means compaction is the bottleneck — a correctness-adjacent bug because it lets the WAL grow unboundedly.

D3 — Small-file count under sustained load

• Scope: tuning-goal.
• Bar: should-win.
• Metric: number of Parquet files per tenant per day after background compaction has settled.
• Target: file sizes cluster in the 256 MiB–2 GiB band per CLAUDE.md §4 / hazard 4. Fewer than 5% of files below 128 MiB at steady state.
• Why: the small-file problem is a named hazard, not a nice-to-have.

6. Honesty goals (Category E)

Not performance. Not falsifiable by a benchmark in the usual sense. Listed here because the benchmark harness asserts them on every run.

E1 — Zero silent merges

• Scope: correctness invariant (not a benchmark).
• Metric: in the corpus-test suite, for every row whose template_id changed over its lifetime in the tree, an audit event exists with matching timestamp and tenant.
• Target: 100%. This is a proptest, not a measurement.

E2 — Zero cross-tenant leakage

• Scope: correctness invariant (not a benchmark).
• Metric: no template mined under tenant A ever appears in tenant B’s tree or in a row for tenant B.
• Target: 100%. Asserted via corpus tests that interleave lines from two synthetic tenants and verify complete isolation.

7. The thesis-gate summary

The five [THESIS]-tagged goals, consolidated:

Policy: if one thesis-gate fails on one representative corpus, that is a corpus-specific tuning RFC. If two or more thesis-gates fail on any representative corpus, that is a pillar-level RFC — we pause implementation and revisit CLAUDE.md §2 before continuing.

This escalation rule is the point of the whole document. The worst failure mode for a greenfield project is shipping something whose central claim quietly fails on real data and then papering over it with more implementation. These goals exist so we cannot do that to ourselves without noticing.

8. What is deliberately out of scope

• SIEM-style full-text search latency — explicitly out of scope (CLAUDE.md §1).
• Cross-tenant aggregation queries — tenancy is isolation-first (CLAUDE.md §3.7). Aggregations that cross tenants are an RFC topic, not a benchmark.
• LLM-based parser comparisons — interesting, deferred. Listed in RFC 0001 §7 as an alternative. Benchmarking it would be a separate RFC.
• Cold-start query latency — below a corpus size of ~1 GiB the overhead of Parquet metadata dominates, and the thesis is uninteresting. Benchmarks start at 1 GiB.

9. Status

First measurements landed 2026-06-01 (the writer-side gates A1 / C1 / C2 — see §9.1). They are diagnostic, not canonical: they ran on a GitHub-hosted runner (ci-runner), not the §1 hardware baseline (baseline-8vcpu-32gib), against an OTel-Demo corpus that is shape-representative (real multi-service template + envelope diversity) but not size-representative — every corpus is well below §8’s ≥ 1 GiB canonical minimum, so this run is intentionally diagnostic, not a thesis verdict. The query-side gates now have instruments — B1 and B2 are proven structurally in ourios-querier, and both have criterion latency benches with real-corpus arms (§B1/§B2 “Instruments”; OTel-Demo for B1, OTel-Demo + the bench-time-fetched LogHub HDFS_v1 for B2, run on ci-runner via .github/workflows/query-bench.yml as indicative numbers). 2026-06-11 extended the writer-side scale series to ~1 GB (§9.2) and landed the first B1/B2 query readings (§9.3) — recorded here as indicative ci-runner entries per the maintainer’s 2026-06-12 authorization. 2026-06-12 landed the authoritative baseline run (§9.4): every gate measured on the §1 hardware (baseline-8vcpu-32gib), recorded per the maintainer’s 2026-06-12 authorization. B1, B2, C1, and C2 pass authoritatively; on that basis RFC 0007 flipped to validated (its gates, per docs/verification.md §3, are the querier-pillar ones — B1/B2). A1 fails authoritatively and carries a hardware-sensitivity caveat (§9.4). (A1 was subsequently reclassified a recorded diagnostic, not a gate — RFC 0011, accepted 2026-06-14. The A1 readings throughout §9 are diagnostic; A1 gates nothing, and the “open gate” / “must-win” framing in the dated entries below is superseded.)

Reviewers: a PR that materially affects the hot path must either (a) cite the benchmark result and its delta against the relevant goal, or (b) explain why the hot-path effect is bounded below measurability. “I did not run the benchmarks” is a PR rejection, per CLAUDE.md §6.6.

No ourios-bench --update-benchmarks-md run has populated this region yet. It is the bench-managed results area — automated runs replace everything between these markers with one table per (git-sha, hardware). The hand-written §9.1 below is the curated diagnostic narrative and lives outside the region so automated runs never touch it. (This empty region is pre-placed so the first --update-benchmarks-md run replaces it in place rather than appending a second results section at end-of-file.)

9.1 Results — 2026-06-01 (diagnostic, ci-runner)

Corpus. corpus/otel-demo-v{1..4} — OTel Demo 2.2.0 logs captured via the collector fileexporter (workflow .github/workflows/capture-otel-demo-corpus.yml), business-service logs only (collector self-telemetry + load-generator filtered out), OTLP/JSON. Sizes 30 / 136 / 272 / 547 MiB — all below §8’s ≥ 1 GiB canonical benchmark minimum (this run is deliberately sub-minimum, to chart the trend, hence diagnostic). Hardware. ci-runner (hosted, ~4 vCPU) — not the §1 baseline, so deltas are indicative, not authoritative.

A1 — compression (target: ourios ≥ 3.0× zstd-19).

Scale series (ourios at the production ZSTD-3 default):

Codec sweep (v4 = 547 MiB, ourios ZSTD level varied):

A1 verdict: FAIL (target 3.0×; best observed 0.829). Both levers are bounded. Scale lifts the delta but plateaus ~0.78 (ourios asymptotes ~25×; zstd-19 is flat ~32× — the logs are locally repetitive, so zstd compresses them well at any size, not via a whole-corpus window). Raising ourios’s codec to ZSTD-19 adds only ~+0.07 and saturates by level 9. Even at equal codec strength, ourios stays ~17% larger than monolithic zstd-19: a structural cost of columnar Parquet (per-column/per-chunk framing, page indexes, row-group metadata, bloom filters) versus zstd-19 over one concatenated stream. That same chunking is what enables row-group skipping — so the ~17% space premium is the price of queryability, not an optimisation target. On pure compression of this corpus, ourios ≈ 0.83× zstd-19; the thesis rests on query performance (B1/B2), not on beating a byte codec.

C1 — reconstruction (target: 100% bit-identical or flagged lossy). PASS at every size: 1.0 reconstruct rate, ~1.1% of records flagged lossy (structured/kvlist bodies) and retained verbatim per CLAUDE.md §3.3.

C2 — template-count convergence (target: sub-linear). PASS (supportive). Templates grew 282 → 429 → 722 → 1322 while records grew 38k → 183k → 366k → 735k — sub-linear throughout. The formal gate abstains below 1 M lines (§3.4.3), but the curve shape is the strongest evidence yet for the template-mining premise.

Escalation (§7). One gate (A1) fails, on a size-non-representative corpus (all < §8’s 1 GiB minimum) and non-baseline hardware — so this is “corpus-specific,” not the two-gate pillar-level pause. C1 + C2 support the thesis. The production ZSTD-3 default is retained: the codec gain is small, saturates by level 9, and the residual gap is structural, so a higher default isn’t worth the ingest-CPU.

9.2 Results — 2026-06-11 (diagnostic, ci-runner) — A1 / C1 / C2 at ~1 GB

Corpus. corpus/otel-demo-v5 (1,042,274,219 B) and corpus/otel-demo-v6 (1,034,615,505 B) — same capture pipeline as §9.1, extending the scale series to ~1 GB (both within 4% of, but still just under, §8’s ≥ 1 GiB binary minimum). v6 was captured with the OTel Demo failure flags enabled (adFailure cartFailure productCatalogFailure), so it carries a real error band; v5 is an unflagged capture. Hardware. ci-runner — indicative, not the §1 baseline. Runs. bench.yml 27370641352 (v5), 27373716667 (v6).

A1 — compression (target: ourios ≥ 3.0× zstd-19).

A1 verdict: FAIL (target 3.0×). The scale series now reads 0.465 (v1, 30 MiB) → 0.666 (v2) → 0.725 (v3) → 0.758 (v4) → 0.828 (v5) / 0.824 (v6): the delta is size-driven and still rising, but decelerating — the crossover is not reached at ~1 GB, consistent with §9.1’s structural reading (ourios asymptotes ~26×; zstd-19 stays flat ~32×). v5 ≈ v6 shows the failure-flag error band does not perturb A1. This is the first A1 miss at (essentially) canonical size, so §9.1’s “size-non-representative” mitigation no longer applies; it remains a single-gate fail (no §7 two-gate pause), the §9.1 structural explanation stands, and the thesis-deciding counterpart — B1/B2 — now passes indicatively (§9.3). Whether the §7 corpus-specific tuning-RFC response triggers is a maintainer decision, sensibly taken once an authoritative baseline-8vcpu-32gib run confirms the number. (Resolved 2026-06-12: the §9.4 baseline run confirms — and slightly worsens — the deltas; the decision is now live with the maintainer.)

C1 — reconstruction (target: 100% bit-identical or flagged lossy). PASS on both: 1.000000 — v5 reconstructs 1,213,004 / 1,213,004 non-lossy rows exactly (lossy ratio 0.0114); v6 1,208,323 / 1,208,323 (lossy 0.0112).

C2 — template-count convergence (target: ratio ≥ 0.5 at 1 M lines). PASS on both — and for the first time on ≥ 1 M-line corpora, so the formal gate applies rather than §9.1’s abstention: v5 convergence ratio 0.756 (end count 1605, sample cadence 1336); v6 ratio 0.760 (end count 1606, cadence 1329).

9.3 Results — 2026-06-11 (indicative, ci-runner) — first B1 / B2 query readings

Corpus. corpus/otel-demo-v{4,5,6} (the §9.1 / §9.2 captures). The LogHub HDFS_v1 B2 arm did not run (fetch_hdfs off — memory-bound on the hosted runner), so only one corpus family has fed the query gates. Hardware. ci-runner — indicative, not the §1 baseline. Runs. query-bench.yml 27379085890 (B1 + the B2 structural metrics, after the effective-timestamp stack #178/#179) and 27357104694 (the prior run; its windowed / full-span latencies are quoted where noted). Recording. B1/B2 entries land in §9 per the maintainer’s 2026-06-12 authorization. RFC 0006 never reserved §9 (its §1 anticipated B1/B2 landing “in a follow-up extension PR once the querier is live” — RFC 0007); the workflow itself never writes §9 — every entry here is curated by hand.

B1 — predicate pushdown vs zstdcat | grep (target: ≥ 10× at 1 GiB). Query: severity ERROR, full corpus span. Run 27379085890:

Row counts agree exactly with the reference pipeline on both corpora. v4 is skipped: the unflagged 100-user capture genuinely contains zero error-band rows, so the predicate selects nothing.

B1 verdict: PASS (indicative) — both corpora clear the ≥ 10× bar at 3–4× margin, on the first real-corpus reading. Caveats, stated plainly: ci-runner, not the §1 baseline; the error bands are ultra-thin (11 / 28 rows — extreme selectivity is the friendliest case for pruning); both corpora sit just under the §8 1 GiB minimum. An authoritative baseline-8vcpu-32gib rerun (ideally with a denser error band) is required before this counts as the canonical B1 number.

B2 — template-exact latency ∝ result, not corpus. Windowed 1-hour template-exact query, result roughly constant as the corpus grows. Structural metrics (run 27379085890): scanned row groups stay flat at 1 — v4 1/5, v5 1/6 (17,632 rows, 1.86 MB), v6 1/6 (11,750 rows, 1.59 MB). Wall-clock (prior run 27357104694): windowed latencies sit in a flat ~3.4–4.1 ms band (v4 3.59 / v5 4.13 / v6 3.40 ms) while the full-span variant grows with corpus size (7.3 / 10.6 / 10.6 ms) — exactly the result-bound-vs-corpus-bound split the gate asks for.

B2 verdict: PASS (supportive, indicative) — the flat shape is confirmed on real corpora at ~1 GB; the formal target speaks above ~10 GiB, which remains unmeasured, and the second corpus family (HDFS_v1) hasn’t fed the arm yet.

RFC 0007 validated assessment. These are the measurements the RFC 0007 green → validated gate needs, but not yet in the form the ladder requires (§1 quotes must-win numbers against baseline-8vcpu-32gib): see the status note in docs/rfcs/0007-querier.md. The RFC stays green with a validated-pending note — authoritative baseline rerun required; denser error band and a second corpus family supporting. (Resolved 2026-06-12: the §9.4 authoritative run delivered the baseline rerun and the second corpus family (HDFS_v1); RFC 0007 is validated. The denser error band remains an open quality improvement.)

9.4 Results — 2026-06-12 (authoritative, baseline-8vcpu-32gib)

Corpus. corpus/otel-demo-v{1..6} (the §9.1 / §9.2 captures; 30 MiB → ~1 GB) for A1 / C1 / C2 and B1/B2’s OTel-Demo arms, plus — for the first time — the bench-time-fetched LogHub HDFS_v1 (§1; ~1.47 GiB plain text, 11,175,629 rows ingested across 5 files) feeding the B2 arm as the second corpus family. Hardware. baseline-8vcpu-32gib — the §1 baseline (8 dedicated vCPU, 32 GiB RAM, local NVMe-class SSD). These are the authoritative numbers the §1 methodology quotes must-win gates against; the §9.1–§9.3 ci-runner entries remain indicative history. Runs. Dedicated baseline host (no CI run id): one ourios-bench run per corpus (A1/C1/C2) plus one query-bench run (B1 + B2), executed 2026-06-11/12; raw logs retained by the maintainer. Recorded per the maintainer’s 2026-06-12 authorization.

A1 — compression (target: ourios ≥ 3.0× zstd-19).

A1 verdict: FAIL (authoritative) (target 3.0×; best observed 0.751). The delta is monotonic with corpus size and the crossover is unobserved, consistent with §9.1’s structural reading. One finding must be recorded honestly: the authoritative deltas sit below the ci-runner series (0.465 → 0.828) at every size — the ourios side compressed less effectively on this hardware (e.g. v5: 23.8× vs CI’s 26.3×) while zstd-19 stayed essentially stable (31.7× on both) — i.e. the ourios writer’s output is environment-sensitive (suspected row-group sizing / threading effects on the resulting encodings). That is now an open A1 investigation item alongside the structural gap itself. A1 gates the compression pillar (RFC 0006’s remit); the §7 escalation response is with the maintainer.

C1 — reconstruction (target: 100% bit-identical or flagged lossy). PASS (authoritative) on every corpus: 1.000000 throughout — v5 reconstructs 1,213,004 / 1,213,004 non-lossy rows exactly (lossy ratio 0.0114), v6 1,208,323 / 1,208,323 (lossy 0.0112); v1–v4 likewise 1.000000 (lossy 0.0097–0.0112). The formal ≥ 1 M-line gate passes on the baseline.

C2 — template-count convergence (target: ratio ≥ 0.5 at 1 M lines). PASS (authoritative) on both ≥ 1 M-line corpora: v5 ratio 0.756 (end template count 1605, sample cadence 1336), v6 ratio 0.760 (end count 1606, cadence 1329). v1–v4 abstain (< 1 M lines), as in §9.1.

B1 — predicate pushdown vs zstdcat | grep (target: ≥ 10× at 1 GiB). Query: severity ERROR, full corpus span. v4 is skipped (zero error-band rows, as in §9.3).

Row counts agree exactly with the reference pipeline on both corpora (11 and 28).

B1 verdict: PASS (authoritative) — both corpora clear the ≥ 10× bar at 2.5–3.4× margin on the §1 baseline. Remaining caveat, non-blocking: the error bands are still ultra-thin (11 / 28 rows — the friendliest case for pruning); a denser error band stays an open quality improvement.

B2 — template-exact latency ∝ result, not corpus.

Full-span template-exact (result grows with the corpus, so latency may too):

Windowed 1-hour template-exact (the gate’s shape: result roughly constant as the corpus grows):

The HDFS_v1 row is the first reading from the second corpus family (plain-text, the template-diversity case): the corpus is 8–15× the OTel-Demo row counts, yet the windowed scan still touches 1 row group (13 pruned) and stays inside the same flat latency band, while the full-span variant grows with the corpus (6.84 → 30.19 ms) — exactly the result-bound-vs-corpus-bound split the gate asks for.

B2 verdict: PASS (authoritative) — windowed ~10–28 k-row results answer in 4.2–5.9 ms (gate: ≤ 200 ms for ~10 k rows), flat from 735 k to 11.2 M rows across two corpus families. The formal target’s ≥ 10 GiB regime remains a future scale extension; the measured shape is the operative evidence.

RFC 0007 green → validated (resolved). The docs/verification.md §3 ladder reads: “Every thesis-gate in benchmarks.md §7 that the RFC’s pillars touch passes on representative corpora.” RFC 0007’s pillar is the query engine (pillar #3); its gates are B1 and B2, both now passing authoritatively on the §1 baseline over ~1 GB+ corpora including a second family. A1 does not gate RFC 0007 — it belongs to the template-mining/compression pillar, measured under RFC 0006. RFC 0007 is therefore flipped to validated (see its status note); accepted awaits maintainer sign-off per the ladder.

9.5 Results — 2026-06-13 (diagnostic, local unknown hardware) — A1 / C1 / C2 on HDFS_v1

Corpus. LogHub HDFS_v1 (Zenodo record 8196385, md5 76a24b4d…) — 11,175,629 lines, 1,577,982,906 raw bytes; fetched at bench time, never redistributed (query-bench.yml). The maximally-templated log corpus (a handful of templates over 11.2 M lines) — the single best case for the template-mining compression premise. Run via ourios-bench --gates a1,c1,c2 --parquet-zstd-level 19 --allow-unknown-hardware. Local hardware → diagnostic, not authoritative; A1’s verdict is corpus-structural and hardware-independent (compressed bytes are deterministic), C1/C2 are ratios, so the findings hold regardless of the runner.

A1 — the decisive finding (→ RFC 0011). A1 had only ever been measured on OTel-Demo (best 0.829×, §9.1/§9.4). HDFS_v1 is the corpus that should most reward template mining, yet A1 fails harder (0.516×): the more templated the corpus, the more completely monolithic zstd-19 captures its redundancy in one window (16×), while template mining’s extracted params (block IDs, timestamps, IPs) are high-cardinality columns that don’t compress as well and the columnar layout adds framing. The best case for template mining is the best case for the byte codec. So ≥ 3× over zstd cannot hold on any realistic log corpus — A1 is demoted to diagnostic and template mining’s compression value is recognised as logical/query-pruning (B1/B2), not on-disk bytes. See RFC 0011.

C1 + C2 — the miner pillar’s real gates, PASS on a representative corpus. At 11.2 M lines C1 is bit-identical (1.0) with a 4.6e-06 lossy ratio, and C2 plateaus at 40 templates with the formal gate applying (not abstaining, unlike the §9.1 sub-1 M runs). Under RFC 0011 these are RFC 0001’s validated thesis gates — both pass here. The authoritative baseline-8vcpu-32gib representative rerun (for the actual RFC 0001 validated flip) followed on 2026-06-14 (§9.6); as expected of deterministic verdicts, the numbers are identical.

9.6 Results — 2026-06-14 (authoritative, baseline-8vcpu-32gib) — C1 / C2 on HDFS_v1

Corpus. LogHub HDFS_v1 (Zenodo record 8196385, md5 76a24b4d…) — 11,175,629 lines, 1,577,982,906 raw bytes; fetched at bench time on the baseline host, md5-verified, never redistributed. Hardware. baseline-8vcpu-32gib — the §1 baseline (8 dedicated vCPU, 32 GiB RAM, local SSD), provisioned for this run and torn down immediately after. These are the authoritative C1 / C2 numbers for RFC 0001’s validated gates. Run. Dedicated baseline host (no CI run id): one ourios-bench --gates c1,c2 --hardware-kind baseline-8vcpu-32gib run at git 9a57ace; results JSON retained by the maintainer (2026-06-14T00-36-23.225Z-9a57ace.json). A1 was deliberately not run — it is diagnostic, not gating (RFC 0011); the §9.5 diagnostic A1 reading stands.

Authoritative confirmation. The verdicts match §9.5’s local diagnostic run bit-for-bit — expected, since C1 (reconstruction fidelity) and C2 (template-count convergence) are deterministic functions of (corpus, miner) with no wall-clock or hardware-sensitive component (contrast A1’s writer-environment sensitivity, §9.4). The value of this run is the authoritative hardware_kind stamp on the two gates that, under RFC 0011, define RFC 0001’s validated: both PASS on a representative ≥ 1 M-line corpus on §1 baseline hardware.

9.7 Results — 2026-06-15 (authoritative, baseline-8vcpu-32gib) — D2 / D3 / B2-post (RFC 0009 compaction)

Hardware. baseline-8vcpu-32gib — the §1 baseline (8 dedicated vCPU, 32 GiB RAM, local SSD), provisioned for this run and torn down immediately after. These are the authoritative D2 / D3 / B2-post numbers for RFC 0009’s validated measure (RFC0009.7). Run. Dedicated baseline host (no CI run id): the ourios-bench compaction bench at git 4d52288. Two invocations — the band-scale one-shot (OURIOS_COMPACTION_BASELINE=1, FILES=32, ROWS=4800, BODY_BYTES=4096) for D2/D3, then the b2-post-compaction criterion group. Synthetic (no corpus): D2/D3 drive one partition of 32 small files (~485 MiB) through compact_partition; B2-post queries 32-files-vs-1-file with the result set held constant.

Reading. D3 is the headline: a band-scale compaction lands its output squarely in the H4 256 MiB–2 GiB target with zero sub-128 MiB files — the small-file problem, eliminated. D2 shows consolidation runs at ~167 MiB/s on one partition/thread, far above any plausible per-partition seal rate, so a backlog drains (the “keeps up” property). B2-post quantifies the query payoff that motivated RFC 0009 (the PR #92 B2 finding that per-file footer/metadata reads dominate): collapsing 32 files → 1 cuts the footer reads ~6× on this query. The structural reductions (32 → 1 files / row groups, rows conserved) are hardware-independent and also pinned in ourios-parquet’s rfc0009_1_* / compaction_conserves_every_row tests; these wall-clock figures are the baseline-hardware stamp for RFC 0009’s validated. The full sustained-ingest soak (D2’s “backlog returns to zero in a one-hour window at D1’s rate”) and D1 itself remained unrun until §9.19 (2026-07-20: D2 soak PASS; D1’s per-core bar is the open reading) — the throughput here is the RFC0009.7 D2 measure, not that soak.

9.8 Results — 2026-06-18 (authoritative, baseline-8vcpu-32gib) — ingest write-path + recovery (criterion) and real-corpus A1 / C1 / C2 + B1 / B2

Hardware. baseline-8vcpu-32gib — the §1 baseline (8 dedicated vCPU, 32 GiB RAM, local SSD), provisioned for this run and torn down immediately after. Two such hosts (one per invocation set), both at git d3f2cae. Run. (a) the self-contained ourios-bench criterion benches ingest_write_path (RFC 0014) and recovery (RFC0008.3) — synthetic, no corpus — at full criterion settings; (b) the ourios-bench binary --gates a1,c1,c2 against two real corpora, plus the b1/b2 criterion benches (--warm-up-time 1 --measurement-time 3, matching query-bench.yml) over those corpora. Corpora: LogHub HDFS_v1 (Zenodo record 8196385, md5 76a24b4d… — 11,175,629 lines / 1,577,982,906 raw bytes (1.47 GiB) of real Hadoop production logs, above §8’s ≥ 1 GiB canonical minimum) and the frozen OTel-Demo v1 (corpus/otel-demo-v1, 38,782 lines / 31.5 MiB). HDFS is fetched in-job and never redistributed (§1).

(a) Ingest write path + recovery — supportive wall-clock (criterion).

Single-threaded micro-benches on synthetic records — supportive wall-clock (the structural sides are pinned by ourios-ingester’s RFC 0014 / ourios-wal’s RFC0008.3 tests), not gates. Dedicated hardware ran ~20–30% faster with much lower variance than the indicative ci-runner figures.

(b) Thesis gates A1 / C1 / C2 on real corpora.

C1 reconstructs every non-lossy row bit-for-bit across 11 M real production lines — the §3.3 invariant holds on real data at scale. C2 converges on HDFS (40 templates over 11 M lines; ratio 0.825 ≥ the threshold) — the template-mining thesis on a real corpus. A1 fails as expected: it is a recorded diagnostic, not a gate (RFC 0011) — template mining’s value is query pruning (B1/B2), not on-disk bytes beating a whole-stream codec.

(c) Query gates B1 / B2 on real corpora.

The windowed HDFS arm is the headline: a time-bounded query on the real 11 M-line corpus prunes 13 of 14 row groups via Parquet min/max statistics — the predicate-pushdown thesis (pillar #1) on real production data, ~5× faster than the unwindowed scan. (B1’s real-corpus arm skipped: OTel-Demo v1 has no error-band severity_text rows for the selectivity probe.) B1/B2’s structural pruning is the gate (pinned in ourios-querier); these are the baseline-hardware wall-clock stamp.

Not committed by the bench tooling — this is the curated narrative; the managed BENCH-RESULTS region above is for --update-benchmarks-md runs. The b1/b2 criterion timings use the reduced --warm-up-time 1 --measurement-time 3 (matching query-bench.yml); the structural pruning/template numbers are exact and criterion-setting-independent.

9.9 Results — 2026-07-03 (indicative, ci-runner) — B1 / B2 post-RFC 0022 (promoted attribute columns)

Purpose. The RFC 0022 §5 RFC0022.5 note: the promoted-attribute write path (per-key resource.<k> / attr.<k> columns + the two-arm predicate compile) must leave B1/B2 unchanged. This is the indicative re-run after RFC 0022 went green (#345–#348); the pruning counters are pinned structurally in crates/ourios-querier/tests/rfc0022_attr_columns.rs, this entry is the wall-clock stamp. Corpus. corpus/otel-demo-v4 (107,332 records → 735,377 mined rows / 5 files) and corpus/otel-demo-v5 (163,929 records, ~1.04 GB raw → 1,367,532 mined rows / 6 files). The LogHub HDFS_v1 arm did not run (fetch_hdfs off — memory-bound on the hosted runner). Hardware. ci-runner — indicative, not the §1 baseline. Run. query-bench.yml 28686650566 at git 6e3301b (the RFC 0022 green merge).

B1 — predicate pushdown vs zstdcat | grep (target: ≥ 10× at 1 GiB). Query: severity ERROR, full corpus span.

Row count agrees exactly with the reference pipeline. v4 is skipped as in §9.3 (its capture has no error-band rows).

B1 verdict: PASS (indicative), no regression — 34.8× against §9.3’s 40.0× on the same corpus, comfortably inside hosted-runner noise and 3.5× above the bar. Same caveats as §9.3: ultra-thin error band, corpus just under the §8 minimum, not the §1 baseline.

B2 — template-exact latency ∝ result, not corpus.

B2 verdict: PASS (supportive, indicative), no regression — the windowed latencies sit in the same flat few-ms band as §9.3/§9.8 while the full-span variants grow with corpus size, and everything is orders of magnitude under the 200 ms bar. The formal target speaks above ~10 GiB, which remains unmeasured on this runner class.

Assessment. The promoted-column machinery (extra column chunks per row group on the write side; the two-arm OR compile on the read side) shows no measurable drag on either gate. The RFC 0022 green → validated step still requires the authoritative baseline-8vcpu-32gib rerun per the standing bench policy (maintainer opt-in); this entry is its indicative precursor, curated by hand as in §9.3 — the workflow never writes §9.

9.10 Results — 2026-07-04 (authoritative attempt, baseline-8vcpu-32gib) — B1/B2 at 16 GiB: run blocked, miner finding

Purpose. The first run in the §8 10–100 GiB band: B2’s formal target speaks above ~10 GiB and had never been measured there. Corpus. LogHub HDFS_v2 (bench-time fetch, never redistributed): 31 files, 17,240,888,465 bytes ≈ 16.1 GiB raw, ~71 M lines of Hadoop daemon logs — the first corpus in our set whose shape (stack traces, multi-format node logs) differs qualitatively from HDFS_v1’s block events. Hardware. baseline-8vcpu-32gib, provisioned for the run and torn down after. Outcome: the run did not complete — it produced a product finding instead. The B2 store build was OOM-killed at 31.5 GiB RSS: the miner mints templates without bound on this corpus shape (template ids ≥ 56,199 by the 1.8 GiB subset mark, busiest template covering 0.67 % of 8.37 M rows; memory ~linear at ≈2× corpus bytes). Two bench-side pathologies were found and fixed en route — the eager corpus load (#350, now streaming: 1.3 GiB flat over hours) and a quadratic harness snapshot capture (#351, ~400× store-build speedup; gdb stacks exonerate the miner’s CPU path). RFC 0023 (bounded template memory) is the response; its RFC0023.7 criterion is this exact run completing.

What did land before the kill (recorded as diagnostic):

B2’s shape — flat windowed latency, window-driven pruning — holds wherever memory allows; the fragmentation itself (56 k templates, busiest at 0.67 %) also means pillar #2’s logical reduction fails on this corpus shape, which is the same finding from the pruning side. B1 did not reach its arms (stopped before the reference build once the OOM trajectory was clear). No gate verdict is claimed from this entry; the §8-band verdict waits on RFC 0023 + the rerun.

9.11 Results — 2026-07-04 (authoritative, baseline-8vcpu-32gib) — B1 / B2 at 16 GiB + RFC0023.7

Purpose. The §8 10–100 GiB band’s first completed measurement (the §9.10 attempt OOM’d), doubling as RFC0023.7 (bounded mining must complete this exact corpus under 8 GiB peak RSS) and the first B1/B2 readings at ≥ 10 GiB — where B2’s formal target speaks. Corpus. LogHub HDFS_v2 (bench-time fetch): 31 files, 17,240,888,465 bytes ≈ 16.1 GiB, 71,116,785 mined rows → 21 files / 80 row groups. B2 ran under the §3.3 Fixed severity baseline; B1 under the opt-in OURIOS_CORPUS_SEVERITY=log4j extraction (#350), stated per the methodology rule. Hardware. baseline-8vcpu-32gib, provisioned for the run, torn down after. Git 19e0886 (RFC 0023 bounds + telemetry merged).

RFC0023.7 — bounded mining at scale: PASS. Peak RSS 1.73 GiB across both benches’ store builds (5 s sampler), vs the §9.10 OOM at 31.5 GiB on identical input — an 18× reduction, under the 8 GiB bar with 4.6× headroom. Both benches completed (B2 phase 35 min; B1 including its zstd-19 reference build ~2.8 h).

B1 — predicate pushdown vs zstdcat | grep (target: ≥ 10× at 1 GiB, widening to ≥ 100× at 100 GiB). Query: severity ERROR, full 16 GiB span.

Row count agrees exactly with the reference pipeline.

B1 verdict: PASS (authoritative) — the ≥ 100× mark projected for 100 GiB is crossed at 16 GiB. With §9.8’s ~35–40× at ~1 GiB, the measured trajectory confirms the widening the target predicted: the reference’s cost grows with corpus bytes while Ourios’s grows with the matching row groups.

B2 — template-exact latency ∝ result, not corpus (formal target: ≥ 10 GiB, ≤ 200 ms for 10 k rows).

B2 verdict: PASS (authoritative, first ≥ 10 GiB reading) — the windowed query answers in the same few-ms band as the ~1 GiB corpora (§9.3/§9.8/§9.9): latency tracks the result, not the 71 M-row corpus.

The fragmentation datum (§9.10’s open question, quantified). The “busiest template” is id 0 — NO_TEMPLATE: under the default 20 k ceiling, ~79 % of HDFS_v2’s rows took the §6.3 parse-failure path (bodies retained bit-faithfully; observable via ourios.miner.parse_failure.reason, RFC0023.6). Template mining contributes little on this corpus shape — and the B1/B2 numbers above show the floor it degrades to (first-class-column + time pruning over Parquet statistics) still clears every gate. Follow-up noted: the B2 bench’s busiest-template picker should exclude NO_TEMPLATE so the full-span arm measures a true template-exact query on such corpora.

Assessment. RFC 0023’s §5 is fully discharged (this entry is the .7 record); the RFC flips red → green alongside this entry. The §8-band thesis verdict on real, hostile-shaped production logs: pruning compounds with scale (B1), result-bound latency holds (B2), and the mining-fragmentation failure mode is now bounded, observable, and priced.

9.12 Results — 2026-07-09 (indicative, local M-series) — otel-demo v8 capture: C1 / C2

The run is dated 2026-07-09; its C2 verdict was re-scored under the per-service gate on 2026-07-10 (#444 / RFC 0006 §3.4.3), so the resolution dates below post-date the heading.

Corpus. corpus/otel-demo-v8 (published GitHub release): a 48-hour OTel-Demo 2.2.0 capture at 150 locust users with the adFailure + paymentFailure feature flags active — 690,355 OTLP LogsData batches / 4,948,596 log records / 2.96 GB uncompressed, the largest and most hostile real capture to date (deliberately injected failure modes, multi-service, long-horizon). Calibration manifest at testdata/calibration/otel-demo-v8.json (RFC 0024 §3.1).

C1 — bit-identical reconstruction: PASS, perfect. The corpus holds 4,948,596 records (the calibration manifest’s count); 17 of them (all kafka, 0.0003 %) took the §3.3 lossy-flag path with their bodies retained, and C1 = 1.000000 over the remaining 4,948,579 rows — the honesty contract holds at 4.9 M rows through failure-mode churn.

C2 — template-count convergence (bar: ratio ≥ 0.5 at 1 M lines, evaluated per service since #444): PASS. Under the per-service gate (RFC 0006 §3.4.3, amended 2026-07-10) the corpus passes: the only service that clears the 1 M-line evaluation floor is cart, which converges at ratio 1.000 with two templates. Every other service abstains for want of volume; the whole-corpus ratio (0.199, end template count 14,631, sample cadence 4,833) is retained below as a diagnostic — it is a category error to grade a multi-service corpus as one Drain stream (§3.4.3 rationale). The per-service decomposition (splitting on service.name and re-running the gates per service) localises the whole-corpus fragmentation completely:

The gate folds over the gated services (those ≥ 1 M lines): cart is the sole such service and it passes, so the corpus passes. cart clears the formal gate at 2.76 M lines with two templates; the smaller services abstain below the 1 M-line floor, so they are not graded — though their observed counts (1–17 templates over 0.5–1.0 M lines) sit at the same near-flat convergence. The kafka broker, also abstaining, is the outlier: it mints 14,608 templates on 2.8 % of the lines. Mechanism (measured): kafka’s cleaner logs emit 3-token lines whose third token is a unique offset-bearing path (Deleted log /tmp/kafka-logs/…/00000000000000000429.log.deleted., 11,651 distinct) — one varying token in a 3-token line is similarity 2/3 ≈ 0.67, below the strict 0.7 threshold (§3.1 no-silent-merges), so each line mints a template; the 4-token siblings of the same family (0.75) merge fine. The failure-flag confound turned out to be a red herring. #444 settled how to handle the fragmentation (2026-07-10, maintainer-approved): of the three options — tokenizer masking, length-aware thresholding, and accept-and-scope-C2-per-service — option 3 shipped (the per-service gate, RFC 0006 §3.4.3, PR #451); masking is parked as a future strategic RFC (no commitment; a Collector transform or redaction processor can polish high-cardinality infra tokens upstream) and length-aware thresholding was rejected. The safety story held throughout (bounded memory per RFC 0023, per-service C1 perfect).

The per-service decomposition is now the first-class bench gate (ourios-bench --gates c2 prints it whenever any service bucket exists — distinct service.name values plus any <unknown>/<other>, so a single-service or plain-text corpus shows its one gated row too); template creation is a globally-monotonic event attributed to the minting service, so per-service creations partition the whole-corpus count exactly (2 + 17 + 1 + 3 + 14,608 = 14,631) in O(services) memory — no per-service id set. As of #444 (option 3) this decomposition is the gate: C2 is evaluated per service and folds over the services that clear the 1 M-line floor, with the whole-corpus ratio kept as a diagnostic (RFC 0006 §3.4.3).

What the fragmentation actually costs — B2 pricing (indicative, local M-series). Running the B2 windowed query on the fragmented (kafka) vs. converged (cart) service isolates the impact:

The deployed time/column pruning floor is identical whether a service has 2 templates or 14,608 — a 1 h window prunes 48 of 49 row groups either way (reconfirming the RFC 0023 graceful-degradation result on a fresh corpus). Fragmentation does not cost query latency or pruning. What it costs is template-exact query precision: probing cart’s dominant template (id 1 in this run — a run-specific identifier, not a canonical one) recovers 1.78 M / 2.76 M rows (one template is most of the corpus) but only 11,523 / 136,790 on kafka, because kafka’s dominant event is scattered across ~11,651 ids — a single template_id probe recovers only that one id’s slice (11,523 rows), not the full dominant event. So the fragmentation is a query-capability / thesis-value tradeoff, not a performance one; the pruning path degrades to the first-class-column floor unharmed. #444 accepted that tradeoff on hostile infra logs: the per-service gate makes C2 acceptance honest without masking, and any future masking is deferred to an upstream Collector processor or a dedicated RFC.

9.13 Results — 2026-07-12 (indicative, ci-runner) — RFC 0031 comparative program vs Grafana Loki (runs #8–#18)

Purpose. The first recorded numbers for the RFC 0031 comparative program — Ourios against Grafana Loki, the incumbent CLAUDE.md §1 defines the project against. These are the §7 calibration inputs the RFC’s open questions ask for, not gate verdicts: the L-gate margins are the RFC’s proposed values (M_L1..M_L4 = 10, F_L6 = 3, wired as ComparativeMargins::default()), the §5 gate scenarios (RFC0031.2–.11) are still red stubs, and the harness reports each pair under its provisional margin rather than asserting it. Every “PASS”/“fail” below is provisional pending the §7 freeze — a maintainer step; the open inputs are enumerated in point (4) of the closing Assessment.

Corpus. corpus/otel-demo-v8 (the §9.12 capture): 4,948,596 log records, 2.96 GB uncompressed — the RFC 0031 §3.3 headline corpus (real OTLP, failure flags active, kafka fragmentation and all). Both systems ingest the identical OTLP stream; an OTLP partialSuccess in any push response fails the run, so neither side can silently drop lines. Reference system. grafana/loki:3.5.3, digest-pinned (sha256:3165cecce301ce5b9b6e3530284b080934a05cd5cafac3d3d82edcb887b45ecd), single-binary mode, fed over its native OTLP endpoint. Flag deviations from stock are documented below — all ingest-replay accommodations, all in Loki’s favour, per the §3.7 anti-strawman commitment. Hardware. ci-runner — indicative, not the §1 baseline; the authoritative baseline-8vcpu-32gib run remains a maintainer opt-in per RFC 0031 §3.2. Bytes-read, the primary channel, is CPU-insensitive by construction, but nothing here is quoted as authoritative. Runs. comparative-bench.yml dispatch runs (curated by hand as ever — no workflow writes §9), each with one harness delta under test. Counted runs are equivalence-gated passes over the full corpus; the two diagnostic failures (#11/#13) are listed with exactly what they carry:

In every counted run, RFC0031.1 result-set equivalence held on every pair: the two systems’ answers, keyed (timestamp_unix_nanos, body_bytes), were multiset-identical at 4.9 M-record scale. Runs #11/#13 were L3-flicker diagnostics (an ingester-visibility artifact, fixed in #490 — see the deviations list); their table rows above note exactly what each carries. Every dispatched run appears in the table, and the per-class tables below carry a row for every run in each quoted streak (L1: #15/#16/#17; L3: #14/#15/#16/#17), so the streaks audit from this entry alone.

The metric (§3.6 as amended 2026-07-12). The Ourios figure is the total bytes fetched from object storage per query: count scan + row materialization + template-registry derivation. Loki is reported on two channels: storage-side (query-stats compressedBytes + headChunkBytes — the conservative apples-to-apples counterpart of Ourios’s fetched compressed-Parquet bytes; the harness evaluates gates primarily on this) and totalBytesProcessed (decompressed engine-side work, which overstates Loki’s storage reads by the chunk compression ratio; reported as context). Which channel the frozen §7 gates ride is an open maintainer decision.

Program history — the biased ruler, retired. Runs #5–#7 predate the §3.6 measurement-fidelity amendment and measured the Ourios side as the count scan alone (e.g. run #7’s severity figure of 609,498 B and its “146.9×”-style ratios), silently excluding the row-materialization and registry IO while Loki’s counterpart figure includes delivering results. Those runs are program history only and are not citable; every number below is on the honest total.

L1 — template-exact lookup (must-win, the flagship class): provisional PASS, widest margins. Pair: template_id == 4323 (2 rows) vs the LogQL line-filter needle "Updated connection-accept-rate max connection creation rate to" over every stream — the picker proves the two select identical row sets before the pair counts. Loki has no template concept, so its honest equivalent is a substring scan of the whole corpus; Ourios rides the writer’s existing bloom filter on template_id.

Above the provisional M_L1 = 10 on both channels, in every run since the pair landed (third consecutive pass at #17). The Loki side is structural: no template id → nothing to prune with.

L3 — trace correlation (must-win, OTLP-native): provisional PASS after blooms. Pair: every log line for one trace_id (9 rows). trace_id is high-cardinality by construction, so it cannot be a Loki label (§3.3’s machine-checked disallowlist); Loki’s honest query is a structured-metadata filter over all streams.

Run #12 is the honest before-picture: without blooms Ourios itself had to fetch the trace_id column corpus-wide, and the storage-side ratio (1.41×) was nowhere near the margin. The blooms (implemented in #489; the RFC 0005 §3.6 amendment recording them, with this as its measured evidence, is #491) collapse the fetch 15×, and the pair has now passed the provisional margin on both channels three runs in a row. As with L1, Loki’s side is structural: a trace cannot be pre-narrowed to a label stream, so it scans and decompresses everything in the window.

L2 — severity predicate (must-win family): parity-plus storage-side, ~33× processed — not a provisional 10× pass. Pair: lowest-volume single-severity_text band on the highest-volume service, full corpus span, 1 row. The run series doubles as the read-path optimisation ledger (component split: count scan + materialize + registry):

(Run #8’s Loki side: 2,880,784 storage / 89,184,711 processed.) Across the later reproductions the storage-side ratio sits at 1.05–1.31× and processed at ~33–39×, the spread being entirely Loki-side wobble (below). Reading: on the honest metric Ourios went from losing the storage channel (0.67×) to parity-plus via two read-path fixes, and wins decisively on engine work — but this is not a 10× storage-side pass, and no amount of wobble makes it one. The remaining named levers: the constant 513,862 B template-registry derivation, 20–29 % of every small-answer query’s total (the RFC 0033 cached-template-map candidate), and write-side page/row-group sizing.

Time-window browses (L6 floor family): published loss on the storage channel. Pairs: all lines of the highest-volume service in a clean k-row window (the promoted-column bloom’s worst case), plus run #17’s diagnostic — the same shape scoped to the lowest-volume service (“ad”, ~34 s window), where the service.name bloom could in principle skip. Floor gate as reported here: a bytes-read floor analog (Ourios ≤ 3× Loki, i.e. ratio ≥ 0.33) — the harness applies the §7 F_L6 factor to this entry’s bytes channels. Note the §5 gate as written (RFC0031.7) defines the L6 floor on latency p50 — measured in run #18 (see the latency section below), where the gate as written passes on all three window pairs; the bytes framing here remains the conservative reporting channel pending the §7 freeze.

This is the honest loss the RFC’s L6 disposition anticipated, and it is published as §5 RFC0031.11 demands: on a browse-k-rows query Loki reads only the tiny chunk slice its label stream + time index point at, while Ourios pays fixed per-query costs (the registry constant plus row-group-granularity materialization) that dwarf a k-row answer. The #486 late-materialization fix halved the loss and lifted k=2000 past the processed floor; storage-side stays 0.007–0.018 vs the 0.33 floor on current code. Run #17’s diagnostic sharpens the why: scoping to a low-volume service improves Ourios only ~22 % and flips the processed floor to pass, but there is no bloom collapse — v8’s hour partitions each hold roughly one row group containing all services, so the promoted service.name bloom has nothing to skip. The tier-changing lever is write-side layout (service clustering / row-group sizing — hazard #4 territory, an RFC-level change), not query-side tuning.

Latency (§3.6 channel, run #18 — the program’s first). Median of 7 warm repetitions per pair per system, measured only on correctness-verified pairs; Ourios timed in-process, Loki over localhost HTTP (negligible at these magnitudes; stated because latency is corroborating, not sole-gating):

Two findings this channel settles. First, the young-engine latency risk the RFC hedged against (“a latency loss + bytes-read win = sound architecture, young implementation”) did not materialize: Ourios answers every pair in 39–86 ms — a flat, fixed-cost-shaped profile — while Loki spans 13.8 ms to 24.1 s, and on the needle classes the wall-clock gap is interactive-vs-batch (75 ms vs 23–24 seconds). Second, scenario RFC0031.7 evaluated as written — on latency — PASSES on all three window pairs (0.34, 3.43, 1.32, all ≥ 1/3 at F_L6 = 3), and Ourios is outright faster on two of the three; the storage-channel loss published above is real as a bytes statement, but the RFC’s own L6 gate holds the floor. Which channel the frozen L6 gate uses is part of the §7 decision.

Determinism note. For repeated measurements of the same build and configuration, Ourios’s bytes are byte-identical (the store build is deterministic) — differences between runs are exactly the harness/optimisation deltas the table names, which is what lets the run series read as an optimisation ledger. Loki’s storage-side figure wobbles run to run (severity pair: 2.67–3.35 MB) with chunk boundaries and flush timing; ratios quoted against Loki carry that band.

Documented Loki flag deviations (all in Loki’s favour, per §3.7). The committed harness starts Loki with, and comments, exactly these deviations from stock:

• -validation.reject-old-samples=false — the frozen corpus is weeks old; stock Loki would reject the replay outright.
• -querier.query-ingesters-within=0 — stock Loki (default 3 h) skips ingesters for queries over weeks-old ranges, making rows still in unflushed low-volume chunks invisible (the run #11/#13 L3 flicker; diagnosed via ingester.totalReached: 0, fixed in #490). Disabling the cutoff means ingesters are always consulted — without it Loki’s answer to an old-range query is silently incomplete.
• Raised ingestion + per-stream rate limits (-distributor.ingestion-rate-limit-mb=512, -distributor.ingestion-burst-size-mb=1024, -ingester.per-stream-rate-limit=512MB, -ingester.per-stream-rate-limit-burst=1GB) — replay is far faster than the capture’s real-time rate.
• Raised internal gRPC message caps (-server.grpc-max-recv-msg-size-bytes=16777216, -server.grpc-max-send-msg-size-bytes=16777216) — runs #2–#4 failed on the same ~5.27 MB internal message regardless of our outer batch size: a single kafka-service LogsData line’s content alone inflates past Loki’s stock 4 MiB internal cap. Raising it (standard operator tuning) lets Loki accept the data at all, preserving the identical-ingest precondition the equivalence check requires.

Assessment. (1) The two classes the thesis stakes itself on hardest — L1 template lookup and L3 trace correlation — pass their provisional must-win margins on both channels, reproduced across three consecutive runs, and in both cases Loki’s cost is structural rather than tuning: no template concept, and no way to index a trace id. (2) L2 is parity-plus on storage and a ~33× processed win, honestly short of a 10× storage claim, with two named levers still on the table. (3) The window browses are a published storage-channel loss whose mechanism is understood (fixed per-query costs vs v8’s one-row-group-per-hour layout); the lever is write-side and RFC-sized. (4) Nothing here is frozen: the §7 inputs — the primary metric channel (storage-side vs processed), the must-win margins and floor factors, and whether the time-window pairs reclassify from gated floor to diagnostic — are open maintainer decisions, and this entry is the calibration evidence for them, not their resolution.

9.14 Results — 2026-07-13 (indicative, ci-runner) — comparative run #20: frozen gates on main, RFC 0033 acquisition

First dispatch on main after the §7 partial freeze and after the RFC 0033 cached template map merged (#511–#513). Job: run #20 (29255000054), exit 0.

Frozen gates. All asserting gates pass on main — M_L1/M_L3 storage margins and the F_L6 latency floors held; equivalence held on every pair. The dispatch is functioning as the regression gate the freeze intended (run #19 proved it on the branch; this run proves it on main).

RFC 0033 acquisition (the run’s purpose). Every pair reports:

template-map acquisition (RFC 0033): cold (audit fold, 513862 B; no artifact published)

• The registry component is byte-identical to run #8’s baseline (513,862 B constant per body-rendering query): the cache regressed nothing, exactly as the advisory design promised.
• But the write-through never published on this corpus, so no pair ever ran warm and the RFC0033.6 corpus gate (warm/cold ≤ 1/10) could not be measured.
• The explanation consistent with the run’s outputs is §3.2’s size abstention: the artifact is uncompressed JSON carrying every (template_id, version) canonical template string, while the 513,862 B it must undercut is zstd-compressed Parquet of the same strings (plus their event history). On v8’s template set the JSON evidently meets or exceeds the fold, and the guard refuses a publish that would make warm acquisition cost more bytes than the fold it replaces. (A publish IO failure would leave the same “no artifact” label; the §3.7 publish-outcome telemetry distinguishes the two in a served process, but the bench harness does not export metrics — the amendment run should print the outcome explicitly.)

Consequences recorded.

1. RFC 0033 status reverted green → red (this PR): RFC0033.6’s corpus arm is undischarged. The local-shape arm (55.8× on the 64-event fixture) stands.
2. M_L2 stays frozen-deferred — §7’s unfreeze condition (the RFC 0033 warm measurement on the headline corpus) was not met.
3. The lever is an artifact encoding amendment (format_version 2, compressed body). The same template strings zstd-compress into the 513,862 B audit Parquet with full event history alongside, so a compressed artifact is expected to land well below the fold size — to be measured, not assumed. Abstention semantics stay: publish only when the artifact beats the fold.

9.15 Results — 2026-07-14 (indicative, ci-runner) — comparative run #21: the v2 compressed artifact publishes and runs warm

Dispatched from the RFC 0033 v2 implementation branch (PR #522, the measure-before-merge step). Run 29343438434.

The RFC 0033 answer. The zstd artifact published on the corpus (no abstention — the run #20 ambiguity is resolved by the new per-pair outcome labels), and every measured pair ran warm:

template-map acquisition (RFC 0033): warm (one artifact GET, 187904 B compressed)

• warm = 187,904 B (the compressed artifact, GET cost) vs cold = 513,862 B (the audit fold, byte-identical to run #8) — warm/cold ≈ 1/2.73, a ~326 KB cut off every body-rendering query’s honest total.
• The original RFC0033.6 ratio gate (≤ 1/10) does not pass on this corpus: the artifact is O(live template state), the fold is O(audit history), and otel-demo-v8 is young — the amended gate (≤ 1/2, dated 2026-07-14 in the RFC) asserts the real margin and ages upward. See the §5.6 amendment for the full argument.

The test failure is not an Ourios finding. The run exited 1 on one pair: loki returned 0 of 9 expected rows for [trace correlation, L3] before timeout — the Loki-side low-volume-chunk race (the run #12-era flicker), resurfacing on the shared runner despite the #490 flag fixes. All other pairs measured; the report and every RFC 0033 number printed before the panic. A rerun for a clean L3 pair is queued as run #22.

9.16 Results — 2026-07-14 (indicative, ci-runner) — runs #22 and #23: the v2 artifact asserting, M_L2 unfrozen

Two dispatches after the RFC 0033 v2 merge (#522):

• Run #22 (29352282162, from main): exit 0 — the clean-record run. All then-frozen gates passed, the L3 pair measured cleanly (run #21’s Loki-side flake did not recur), and every pair ran warm on the compressed artifact.
• Run #23 (29353634499, from the M_L2-unfreeze branch): exit 0 — the first run with the full assertion set live. L2 processed (PRIMARY, frozen 10) 43.97×; L2 storage-side floor (frozen 11/10) 1.49×; L1 storage 108.3× and L3 storage 24.9× against their frozen 10s; latency floors held; and the RFC 0033 §5.6 acquisition gate asserted warm = 187,905 B compressed on every pair against the 513,862 B fold (ratio ≈ 1/2.73, gate ≤ 1/2).

With #528 merged, §7’s measurable gates (M_L1, M_L2, M_L3, F_L6) are all enforcing on every comparative dispatch; M_L4/F_L7 stay deferred until measured. RFC 0033’s §5 is fully discharged: the corpus arm passed as measured (#21), and passed again as an asserting gate (#23) — the status flips red → green with this record.

9.17 Results — 2026-07-17 (indicative, ci-runner) — L4 frequency aggregation measured (PR #536 arc)

The last unmeasured must-win class. The L4 workstream’s own dispatch sequence (~23 real comparative-bench runs across the arc — a numbering distinct from §9.16’s) fixed three genuine harness bugs early (LogQL escaping, a control-flow ordering bug, a missing picker row ceiling), then spent the balance of the runs on a persistent completeness shortfall that no harness-side fix closed: Loki never returned 100% of any L4 candidate’s expected rows on this corpus. Every mechanism checkable from the harness side was ruled out directly — exact (timestamp, body) ingester dedup (corpus analysis found zero collisions), push-path drops (partial_success asserted clean on every push), Loki’s own warn/error logs (silent), and its loki_discarded_samples_total accounting (zero, of any kind). The residual matches open upstream grafana/loki#10658 (wide-time-range queries silently missing a small percentage of lines, no maintainer-identified root cause). RFC 0031 §7 records the resulting amendment: L4_COMPLETENESS_MARGIN = 0.90, checked per group_key with phantom-cell and per-key-overcount hard-fails — the full five-iteration comparator design trail lives there.

The measured pair (picker floors L4_MAX_ROWS = 100_000, L4_MIN_AVG_INTERVAL_SECONDS = 100 — lower-frequency candidates measure more completely; mechanism uncharacterized, NOT dedup): template_id=60 (Periodic task <type> generated), param(0), bucket(12h), 1,197 expected rows, group cardinality 4.

Ourios’s side is constant at 47,995,205 B total (the honest §3.6 metric). Four consecutive equivalence-verified measurements in a 3.69–3.73× / 86.5–87.1× band: the shape mirrors L2 pre-freeze — a strong processed-channel win with storage closer to parity. M_L4 stays §7-deferred (both channels reported, nothing asserted); the proposed freeze shape on #498 is the L2 precedent — processed-channel must-win at 10× plus a storage-side floor (L2’s is frozen at 1.1×; L4’s measured 3.69–3.73× storage band would clear a similar floor with real headroom).

Follow-on hardening, so the 2 h dispatch confirms rather than discovers (#538/#499, closed via #539–#542): a proptest property suite over the margin comparator (its properties were verified by manually re-introducing the three historical comparator bugs and confirming each is caught — the evidence trail is PR #539’s record, not a standing mutation-testing harness), per-pair completeness recorded as a machine-readable artifact on every dispatch, a backdated wide-time-range arm in the per-PR loki-interop job running the dispatch’s exact Loki flags (one shared constant — config drift between the 1-minute test and the 2 h run is now unrepresentable), and a dispatch class filter for targeted re-runs.

9.18 Results — 2026-07-18 (indicative, ci-runner) — the M_L4 freeze’s first asserting run

Workflow run 29659514874, dispatched from the freeze branch (PR #548) with the L4 gates enforcing for the first time — M_L4 = 10 on the processed channel (primary) plus the 1.1× storage-side floor (m_l4_storage_floor_tenths = 11), the L2 shape per the §7 decision (maintainer, 2026-07-18). Exit 0; every frozen gate Decided { pass: true }:

L4 completeness this run: 1151/1197 = 96.2% — inside the §7 margin and the §9.17 band (95.3–97.5%). (Latency-floor advantages are oriented loki_p50/ourios_p50 — above 1 means Ourios faster; the floor passes at ≥ 1/3.) With this run the dispatch asserts every §7 value except F_L7 (deferred until L7 is first measured): an L4 band degradation below either frozen value now fails the run instead of printing a smaller ratio.

9.19 Results — 2026-07-20 (indicative, ci-runner) — first D1/D2 sustained-ingest soak (#558)

Purpose. The first run of the RFC 0009 D1/D2 soak harness (#558, ourios-bench soak + soak-bench.yml): the full in-process pipeline — IngestPipeline group commit (100 ms WAL batch window) → seal → sweep → compact — under one hour of paced synthetic OTLP load. Record timestamps ride a ×60 synthetic clock and the same synthetic “now” feeds the sweeps, so hour-sealing exercises continuously; D1’s ack latencies are wall-clock. Workflow run 29717165102; the JSON report is the run’s artifact. Hardware: ci-runner (4 vCPU) — indicative, not the §1 baseline.

Numbers (release build, defaults: total offered load 100,000 lines/s — the paced target, not D1’s per-core bar — batch 1,000, 4 workers, 10 s sampling):

Reading. D2 is a clean pass: compaction kept pace with a full hour at target with a backlog that never exceeded one partition. D1 splits: the machine sustained the 100k lines/s target with ack p99 inside the bar, but normalized per core (÷4 workers on 4 vCPU) it lands at ~25k lines/s/core against the ≥ 100k/core bar. Open reading for the maintainer: whether the bar means single-core-scaled throughput (then this is a real 4× efficiency gap to close) or per-node throughput on baseline hardware (then the authoritative baseline-8vcpu-32gib run decides). The commit path’s fsync overlap needs ≥ 2 threads, so a literal 1-worker measurement under-credits by construction. Either way the harness now measures instead of guessing, and the number is honest: no bar was reworded to fit the result.

9.20 Results — 2026-07-20 (authoritative, baseline-8vcpu-32gib) — D1 capacity probes: the single-tenant ceiling

Purpose. §9.19 left D1’s per-core bar as an open reading and noted the run had paced at the target rather than probing capacity. These are the first capacity probes, on the §1 baseline class (8 dedicated vCPU / 32 GiB) — baseline runs are maintainer opt-in per RFC 0031 §3.2, and this one was. Ad-hoc VM run (not a workflow dispatch): ourios-bench soak at 0979d14, release build; per-probe JSON reports retained by the maintainer alongside this record’s source run log.

Ladder (10-minute probes, 8 workers, one tenant, ×60 synthetic clock):

The identical ~86k plateau at every offered rate, with ack latency pinned at ~6 s, is a saturated pipeline: at saturation the harness’s in-flight bound (a 512-batch semaphore in the load loop, #558) sets the latency (512 batches × 1,000 lines ÷ 86k lines/s ≈ 6 s) while the plateau itself is the pipeline’s service rate — raising the permit count would lengthen the queue, not the throughput, because the per-tenant miner hand-off is in-order and sequential by design (CLAUDE.md §3.7 per-tenant trees; the least-common-mechanism choice’s deliberate flip side). Stated carefully: ≈ 86k lines/s is the observed single-tenant ceiling under this probe configuration (batch 1,000, 8 workers, this corpus shape), consistent with the sequential-miner explanation; a different batch size or record mix could move the number some, but not onto a cores axis.

Tenant-parallel check (8 concurrent single-tenant soak processes, 2 workers each, 5 minutes at offered 100k each — an approximation: separate WALs/stores per process): 41,895–43,335 lines/s each, ≈ 341k lines/s aggregate — ~4× the single-tenant ceiling on the same box. Node ingest capacity scales with tenant parallelism, not core count.

Reading. The strict per-core bar (≥ 100 000 lines/s/core) is not merely uncalibrated — for single-tenant load it measures a dimension the architecture deliberately does not scale on. D1’s falsifier (“a meaningful share of production traffic per node”) is a per-node, multi-tenant statement: approximated here at ≈ 341k lines/s (≈ 29 B lines/day) — a multi-process stand-in, not yet a per-node measurement (separate WALs/stores neither share a commit stream nor contend on one store); the honest in-process --tenants N measurement is tracked in #567. D2, by contrast, passed at full saturation on every probe. The bar recalibration (per-node multi-tenant must-win + per-tenant ceiling as recorded diagnostic, the RFC 0011 must-win/diagnostic precedent) is a pending maintainer decision; until it lands, D1 stays FAIL-as-written and this record is the evidence, not the verdict.

9.21 Results — 2026-07-20 (authoritative, baseline-8vcpu-32gib) — in-process multi-tenant ceiling + the serialization profile

Purpose. The §9.20 tenant-parallel figure (≈ 341k lines/s) was a multi-process approximation flagged as such; #567’s in-process --tenants N mode (merged #570) is the honest instrument. Ad-hoc VM runs at 9ad3158-era code; the per-run JSONs and the profile artifacts are retained by the maintainer outside the repository (the gitignored local scratch/ tree), as with every ad-hoc VM record in this series.

Finding 1 — node capacity is FLAT across tenants. 10-minute saturating soaks, 8 workers, one shared WAL/commit stream: 1 tenant ≈ 86.0k, 8 tenants 85.8–86.2k (three offered rates), 16 tenants 87.3k lines/s. The multi-process 341k was ~4× optimistic precisely because separate processes had separate commit streams.

Finding 2 — the ceiling is software serialization, not hardware. Profile at saturation (flamegraph + per-thread pidstat): 1.2 of 8 cores busy — ~85% idle; no thread above ~33%. Root cause (crates/ourios-ingester/src/receiver/pipeline.rs:314–354 at that commit): the global WAL-seq gate + the global miner mutex serialize all tenants, with the miner match AND the sink emit (including size-triggered Parquet encode + store put — I/O) inside the single-file section. Issue #571; design → RFC 0035 (specified). D2 passed at full saturation in every run.

9.22 Results — 2026-07-20 (authoritative, baseline-8vcpu-32gib) — RFC 0035 Design A prototype A/B

Purpose. The RFC 0035 §6 pre-implementation measurement: main (9ad3158 lineage) vs the Design A prototype (rfc0035-prototype 987b781 — ordered mining under the gate, sink emit + triggered publish moved to a bounded concurrent pool, crude quiesce barrier). Same VM, back-to-back 10-minute saturating soaks (8 tenants, offered 800k, 8 workers). Artifacts retained by the maintainer outside the repository (gitignored local scratch/ tree).

Design A multiple on the baseline class: 1.61× (implied residual serial fraction ≈ 0.62 — the ordered miner phase + WAL group commit). Honest note: this is below the M-series indicative 1.82× — the prediction that the baseline multiple would land higher was wrong; the EPYC’s slower single-thread makes the still-serial ordered phase relatively costlier. Saturation ack latencies are queue-bound in both arms (the in-flight bound), per §9.20’s reading — the p99 bar applies at sustained rates below capacity. This 132k lines/s/node figure is the §6 input to RFC 0035’s target and to the RFC 0034 D1 recalibration; whether Design B (§4) is ever escalated is judged against the recalibrated bar, not the old per-core one.

9.23 Results — 2026-07-21 (authoritative, baseline-8vcpu-32gib) — the RFC0034.2 / RFC0035.4 asserting soak

Purpose. The asserting run for the recast D1 bar: RFC 0034’s RFC0034.2 (per-node must-win with the observable below-saturation condition) and RFC 0035’s RFC0035.4 (the serialization is actually relaxed) in one measurement — the RFC 0034 §7 one-run-two-records question resolved as one run, one record, cited by both. Ad-hoc VM run at main d2c622e (the RFC 0035 production implementation, #577, plus its review fixes, #579), release build; the JSON report is retained by the maintainer outside the repository (gitignored local scratch/ tree), as with every ad-hoc VM record in this series.

Shape (the RFC 0034 §3.1 asserting shape): one-hour soak, soak --tenants 8 (N = cores), offered exactly 100,000 lines/s — the bar rate, paced, not a saturating probe — batch 1,000, 8 workers, ×60 synthetic clock, 30 s sampling.

Reading. The recast D1 must-win asserts for the first time and passes: 0.08% pacing loss at the bar rate is achieved ≈ offered, which is the below-saturation proof, and the p99 (153.63 ms) is measured over that same run, inside the 200 ms bar. The p50 rides the 100 ms group-commit window (the CLAUDE.md §3.4 batch default) — ack latency at this rate is floored by batched fsync, not queue-bound. The 723.70 ms max is a tail spike outside the bar’s percentile; the bar is p99, and it holds. One honesty note: the harness’s printed per-core “FAIL” line is the pre-RFC-0034 bar mechanically applied — per RFC 0034 that number (12,490 lines/s/core) is now a recorded diagnostic, and the recalibrated per-node must-win is what judges the run. This run satisfies RFC0034.2 and RFC0035.4 simultaneously: the production Design A sustains the recast bar with margin to spare over the pre-RFC ~82k saturation baseline (§9.22), with D2 passing over a full hour of sustained multi-tenant load.

9.24 Results — 2026-07-21 (authoritative, baseline-8vcpu-32gib) — first authoritative comparative run: all frozen gates pass

Purpose. The RFC 0031 comparative program’s move from indicative to authoritative — the last open decision on the #498 scoreboard. Every number in the §9.13–§9.18 series was measured on ci-runner and labelled indicative; per RFC 0031 §3.2 the baseline-hardware run is a maintainer opt-in, and this one was. Ad-hoc VM run on the §1 baseline class (8 dedicated vCPU / 32 GiB), not a workflow dispatch, but an exact replica of the comparative-bench.yml dispatch recipe at main 9deecb1: frozen corpus/otel-demo-v8 (4,948,596 records), grafana/loki:3.5.3 digest-pinned via Docker, OURIOS_COMPARATIVE_CLASSES=all, release build, every §7 FROZEN gate asserting. Run logs and the machine-readable comparative-results.json artifacts for both runs are retained by the maintainer outside the repository (the gitignored local scratch/ tree), as with every ad-hoc VM record in this series.

Two runs, recorded honestly. Run 1 (5,798 s) measured every pair and passed every gate it printed — L2 39.51× processed / 1.275× storage floor, L1 97.56× storage, L4 84.93× processed / 3.59× storage floor, both L6 latency floors — but failed overall: the L3 pair hit the known transient Loki-visibility flicker (loki returned 0 of 9 expected rows … before timeout) — the class addressed by the #490 flag fixes, previously hit in runs #11/#13/#21 — so the harness hard-failed the run rather than salvage it. The decision rule applied: one deliberate, configuration-identical retry; had the flicker recurred, the next step would have been engineering, not rerolling. It did not recur. Run 2 (5,556 s) is the counted authoritative run: exit 0, 1 passed / 0 failed, all 11 frozen gate decisions Decided { pass: true }, L3 measured cleanly, RFC 0033 acquisition warm on every body-rendering pair (one artifact GET, 187,906 B compressed), equivalence held on every pair, and L4 completeness 1147/1197 = 95.8% — inside the §7 0.90 margin and the §9.17 band (95.3–97.5%).

Run 2 — the counted numbers (honest §3.6 totals; Loki bytes per channel; latency oriented loki_p50/ourios_p50, where > 1 = Ourios faster):

Reading 1 — the frozen-gate program confirms at authoritative class. Every §7 FROZEN value asserted and passed on the baseline hardware with no recalibration. On the bytes channels the margins are equal or wider than the ci-runner series: L1 storage 97.82× vs the §9.13 calibration streak’s 77.2–77.7× (and in the neighbourhood of the later asserting runs’ 102–108×); L3 22.52× vs 21.2–21.9×; L2 storage floor 1.239× inside its 1.05–1.49× history. L4’s 3.60× storage floor sits a whisker below the §9.17 indicative band (3.69–3.73×) — Loki-side storage wobble scale (§9.13’s determinism note), and more than 3× the frozen 1.1× floor either way; L4 processed (85.14×) is inside its 84.9–87.1× history. Bytes-read being CPU-insensitive by construction (§9.13), agreement here is expected — but now it is measured, and the thesis’s must-win claims are no longer resting on indicative hardware alone.

Reading 2 — the needle latency ratios compress on dedicated hardware, exactly as caveated. L1 90.44× and L3 77.99× against the ci-runner indicative 308×/323×: Loki’s corpus-wide needle scans drop from 23–24 s to ~10 s on the faster box while Ourios stays in its flat fixed-cost profile (57–129 ms warm p50 across every pair). This is §9.13’s / #498’s “a tuned environment compresses latency ratios; the structural asymmetries remain” caveat, now measured rather than hedged: the wall-clock gap on the needle classes is still interactive-vs-batch (~110–129 ms vs ~10 s), the L6 floors still hold (0.37 / 4.34 / 0.76 diagnostic), and the bytes asymmetry — the primary channel — did not compress. Quote the latency channel from this entry, not the indicative one.

Reading 3 — post-RFC-0035 code, gates held without recalibration. 9deecb1 carries the RFC 0035 Design A ingest restructuring (#577/#581). Every read-path byte count and every frozen gate held with no adjustment, consistent with RFC0035.5’s guarantee that the concurrency change alters no on-disk artifact.

With this record the #498 authoritative-rerun checkbox — the last open decision on that scoreboard — is discharged. The remaining storage-side lever named in §9.13 (write-side layout, hazard #4) stays parked on its own line, unchanged by this run.

9.25 Results — 2026-07-22 (indicative, local M-series) — D2 / D3 after the RFC 0036 sorted compaction

Purpose. RFC 0036 slice B (RFC0036.3) makes compaction sort the partition by (promoted service.name, time_unix_nano) via the §3.2 external merge sort and rotate compacted row groups at the smaller compacted threshold (the fixed 32 MiB in effect at this run; later adaptive — §9.30). §5 requires that D2/D3 — RFC 0009’s compaction-throughput and file-band properties — survive the sort. This is the “first measurement” §7 asks for, at red: the band from which the sorted D2 is set. Indicative, not authoritative — the baseline-VM sorted-vs-unsorted rerun is deferred to validated (a paid, maintainer-opted run, per RFC 0031 §3.2 / the “bench on ci-runner first” discipline).

Hardware / run. Local M-series developer machine (P-cores; taskpolicy -B to escape the Claude-Code background-QoS E-core throttle), release build at the slice-B head. The ourios-bench compaction bench, band-scale one-shot mode (OURIOS_COMPACTION_BASELINE=1 FILES=32 ROWS=4800 BODY_BYTES=4096 — the exact §9.7 shape, ~453 MiB of input in one partition), plus the criterion micro-sweep to confirm the groups still execute with the sort in the path.

Reading — “sorting is not free,” measured honestly. Sorted D2 on this box is ~138 MiB/s against §9.7’s 166.8 MiB/s unsorted. The two numbers are not the same hardware (local M-series vs the 8 vCPU EPYC-Milan baseline), so this is not a clean sorting-overhead delta — it is a fresh indicative measurement of the sorted path, plus the observation that adding a full external-merge-sort pass (decode → per-input stable sort → spill/merge, or the in-memory skip-spill short-circuit) to the rewrite lands in the same order of magnitude as the prior unsorted copy-through and comfortably clears “keeps up.” The clean same-hardware sorted-vs-unsorted delta is a validated baseline-rerun item; it is deliberately not asserted here. The D2 band is set from this measurement (indicative floor ~120 MiB/s on this class) and is not an in-repo wall-clock gate — wall-clock gates flake, so the in-repo RFC0036.3 assertion is structural (D3 file band + the §3.2 memory bound); the throughput lives here in §9 and in the bench (RFC 0036 §6).

Reading — D3 holds unchanged. The headline: the sort did not touch the file band. Compaction still emits exactly one file per partition, 452.7 MiB, squarely in the H4 256 MiB–2 GiB target with zero sub-128 MiB files — the RFC 0036 §3.3 amendment drops the compacted row-group threshold (32 MiB, ~14 groups here) but leaves the file band untouched, and D3 measures files. The structural side (one output file, small-file count → 1, rows conserved, sorted layout declared) is pinned deterministically in ourios-parquet’s rfc0036_3_compaction_properties_preserved (tests/it) and the merge proptests; the §3.2 peak-memory bound (forced-spill residency = one input + F×batch, never whole-partition) in rfc0036_3_forced_spill_peak_far_below_whole_partition. These wall-clock figures are the indicative stamp; the authoritative sorted D2/D3 awaits the validated baseline rerun (RFC0036.5’s frozen-gate pass will accompany it).

9.26 Results — 2026-07-22 (authoritative, baseline-8vcpu-32gib) — RFC 0036 comparative rerun: no regression + the single-file-harness finding

Purpose. RFC0036.5’s deferred validated item: rerun the frozen RFC 0031 comparative dispatch at post-RFC-0036 main (HEAD 5e5aa66) on baseline hardware, and confirm the frozen gates still pass on the sorted-compaction code. The v8 corpus (frozen corpus/otel-demo-v8 release), one Loki container + one replay, same methodology as §9.24.

Hardware / run. Fresh ccx33 (8 dedicated vCPU EPYC-Milan / 32 GiB = baseline-8vcpu-32gib), release build, Docker for the Loki testcontainer, box deleted on exit. ~75 min/attempt.

Result — every measured frozen gate passes, in §9.24’s band (no regression):

The finding — RFC 0036 is not exercised by this harness, and the run proves it. Every ourios_bytes_read came back byte-for-byte identical to §9.24 (L1 1,032,727; L2 2,223,173; L4 47,995,205; L6 k=100 1,931,911; L6 k=2000 4,202,473). build_comparative_store writes exactly one ingest file per partition (append_record keeps one Writer per partition), so compact_partition no-ops (it needs ≥ 2 files) and RFC 0036’s compaction-time sort never runs. The comparative store is therefore identical pre/post-0036 — which is why the gates pass unchanged (no regression) and why this harness cannot measure RFC 0036’s window-materialization win. Making it accrue multiple files per partition then compact would re-base RFC 0031’s frozen-gate store; that is deferred as future harness work, not a validated blocker (the RFC0036.2 bytes channel is a §2.2 diagnostic, not a gate).

Two orthogonal noise items (neither an RFC 0036 layout regression): attempt 1 measured L3 cleanly (21.6× storage) but landed the L6 k=100 latency floor at 0.311 < 0.333 (marginal tiny-query noise; attempt 2 passed it at 0.364); attempt 2 hit the known Loki L3 flicker (0/9 rows before timeout — the §9.13 class, #490). One justified retry per the retry-once rule; no third.

9.27 Results — 2026-07-22 (in-repo, deterministic) — RFC0036.2 window-materialization before/after

Purpose. The RFC0036.2 materialization diagnostic that §9.26’s harness cannot produce, measured directly on a genuinely-compacted store. ourios-querier’s rfc0036_2_materialization_before_after (tests/it) runs the identical L6 k=100 window query (1 s at the 10 ms grid, one service) against two stores built from the same synthetic multi-service hour, and reads the materialization bytes — the compressed column chunks of the row groups a service = target ∧ time ∈ window scan cannot prune — from each file’s footer.

Reading. Identical 100-row answer from both; the sort takes the window from materialising the whole file (the single-row-group unsorted ingest parquet — no row group prunes) to a contiguous minority (2 of 6 groups) — a 1.43× materialization-bytes win. Modest by design, and honestly so: the compacted file is physically ~2× larger (six 32 MiB groups compress a little worse than one 128 MiB group — the RFC 0036 §3.3 pruning- granularity-over-bytes trade), and §2.2’s ~188 KB registry floor is exactly why the gate is the scanned-row-group bound (enforced by rfc0036_2_window_materialization_bound), not a bytes ratio. count- scan stats.bytes_read is not the materialization term — it reads only the small filter columns and points the wrong way; the footer survivor-chunk sum is the RFC §9 metric.

9.28 Results — 2026-07-22 (indicative, local M-series, synthetic-compressible) — RFC 0036 §7 compacted row-group threshold sweep (16 / 32 / 64 MiB)

Purpose. RFC 0036 §7’s first open box defers the authoritative 16/32/64 MiB compacted-threshold sweep to the paid baseline-8vcpu-32gib harness. This is the in-repo, indicative half. §9.27’s before/after used a near-incompressible random payload (chosen to cross 32 MiB with few rows) and so read the compacted file as ~2× larger on disk — a worst-case artifact of incompressible bytes, not what real logs do. Real logs are compressible and have per-service locality, and RFC 0036 sorts by service.name, which clusters similar lines. This sweep re-measures the trade on a compressible, service-clustered synthetic corpus to correct that impression and trace the actual curve. Indicative, not authoritative — local hardware, synthetic corpus; the v8-corpus L6-scanned-bytes-vs-L1/L3 sweep stays deferred to validated (RFC 0036 §7).

Hardware / run. Local M-series developer machine (P-cores; taskpolicy -B to escape the Claude-Code background-QoS E-core throttle), dev build at the sweep head. ourios-querier’s rfc0036_7_compacted_threshold_sweep (tests/it, #[ignore]d like the RFC0005.6 sizing test — not a CI gate; the gate is rfc0036_2_window_materialization_bound). It drives the shipped compaction path with the new explicit-threshold seam (compact_partition_with_flush_threshold).

Corpus. 6 promoted services, each with its own fixed log phrase (distinct vocabulary, so the §3.1 sort clusters like text) plus a handful of small varying fields — request id, user, amount, status/region enums, and 8 hex chars of per-line entropy that hold ZSTD to a realistic ratio. 2,160,000 rows (360,000/service), ~230–260 B bodies, 467,239,238 B (445.6 MiB) of raw body text, written as two interleaved ingest files then compacted. Measured compression: 7.20× (raw body ÷ the 61.85 MiB compacted file) — squarely in the realistic 5–15× band, not the ~2.4× of random ASCII.

Finding 1 — the ~2× on-disk bloat is a random-bytes artifact (the headline). On compressible, service-clustered data the compacted file does not grow as the threshold shrinks. 32 and 64 MiB produce a byte-identical file; 16 MiB is larger by only 519,823 B (+0.80%) — the cost of two extra row groups’ footer/index entries and slightly-shorter compression windows. §9.27’s “compacted file ~2× larger” was the incompressible payload amplifying that per-group overhead against near-zero codec gain; it does not generalise to real logs. The §3.1 service sort, if anything, helps compression by clustering like lines — the finer-threshold file barely moves.

Finding 2 — finer thresholds buy real pruning granularity, nearly free. The same fixed 30 s one-service window materialises 14.66 MiB at 16 MiB vs 28.94 MiB at 32/64 MiB — the finer threshold halves (1.97×) the bytes a window browse must fetch, for a +0.80% file-size cost. The answer is identical (3,000 rows) at every threshold; only the IO changed. This is the pruning-granularity-over-bytes trade RFC 0036 §3.3 exists to make, now measured on realistic data: it is a good trade, and it gets better the finer the threshold, bounded only by footer/index overhead.

Finding 3 — 32 and 64 MiB are near-identical because arrow’s default row-group row cap sets a ~30 MiB granularity floor, finer than either byte threshold. The compacted writer sets no max_row_group_size, so parquet-rs’s default 1,048,576-row cap applies. At this corpus’s ~30 encoded bytes/row that cap fills a group at ~30 MiB, which trips before the 32 MiB byte flush (and well before 64 MiB): so 32 and 64 MiB are both row-capped at ~3 groups of ~30 MiB (byte-identical files), and only 16 MiB (~590 k rows → ~15 MiB, under the cap) is genuinely byte-governed → 5 finer groups.

Correction to an earlier framing. A prior draft of this finding called the row cap a “gap” and suggested raising max_row_group_size “so the byte threshold bites.” That is backwards: raising the cap would let the 32/64 MiB byte flush govern and produce fewer, coarser groups (~2 of ~32 MiB) — the wrong direction for pruning. The row cap is a granularity floor that is currently helping. The lever that unambiguously improves window pruning is a smaller byte threshold (16 MiB halves window materialization here), the §7 authoritative-sweep question — not a larger row cap. Making group sizing byte-uniform (raising the cap) is a separate predictability choice, not a pruning win. Net: finer effective groups win the materialization trade at sub-1% disk cost, and the byte-threshold value — not the row cap — is the lever.

Conclusion. 32 MiB stays a reasonable, defensible shipped default (it already delivers the pruning mechanism the RFC0036.2 gate enforces). The compressible-data trade leans, if anywhere, toward smaller thresholds (16 MiB halves window materialization for +0.80% on disk) — the opposite of the “smaller = bloated file” worry §9.27’s random payload suggested. 16 MiB is flagged as a candidate for the authoritative v8 sweep to evaluate against L1/L3 neutrality (more, smaller row groups add footer/page-index bytes to every scan — the term this in-repo corpus is too small to price against the comparative gates); the default is not changed here — that is a maintainer decision on the authoritative numbers. The interim OURIOS_COMPACTED_RG_BYTES env knob (RFC 0036 §7) lets an operator retune without a rebuild in the meantime.

9.29 Results — 2026-07-22 (indicative, local M-series, real-corpus subset, Ourios-only, no Loki) — RFC 0036 window-materialization before/after on the v8 capture

Purpose. The real-otel-demo-corpus analogue of §9.27’s synthetic 1.43× before/after — the piece §9.26 structurally could not produce (its single-file-per-partition harness no-ops compact_partition). This measures the RFC 0036 window-materialization win on the actual v8 capture (a subset), Ourios-only. Indicative, not authoritative — local hardware, a corpus subset, no Loki, no baseline VM; the authoritative full-v8 + Loki arm stays deferred (RFC0036.2 §5 / §9.26).

What made this measurable — the opt-in compacted harness path. §9.26’s finding was that ourios-bench’s build_comparative_store writes one ingest file per partition, so compaction no-ops and RFC 0036’s sort never runs. This slice adds an opt-in builder, build_comparative_store_compacted[_with_threshold], that round-robins each partition’s rows across two interleaved ingest files, then calls compact_partition on every partition — so the consolidated file is §3.1-clustered by (service.name, time), rotates row groups, and declares sorting_columns. The default build_comparative_store path — the frozen RFC 0031 dispatch’s store — is byte-for-byte untouched (a separate function; the frozen gates are not re-based).

Hardware / run. Local M-series developer machine (P-cores; taskpolicy -B to escape the Claude-Code background-QoS E-core throttle), dev build. ourios-bench’s rfc0036_realcorpus_window_materialization_before_after (tests/rfc0036_realcorpus.rs, #[ignore]d, skips with a clear message when the gitignored capture is absent so CI/other machines never fail). The store is built two ways from the identical subset — before = default single-file build_comparative_store (no compaction), after = build_comparative_store_compacted_with_threshold — then the same L6-shape window query runs against each.

Corpus / query. First 120,000 LogsData batches of otel-demo-v8/logs.jsonl (513,752,573 B of the 2.96 GB capture, ~21 h of wall-time). The measurement picks the busiest compacted partition (most row groups), then the most-prunable real service in it — which resolves to ad, the lowest-volume otel-demo service and the exact §9.13 run #17 case — and the busy hour as the window: service == "ad" | range(2026-07-07T08:00:00Z, 2026-07-07T09:00:00Z). Materialization bytes = the footer survivor-chunk sum (the RFC 0036 §9 metric: compressed column chunks of the row groups a service = ad ∧ time ∈ window scan cannot prune, keyed on the effective_time_unix_nano column the querier prunes on — not the count-scan stats.bytes_read); the live query’s row_groups_scanned cross-checks the footer prediction.

Reading — a 5.20× real-corpus materialization-bytes win, identical answer. The sort takes the one-service window from materialising the whole hour (the single unsorted ingest row group — nothing prunes) to one of seven clustered row groups (ad lands in a single group; the other six prune on plain footer service.name statistics). Same 10,110-row answer; the live query confirms the footer prediction (before scans 1/1, after scans 1/7). This is larger than §9.27’s synthetic 1.43× — and honestly so: §9.27’s random payload was incompressible and forced the compacted file ~2× larger, whereas real logs compress and the §3.1 service.name sort clusters like lines, so finer clustered groups prune sharply (the §9.28 finding-2 mechanism, now on real data).

The threshold caveat — the then-fixed 32 MiB default did not prune a real hour (§9.28 finding 3 confirmed; fixed by the §9.30 adaptive amendment). Real per-hour v8 volume (~a few MiB compressed) is far below the fixed 32 MiB MAX_COMPACTED_RG_BYTES (the old COMPACTED_ROW_GROUP_FLUSH_BYTES), so at 32 MiB the busy hour compacts to a single row group and nothing prunes — the measurement skips with a “raise subset / lower threshold” message (verified). The win therefore requires a finer compacted threshold (2 MiB here) to rotate the real hour into the several service-clustered groups the pruning mechanism needs. This is not a defect of the layout — it is the same row-cap/volume reality §9.28 finding 3 documented on synthetic data, now reproduced on the real corpus: on realistic per-hour volumes the byte threshold must be finer than 32 MiB to bite. Indicative sensitivity on this subset: 2 MiB → 7 groups, 5.20×; 4 MiB → 2 groups, 1.32×; 32 MiB → 1 group, no prune. The finer the threshold, the sharper the window prune — the §9.28-flagged lever, confirmed on real data.

Disposition (superseded 2026-07-22 — see §9.30). This complements §9.26 (no-regression, but structurally could not show the win) and §9.27 (synthetic 1.43×) with a real-corpus before/after (5.20× at an explicit 2 MiB threshold). Its central finding — that the fixed 32 MiB shipped default is inert on a real per-hour v8 hour (compacts to one group, prunes nothing, the measurement skipped), and the win only appears at a hand-set finer threshold — is exactly what motivated the RFC 0036 §3.3 adaptive-threshold amendment (2026-07-22): the threshold now scales as clamp(input_total / 8, 1 MiB, 32 MiB), so a small real hour floors at 1 MiB and prunes without any operator tuning. §9.30 re-runs this same measurement at the adaptive default and records the win there. The authoritative full-v8 L6-scanned-bytes-vs-Loki arm, and the ceiling/target-K sweep, stay deferred to the paid baseline-8vcpu-32gib harness (RFC 0036 §7 / RFC0036.2).

9.30 Results — 2026-07-22 (indicative, local M-series, real-corpus subset, Ourios-only, no Loki) — RFC 0036 window-materialization at the ADAPTIVE default

Purpose. §9.28/§9.29 established that a fixed compacted row-group threshold is the wrong shape: 32 MiB is inert on a real per-hour v8 hour (a few MiB compressed → one row group → nothing prunes; §9.29 skipped at the 32 MiB default), while the same fixed value fragments large hours. The RFC 0036 §3.3 adaptive amendment (2026-07-22, maintainer-approved) replaces it with adaptive_flush_bytes = clamp(estimated_output_bytes / 8, 1 MiB, 32 MiB) — target ~8 groups per partition, floored at 1 MiB (the lever for small hours) and capped at 32 MiB (the old fixed value, now the ceiling for huge hours). This run is the decisive check: re-run §9.29’s real-corpus before/after at the adaptive default (OURIOS_V8_COMPACTED_RG_BYTES unset) and confirm the win now appears at the shipped default — not only under a hand-set threshold. Indicative (local, subset, Ourios-only, no Loki), same caveats as §9.29.

Hardware / run. Local M-series developer machine (P-cores; taskpolicy -B), dev build. ourios-bench’s rfc0036_realcorpus_window_materialization_before_after (tests/rfc0036_realcorpus.rs, #[ignore]d), threshold env unset so the adaptive default governs. Same corpus and same selection procedure as §9.29 (not a fixed query — the harness derives the target/window from the data): the first 120,000 LogsData batches (513,752,573 B) of otel-demo-v8/logs.jsonl, then it dynamically picks the busiest compacted partition (most row groups), the most-prunable real service in it — which again resolves to ad (lowest-volume otel-demo service, the §9.13 run #17 case) — and that partition’s own time span as the window. Here the selected partition is the 07:00–08:00 hour, so the query is service == "ad" | range(2026-07-07T07:00:00Z, 2026-07-07T08:00:00Z) (a different hour from §9.29’s dynamically-selected 08:00–09:00 — expected, since the partition is re-chosen from the store the adaptive build produced, not hard-coded). Materialization bytes = the footer survivor-chunk sum (RFC 0036 §9 metric, keyed on effective_time_unix_nano); the live query’s row_groups_scanned cross-checks the footer prediction.

Reading — a 7.61× real-corpus materialization-bytes win at the shipped default, identical answer. With the threshold unset, the adaptive value floors at 1 MiB for this small real hour and rotates it into 21 service-clustered row groups; the ad window lands in a single group and the other 20 prune on plain footer service.name/time statistics. The sort takes the one-service window from materialising the whole hour (3,812,581 B, one unsorted group) to one clustered group (500,537 B) for the identical 9,984-row answer — the live query confirms the footer prediction (before scans 1/1, after scans 1/21). This is the point of the amendment: the win now appears at the shipped default, where the fixed 32 MiB threshold had skipped (§9.29). It is even sharper than §9.29’s explicit-2 MiB 5.20× — the 1 MiB adaptive floor is finer still, so it clusters into more, tighter groups.

Disposition. The adaptive default makes RFC 0036 non-inert on real v8 — the fix §9.28/§9.29 pointed at, now measured. The rfc0036_2_* in-repo gates recompute T = adaptive_flush_bytes(input_total) and track the layout whatever it resolves to. The authoritative full-v8 L6-scanned-bytes-vs-Loki arm and the ceiling/target-K sweep stay deferred to the paid baseline-8vcpu-32gib harness (RFC 0036 §7 / RFC0036.2).