rfc: 0002 title: Query DSL — the Ourios logs query language (Branch B, surface β) status: green author: Jens Holdgaard Pedersen jens@holdgaard.org drafting-assistance: Claude created: 2026-04-24 supersedes: — superseded-by: —
RFC 0002 — Query DSL
Status note. The prior decision (§3) is resolved: the predicate sublanguage takes Branch B (distance from OTTL), on the β (pipe-composable) top-level surface. Decided 2026-06-07 from the audience analysis in §3.6 (primary: Perses dashboard authors; future: MCP agents). This RFC is now
green— all 11 §5 acceptance criteria (RFC0002.1–.11) have passing tests (crates/ourios-querier/tests/rfc0002_dsl.rs), landed across PRs #143 (this spec) and #144–#154 (the red gate + implementation): the Branch-B parser + structured surface → one IR, the IR→DataFusion compile, YAML-embeddability + the structured JSON Schema, andresolves_toalias-set expansion via the RFC 0001 §6.7 operator alias map. §6 gives the design, §7 the grammar, §5 the criteria. Per thedocs/rfcs/README.mdladder,validatedand finallyaccepted(a maintainer flip) follow the §9 validation. Hazard 6 (CLAUDE.md§4 — no DataFusion/SQL leakage) constrains the whole design.
1. Summary
Ourios exposes a logs query DSL that does not leak DataFusion/SQL to
users (CLAUDE.md §4 hazard 6). This RFC specifies it:
- Predicate sublanguage — Branch B (distance from OTTL). An
Ourios-native, query-ergonomic syntax over the OTel data model (the
ingest contract): bare top-level fields (
body,severity,trace_id),resource./attr.prefixes, bare-identifier severity (severity >= error), first-class template + OTel-canonical primitives. - Top-level surface — β (pipe-composable). A predicate followed by
pipe stages:
… | range(-1h, now) | count by template_id | sort count desc | limit 10. Compact, single-line, and embeddable as a YAML scalar in Perses dashboards. - Two front-ends, one core. The string DSL (for humans, esp. Perses
YAML) and a structured JSON surface (for MCP agents + programmatic
clients) parse to the same query IR and compile to the same DataFusion
LogicalPlan. Agents emit JSON, not syntax.
The design rests on ourios-querier (RFC 0007), whose execution layer
(predicate pushdown, tenant isolation, QueryStats) is already
implemented and tested (RFC 0007 §5 criteria all live; the RFC itself
stays specified pending this DSL); this RFC adds the user-facing
language in front of it.
2. Motivation
2.1 Why a DSL at all?
CLAUDE.md §4 hazard 6 commits Ourios to a DSL that does not leak DataFusion SQL.
The reasons are stability (evolve the backend without breaking user
queries), safety (full SQL exposes cross-tenant joins, unbounded
scans, recursive CTEs we cannot audit), and fit (logs are a narrow
domain; a narrow DSL is more ergonomic than a general one). This is
branch-agnostic.
2.2 Why the prior decision mattered
“OTTL-inspired” was not a free decision. Borrowing OTTL syntax in a query context promises OTTL-literate users that their mental model transfers; if the syntax looks the same but behaves differently (OTTL mutates; a query filters), the surface actively misleads. §3 records how that decision was made.
3. The prior decision (resolved): distance from OTTL
Both positions were defensible; §3.1–3.4 keep the honest case for each for the record. §3.5 records the resolution; §3.6 the reasoning.
3.1 The case for borrowing (Branch A) — not chosen
- Positive transfer for Collector-literate SREs. Engineers who write Collector/OTTL pipelines reuse that mental model at zero onboarding.
- Reduces bikeshed surface. A pinned external spec inherits decisions rather than re-litigating them.
- Ecosystem alignment. Diverging on surface syntax in the OTel orbit can read as gratuitous.
- OTTL’s path grammar is correct about the data model, which any alternative must address anyway.
3.2 The case for distancing (Branch B) — chosen
- The OTTL-literate population is a minority of OTLP users — most emit logs via an SDK and never touch OTTL.
- Collector ergonomics become query verbosity.
resource.attributes["service.name"] == "api"is loud in a query. - Shared syntax + different semantics misleads. Unfamiliar syntax is a safer failure mode than almost-familiar-but-wrong.
- No evolving external spec to track (OTTL has had breaking changes).
- Design freedom for query-context idioms (
severity >= error,attr.foo).
3.3 What is shared regardless of branch
- The OTel data model is the schema of log records (the ingest contract, not a design choice): attributes, resources, severity, body, timestamps, trace context.
- The template + correctness primitives (
template_id,confidence,lossy; drift-alias membership viaresolves_to) are first-class (§6.3). - The compilation target is a DataFusion
LogicalPlan, no SQL leakage (§6.5).
3.4 Consequences
| Dimension | Branch A (borrow) | Branch B (distance) |
|---|---|---|
| Onboarding for Collector-literate SREs | Near-zero | Mild (new syntax, familiar semantics) |
| Onboarding for SDK / dashboard users | Same (OTel data model) | Same |
| Maintenance cost | Track pinned OTTL, amend on bumps | Own the grammar |
| Same-syntax/different-meaning confusion | Real | Avoided |
| Spec size | Smaller | Larger (owned) |
| Ecosystem signalling | Aligned with OTel | Independent (in the data model: still aligned) |
| Design freedom | Constrained by OTTL | Free within the OTel data model |
3.5 Resolution
Branch B (distance from OTTL), surface β (pipe-composable). Decided
2026-06-07 by the maintainer, on the audience analysis in §3.6 in lieu of
the formal user research originally gated here (§9 now scopes that
research to the accepted gate, not specified).
3.6 Why — the two audiences
The decision turns on two audiences that re-weight §3.1–3.4:
- Primary — Perses dashboard authors (declarative YAML/CRDs). Queries live as string scalars in versioned YAML. Brevity and low bracket/quote density win (readable scalars, clean diffs); the audience thinks in dashboard query languages (PromQL/LogQL), not OTTL. Branch B’s flat syntax + the β pipe surface embed cleanly on one line; Branch A on surface α would be multi-line and bracket-heavy.
- Future — MCP agents. Borrow-but-diverge (Branch A + the §6 divergence list it required) is the worst case for LLMs: strong public-OTTL priors pull a model toward real-OTTL constructs we do not support → plausible-but-invalid queries. A small, self-owned grammar (Branch B) has no priors to fight, is cheaper to embed in an MCP tool schema, and is enforceable with grammar-constrained decoding. And — decisively — agents need not generate syntax at all: they target the structured surface (§6.4).
The one strong case for Branch A (onboarding + signalling for Collector-literate SREs) lands on the audience that is not primary here, while its costs (semantic-confusion in the overlap zone; an externally-driven breaking cadence against long-lived dashboards + cached agent schemas) land squarely on these two. Distancing on surface syntax costs little ecosystem goodwill because we stay faithful to the OTel data model (§3.3) and because bespoke query syntax is the norm (LogQL, PromQL, CloudWatch Insights all diverge from any transformation language).
The full audience analysis is the drafting-assistance recommendation that informed this decision; its three OTel-ecosystem questions (is there an OTel query language to align with? is OTTL the expected querying surface? Perses+OTel query conventions?) are folded into §9.
4. Design principles
- Familiarity beats cleverness. A first-time reader understands a query within 30 seconds without a reference. No heavy sigils.
- No DataFusion/SQL leakage (
CLAUDE.md§4 hazard 6). If explaining a surface form requires naming a DataFusion type, the form is wrong. - Predicate, then pipeline. A query is a predicate (the
where) followed by ordered stages (range, aggregate, sort, limit, project). Each reads independently. - Template + OTel-canonical fields are first-class vocabulary, not
pseudo-columns:
template_id,confidence,lossy(drift-alias membership viaresolves_to);service,trace_id,span_id,scope(the primary correlation/query dimensions per OTel maintainer guidance, §6.2). - Every query has a time range — explicit
range(...)or a tenant-configurable default window. Never an unbounded scan. - One core, two surfaces. The string DSL and the structured surface are equivalent front-ends over one IR (§6.4); neither can express a query the other cannot.
- YAML-embeddable. A query is expressible as a single-line scalar that survives a YAML round-trip — a first-class constraint for the Perses audience, not an afterthought.
- The grammar is owned and versioned by this RFC (§7), not “inspired by” anything. Compatibility pledges are written, not implied.
5. Acceptance criteria
Given/When/Then, ids greppable from tests: each test carries thedocs/verification.md§2.2 doc-comment form —/// Scenario RFC0002.<n> — <title>.plus/// See docs/rfcs/0002-query-dsl.md §5.. These specify the parser + compiler that front-ends the (already-implemented, RFC 0007 §5) execution layer.
-
RFC0002.1 — A Branch-B predicate parses and compiles to a filter
[CLAUDE.md §4 hazard 6]- Given a Branch-B predicate (e.g.
template_id == 42 and severity >= error) - When it is parsed and compiled
- Then it yields the query IR and an internal DataFusion
Filter(a private compilation artifact — never surfaced through the public API, RFC0002.3). Predicates over RFC 0007 §4.3’s pushdown keys prune the scan per that section’s split —template_idskips row groups (B1),time_unix_nanoprunes partitions and row groups,tenant_idprunes partition directories (not row groups); for the subset the currentourios_querierstructured request can express (template + time) the DSL result is identical to it. Severity compiles via the §6.2/RFC0002.5severity_numbermapping (the column is RFC 0005’sseverity_number), not theseverity_textequality the current request supports, and predicates over non-indexed fields (service,attr.*) compile to a correctFilterwith no row-group-pruning claim (indexedservice.namepushdown would be a future RFC 0005 §3.6 amendment).
- Given a Branch-B predicate (e.g.
-
RFC0002.2 — String DSL and structured surface compile to the same plan
[§6.4]- Given a query expressed both as a β string and as the structured JSON surface
- When both are compiled
- Then they produce the same query IR (and hence the same
LogicalPlan) — the one-core/two-surfaces invariant.
-
RFC0002.3 — No DataFusion/arrow/SQL leakage
[CLAUDE.md §4 hazard 6]- Given the public DSL API (parse, compile, error types)
- When a query parses, compiles, or fails
- Then no
datafusion/arrow/SQL type or message appears in any public signature or error string (compile- and string-level boundary test, mirroring RFC0007.3).
-
RFC0002.4 — A query without an explicit range gets the tenant default window
[§4 P5]- Given a query with no
range(...)stage - When it is compiled in a tenant context with a default window W
- Then the plan carries a time-column filter equal to W — never an unbounded scan.
- Given a query with no
-
RFC0002.5 — Bare-identifier severity maps to its SeverityNumber
[§6.1]- Given
severity >= error(andwarn,info,debug,trace,fatal) - When compiled
- Then each maps, case-insensitively, to the §6.1
SeverityNumberfor that level (error→ 17, etc.) and compiles identically to the numeric form (severity >= 17). The name→number mapping is the documented §6.1 one (Ourios’s, aligned with the OTel ranges) — not an OTel-standardised threshold.
- Given
-
RFC0002.6 — First-class OTel-canonical fields resolve correctly
[§6.2]- Given
service,trace_id,span_id,scopeused as bare fields - When compiled
- Then each resolves to the RFC 0001 §6.1 column / resource-attribute
it names (
service→resource["service.name"]), with no string-flattening required of the user.
- Given
-
RFC0002.7 — Parse/serialise round-trip is idempotent
- Given any well-formed query (property-generated)
- When parsed → serialised → parsed
- Then the second parse equals the first (AST idempotence).
-
RFC0002.8 — A malformed query yields a specific, leak-free error
- Given a syntactically or semantically invalid query
- When parsed/compiled
- Then it returns a specific error citing the offending token/clause and the §7 grammar — never a panic, never a DataFusion message.
-
RFC0002.9 — Template primitives compile
[§6.3]- Given
template_id == 42,resolves_to(42),lossy == true,confidence < 0.7 - When compiled
- Then each compiles to the documented plan (
resolves_toexpands to the alias-set membership of RFC 0001 §6.7), without leaking the underlying representation.
- Given
-
RFC0002.10 — A query is a YAML-safe single-line scalar
[§4 P7]- Given the canonical serialisation of any well-formed query
- When embedded as a YAML scalar and round-tripped through a YAML parser
- Then the recovered string parses to the same query (the Perses- embedding guarantee).
-
RFC0002.11 — The structured surface validates against its published schema
[§6.4]- Given the structured (MCP) query surface
- When a request is validated against the published JSON schema
- Then well-formed requests pass and compile; malformed ones are rejected by the schema before reaching the planner.
6. Design
6.1 Predicate sublanguage (Branch B)
A predicate is a boolean expression over paths, operators, and
literals against the OTel log data model. The bare literal true is
the match-all predicate (for queries that filter only by range/other
stages); false matches nothing.
Paths.
- Top-level fields are bare identifiers mapping to the OTel log
data-model fields:
body(Body — an OTelAnyValue: string, bool, int, double, bytes, array, or kvlist/map),severity(SeverityNumber),ts(Timestamp),observed_ts(ObservedTimestamp),trace_id(TraceId),span_id(SpanId),scope(InstrumentationScope name),flags(TraceFlags). (Backend treatment of structuredbodyvsattr.*is not uniform across the ecosystem; the DSL keeps the split explicit rather than flattening.) - Resource attributes:
resource.<key>where<key>is the OTel attribute key taken literally including dots (resource.service.name→ resource attribute"service.name"). Bracketed formresource["..."]for any key not expressible as dotted bare identifiers — characters outside the bare-identifier set, a segment starting with a digit, or a reserved-word collision (resource["k8s.pod.name"],resource["3rd.party"]). - Log-record attributes:
attr.<key>(attr.http.status_code→ attribute"http.status_code"); bracketedattr["..."]for the same non-bare-identifier cases. - Severity:
severitycompares against a bare severity name (severity >= error), case-insensitive, or a numeric form (severity >= 17). All severity comparisons — including ordering (</<=/>/>=) — are defined on the OTelSeverityNumber, never on the free-formseverity_text(per the OTel comparing severity guidance). Bare names map to the floor of the matching OTelSeverityNumberrange:trace→1,debug→5,info→9,warn→13,error→17,fatal→21. The spec standardises the ranges and says to compare onSeverityNumber; this name→number mapping is Ourios’s, aligned with those ranges, not separately mandated by OTel.
Operators. Comparison: ==, !=, <, <=, >, >=, =~
(regex match), !~ (regex non-match). Boolean: and, or, not, with
terse aliases &&, ||, !; grouping with ().
Literals. Double-quoted strings ("api"), numbers (500, 0.7),
booleans (true/false), null, duration literals (30s, 1h, 1d,
1w), and RFC 3339 timestamps.
Functions (read-only, bespoke names tuned for queries) — boolean
predicate terms: matches(path, regex), contains(path, s),
starts_with(path, s), ends_with(path, s). They require a string
operand: applying one to a non-string path (severity, a numeric/bool
attribute, lossy, ts) is a compile-time type error (RFC0002.8), not a
silent coercion. (Scalar-returning functions such as len(path) are
deferred: the grammar admits a call only as a boolean term, so a
numeric len(...) > n would need a scalar-comparison form — added under a
future minor version when a need surfaces.)
Worked predicate.
service == "api" and severity >= error and attr.http.status_code == 500
6.2 First-class OTel-canonical fields
Per OpenTelemetry maintainer guidance (the primary dimensions a log backend is judged on), these get named, bare surface rather than hand-written attribute lookups, resolving the last open question of the prior draft:
| Surface | Resolves to (RFC 0001 §6.1) |
|---|---|
service | resource["service.name"] |
trace_id, span_id | the dedicated columns (log↔trace correlation) |
scope | scope_name |
severity | severity_number (via the §6.1 mapping) |
ts | time_unix_nano (the verbatim event timestamp) |
observed_ts | observed_time_unix_nano |
Amendment 2026-06-11 —
range(...)filters the effective timestamp. This table previously noted thatts/time_unix_nanois “whatrange(...)filters”. Per RFC 0005 §3.2 (amendment of the same date), the time window shall compile against the derivedeffective_time_unix_nanocolumn —time_unix_nanowhen non-zero, elseobserved_time_unix_nano.unwrap_or(0)(RFC 0005 §3.2 is the normative derivation; a record with neither timestamp stays at0). The implementing slice follows this amendment; until it lands, the querier filterstime_unix_nanodirectly. The change makes records whose source timestamp is unknown (time_unix_nano = 0— ~15 % of real OTel-Demo corpora, per the OTLP logs data model’s “UseTimestampif it is present, otherwise useObservedTimestamp” recommendation) addressable by time. The baretsfield is unchanged — it still resolves totime_unix_nano, the verbatim wire value (RFC 0001 scenario RFC0001.10). For files written before the column existed the window applieseffective := time_unix_nano(the RFC 0005 §3.9 documented default — exactly the pre-amendment behaviour), not the absent-OPTIONAL-column ⇒ predicate-false convention. The window bounds are half-open —range(from, to)selectsfrom <= effective < to. The half-open shape is what the querier already implements today (overtime_unix_nano) and matches RFC 0010’s locally-pinned[from, to)(which noted this RFC had not pinned boundary semantics; it now does).
trace_id / span_id literals are hex strings (32 and 16 hex digits
respectively, no separators), parsed case-insensitively so uppercase
OTLP/JSON ids are accepted; the canonical/serialised form is lowercase.
The compiler hex-decodes them to match the stored byte columns — the
OTLP/JSON id convention, consistent with RFC0003.6.
6.3 Template + correctness primitives
First-class vocabulary — Ourios-specific extensions (RFC 0001 §6.3/§6.7), not OpenTelemetry log-data-model fields; they live in the Ourios schema + query layer alongside the OTel-canonical fields of §6.2:
template_id == 42— exact template; resolves to thetemplate_idcolumn.resolves_to(42)—Xplus its drift aliases (the RFC 0001 §6.7 drift question); compiles to alias-set membership overtemplate_id.confidence— miner confidence (e.g.< 0.7); theconfidencecolumn.lossy— the lossy-reconstruction flag; resolves to the RFC 0001 / RFC 0005lossy_flagcolumn (lossy == true).render(pipe stage, §6.5) reconstructs the original line, honouringlossy.
The drift question is answered by resolves_to (alias membership). A
bare drift predicate (“has this template drifted?”) is deferred:
per RFC 0001 §6.7 drift is an audit-stream property, not a column in the
RFC 0005 data files, so it needs an audit-stream query path — a future
capability, not a row predicate in this grammar.
6.4 Two front-ends, one core
flowchart LR A["string DSL (β)<br/>Perses YAML, humans"] --> P[parser] B["structured surface<br/>JSON, MCP agents + clients"] --> V[schema validate] P --> IR[query IR] V --> IR IR --> C["compiler<br/>(no SQL leakage)"] C --> LP["DataFusion LogicalPlan<br/>(RFC 0007 execution layer)"]
-
String DSL (surface β) is the human surface (esp. Perses YAML). A query is a predicate optionally followed by
|-separated stages:service == "api" and severity >= error | range(-1h, now) | count by template_id | sort count desc | limit 10A predicate-only / “no filter” query uses the match-all atom
true, e.g.true | range(-1h, now) | limit 100.Stages:
range(from, to)(each bound a relative duration, thenowkeyword, or an RFC 3339 timestamp — the §7timeform; defaults per §4 P5),count [by <field, …>](comma-separated, per the §7field_list) and other aggregations (sum,min,max,avgover a path),sort <field-or-aggregate> [asc|desc](the §7sort_key— a field or an aggregate output likecount),limit <n>,project <field, …>/render. The whole query is expressible on one line as shown (whitespace around|is optional) — the §4 P7 YAML constraint. -
Structured surface is the machine contract (MCP tool schema + programmatic clients): a top-level object
{ "predicate": <node>, "stages": [ <stage>, … ] }(stagesoptional, default[]). Afieldis structured (no DSL path syntax for agents to build or escape): a bare top-level name string ("service","severity","body","trace_id", …) or an attribute object{ "resource": "<key>" }/{ "attr": "<key>" }(<key>the raw OTel attribute key, e.g."k8s.pod.name"). Anopis a §7cmp_opstring ("==",">=","=~", …); avalueis a JSON primitive (string / number / bool / null), with durations and timestamps carried as their §7 lexical strings ("1h", RFC 3339). A<node>is a comparison node{ "field": …, "op": …, "value": … }, a call node{ "call": "<fn>", "args": [ … ] }whoseargsfollow the §7 typed signatures —matches/contains/starts_with/ends_withtake[ <field>, <string> ](<field>as above),resolves_totakes[ <number> ], a constant node{ "const": true | false }(the §7bool_litmatch-all / match-none —{ "const": true }is the “no filter” predicate), or a boolean node ({ "and": [ <node>, … ] }/{ "or": [ <node>, … ] }with a child array;{ "not": <node> }unary, per §7). Each<stage>is a tagged object covering the full §7 stage set —range/count/sum/min/max/avg/sort/limit/project/render. Its JSON Schema is published and versioned with the parser (snapshot- tested like the §7 grammar; RFC0002.11), and it compiles to the same IR as the string surface (RFC0002.2). It is the formalised, extended successor to the existingourios_querier::QueryRequest(the RFC 0007 structured API) and is the stable surface agents target — no grammar generation required.
Both parse/validate to the same query IR and compile identically
(RFC0002.2). The tenant is not expressed in either surface — it is
supplied by the executing context (CLAUDE.md §3.7 multi-tenancy;
enforced per RFC0007.5); a query
without a tenant is an API usage error, not a cross-tenant scan.
6.5 Compilation target
Every construct compiles to a DataFusion LogicalPlan:
| DSL construct | DataFusion logical node |
|---|---|
implicit from logs | TableScan on the tenant’s log table |
predicate / range | Filter (range → time-column predicate) |
count / aggregations | Aggregate |
sort | Sort |
limit | Limit |
project | Projection |
render | custom projection honouring the three-zone reconstruction model |
resolves_to(42) | custom node expanding to alias-set membership |
All but render and resolves_to are DataFusion’s built-in algebra;
those two are the only Ourios extensions, both surface-independent.
6.6 Stability and versioning
The grammar (§7) is owned and versioned by this RFC. Additions (new functions, new first-class fields) are minor versions. Behavioural changes that could alter a query’s result set are major versions, require an amending RFC + a deprecation window, and — because the Perses/MCP audiences persist queries (git-versioned dashboards, cached agent schemas) — ship with a documented migration. There is no external spec to shadow, so major versions are deliberate, not inherited.
7. Grammar specification (owned by this RFC)
A compact EBNF; the canonical machine-readable grammar lives beside the parser and is snapshot-tested (§8). Kept small and regular so it doubles as a constrained-decoding grammar for the MCP surface (§3.6).
query = predicate , { "|" , stage } ;
predicate = or_expr ;
or_expr = and_expr , { ("or" | "||") , and_expr } ;
and_expr = unary , { ("and" | "&&") , unary } ;
unary = [ "not" | "!" ] , ( comparison | call | bool_lit | "(" , predicate , ")" ) ;
bool_lit = "true" | "false" ; (* match-all / match-none; a bare `true` = no filter *)
comparison = severity_cmp | scalar_cmp ;
severity_cmp = "severity" , ord_op , ( severity_name | number ) ;
scalar_cmp = scalar_path , cmp_op , literal ;
ord_op = "==" | "!=" | "<" | "<=" | ">" | ">=" ; (* no regex — severity is numeric *)
cmp_op = ord_op | "=~" | "!~" ;
call = str_fn , "(" , path , "," , string , ")"
| "resolves_to" , "(" , number , ")" ;
str_fn = "matches" | "contains" | "starts_with" | "ends_with" ;
path = field | "resource" , key_tail | "attr" , key_tail ;
scalar_path = nonsev_field | "resource" , key_tail | "attr" , key_tail ;
field = nonsev_field | "severity" ;
nonsev_field = "body" | "ts" | "observed_ts" | "trace_id" | "span_id"
| "scope" | "flags" | "service" | "template_id"
| "confidence" | "lossy" ;
key_tail = ( "." , dotted_key ) | ( "[" , string , "]" ) ;
dotted_key = ident , { "." , ident } ;
stage = "range" , "(" , time , "," , time , ")"
| "count" , [ "by" , field_list ]
| agg_fn , "(" , path , ")" , [ "by" , field_list ]
| "sort" , sort_key , [ "asc" | "desc" ]
| "limit" , integer
| "project" , field_list
| "render" ;
agg_fn = "sum" | "min" | "max" | "avg" ;
field_list = field , { "," , field } ;
sort_key = field | ident ; (* ident = an aggregate output, e.g. count *)
literal = string | number | boolean | "null" | duration | timestamp ;
severity_name = "trace" | "debug" | "info" | "warn" | "error" | "fatal" ; (* case-insensitive; only as a `severity` RHS *)
time = "now" | ( [ "-" ] , duration ) | timestamp ; (* e.g. now , -1h *)
integer = digit , { digit } ;
(* lexical: ident = letter , { letter | digit | "_" } ;
string = '"' , { char | escape } , '"' ;
char = any Unicode scalar except '"' , '\' , or a line terminator
(a literal newline must be written as the \n escape — queries
are single-line, §4 P7 / RFC0002.10) ;
escape = '\' , ( '"' | '\' | "n" | "t" | "r" | ( "u" , 4 * hex ) ) ;
number = integer | float ; float = integer , "." , digit , { digit } ;
boolean = "true" | "false" ;
duration = integer , ( "s"|"m"|"h"|"d"|"w" ) ; timestamp = RFC 3339 ;
digit = "0".."9" ; letter = "a".."z" | "A".."Z" ;
hex = digit | "a".."f" | "A".."F"
— strings are double-quoted with backslash escapes; YAML embedding
(RFC0002.10) wraps the whole query in a single-quoted YAML scalar so
these double quotes need no YAML-level escaping *)
8. Testing strategy
Mapping to CLAUDE.md §6.2 and docs/verification.md §3 (red→green
two-loop: #[ignore]’d stubs first, implementations second).
- Unit tests — every grammar production has a positive and negative parse test.
- Property tests — generate well-formed queries; assert the §5 round-trip idempotence (RFC0002.7) and that every generated query is a YAML-safe single-line scalar (RFC0002.10).
- Compilation golden tests — every construct has a golden
LogicalPlan(debug-rendered) checked in; the no-leakage boundary (RFC0002.3) is a compile + string test. - Equivalence tests — string vs structured surface compile to the
same IR (RFC0002.2); a DSL query and the equivalent
ourios_querierstructured request return identical results +QueryStats(RFC0002.1). - Grammar snapshot — the EBNF / parser grammar is committed and snapshot-tested so changes are PR-visible (Branch B owns its grammar).
- End-to-end — against the
docs/benchmarks.md§1 corpora, pinned expected results for a query set spanning each construct.
9. Open questions
Narrowed by the §3 resolution. Must be resolved before accepted.
- Pre-
acceptedvalidation. The §3.6 audience analysis stands in for instinct, not for evidence: beforeaccepted, run a readability pass on 10–20 sample queries with non-author reviewers, and a migration sketch from LogQL/Insights into β. (Replaces the prior §9 user-research gate; not required forspecified.) -
OTel ecosystem alignmentResolved: OpenTelemetry defines the logs data model + API/SDK but no standard query/read language, and OTTL is a Collector transformation language, not a querying surface (see the OTTL README and the OTel logs spec linked in §11 References). There is no canonical OTel read syntax, and no Perses-specific query convention, to align to. Bespoke query syntax over the OTel data model is the norm (LogQL, PromQL, CloudWatch Insights), so Branch B carries no ecosystem-divergence cost — the alignment that matters is at the field semantics, which §6.1/§6.2 honour (ts/observed_ts/trace_id/span_id/flags/body/scope/severity→ canonical data-model fields;severityordering onSeverityNumber). -
--sqladvanced-mode escape hatch — gated + sandboxed, or never? (Currently: never; reconsider under a separate RFC.) - Custom user functions — out for v1 (sandboxing is its own project).
-
params[N]positional access vs named parameters via the template schema. - In-path query cost estimator (“this will scan 400 GB”) before run.
- Pagination / streaming surface for large result sets (mirrors RFC 0007 §8).
Resolved by this RFC (were open in the draft): branch (B), top-level surface (β), severity-text casing (case-insensitive, §6.1), agent- friendliness (the structured surface, §6.4), and first-class OTel- canonical fields (§6.2).
10. Alternatives considered
Alternatives that would replace the whole design, not just one branch.
- Pure SQL (DataFusion dialect) — zero parser cost, but violates
CLAUDE.md§4 hazard 6 (cross-tenant joins, unbounded scans) and binds the user surface to DataFusion. Rejected as default; possible future gated, sandboxed escape hatch under a separate RFC. - LogQL clone — label selectors are less expressive than the OTel log record; adopting them flattens structure and lies about the ingest contract. Rejected as the full DSL; its top-level shape survives as the chosen β surface.
- CloudWatch Insights clone — proprietary, no open spec; attribute model differs from OTel. Rejected; its verb-per-line readability is the γ alternative we did not pick.
- Branch A (borrow OTTL) on any surface — see §3; not chosen for the Perses/MCP audiences.
11. References
- OpenTelemetry log data model: https://opentelemetry.io/docs/specs/otel/logs/data-model/
- OpenTelemetry severity text conventions: https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-severitytext
- OTTL (reference-only under Branch B): https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl
- LogQL: https://grafana.com/docs/loki/latest/query/
- CloudWatch Logs Insights: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html
- Perses (CNCF dashboards-as-code): https://perses.dev/
- Apache DataFusion logical-plan documentation.
- RFC 0001 §6.1/§6.3/§6.7 (the columns + template/drift primitives);
RFC 0007 (the execution layer this DSL targets);
CLAUDE.md§4 hazard 6 (no-leakage hazard) andCLAUDE.md§3.7 (multi-tenancy).