rfc: add RFC 0056, one language for every graph operation - #670
rfc: add RFC 0056, one language for every graph operation#670azimafroozeh wants to merge 2 commits into
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
I propose centering this RFC on a shared typed operation system, with GQ as its textual frontend and deliberately scoped public interfaces. This builds on the useful compiler-owned syntax and handler reuse already described.
One way to represent these layers is:
flowchart TB
subgraph inputs["Language and input layer"]
CLI["CLI arguments"]
GQ["GQ text"] --> Compiler["GQ compiler"]
API["Structured API requests"]
end
CLI --> Operations
Compiler --> Operations
API --> Operations
Operations["Shared typed operations<br/>Targets and result contracts"]
subgraph owners["Execution owners"]
Engine["Graph engine<br/>Planning, policy, execution and recovery"]
Cluster["Cluster controller<br/>Plan / apply and cluster authority checks"]
end
Operations --> Engine
Operations --> Cluster
Engine --> GraphState["Graph publication"]
Cluster --> ClusterState["Cluster resources and applied ledger"]
style Compiler fill:#fff0c2,stroke:#b58500
style Operations fill:#dceeff,stroke:#3273a8
style Cluster fill:#fde2e2,stroke:#b94a48
CLI commands and structured APIs can construct operations directly; GQT can use the GQ frontend. Shared contracts do not require one HTTP endpoint, a single transaction model, or routing every operation through relational query lowering.
In this model, every frontend would preserve three independent checks:
- Interface support: is this semantic operation qualified for the selected interface and connection?
- Authorization: may this actor perform it under the existing grants and policy?
- Execution authority: does it pass through the owning graph, cluster, or maintenance protocol?
This preserves the product model in which agents manage operational state through CLI operations. A branch represents a maintained world in the graph engine; GQ expresses work against it. For future branches with different schemas, schema-dependent compilation would use the selected branch/snapshot's accepted schema. Developing a branch-local schema proposal and adopting it into a cluster-managed deployment are distinct operations; adoption would respect the cluster plan/apply boundary. This is a proposed direction, not a claim that divergent branch schemas ship in this RFC.
The inline comments cover the shared contract, public exposure and interaction with #675, the cluster gate, exact publication receipts, and meaningful GQT assertions. This is a design/source review of 38724aa; no runtime test results are claimed.
| There is no typechecker or lowerer arm: | ||
| `typecheck_query_decl`, `typecheck_query` | ||
| (`crates/omnigraph-compiler/src/query/typecheck.rs:98, 110`), and | ||
| `lower_query` (`crates/omnigraph-compiler/src/ir/lower.rs:34-38`) take a | ||
| `QueryDecl` and never see the file, as RFC 0055's Decision log records. | ||
| The arm refuses wherever the consumer wants a declaration, and dispatches | ||
| in the runner, the CLI, and the server. The compiler classifies and never | ||
| refuses a door. |
There was a problem hiding this comment.
I propose defining shared typed operation, target and result contracts beneath the input syntaxes.
ControlStmt provides typed syntax, and extracting handler bodies provides real reuse. A shared contract would let GQ, CLI arguments and structured API requests construct the same operation and use its common preparation and execution rules. Existing CLI verbs could construct operations directly, without generating and reparsing GQ text.
In this model, GQ sits in the language layer above those contracts, while operation-specific planning, authorization and publication stay with their existing owners. Branch operations would not need a relational typechecker/lowerer arm, and this would not require one giant enum or a new transaction manager. The additional commitment would be semantic agreement across adapters, including which accepted state and result an operation uses.
| Clients. Every client of `POST /mutate` becomes a load, schema, and (after | ||
| phase 5) maintenance client, as RFC 0055 says for branches: server-side | ||
| Cedar is the gate on both fronts, so no actor's permissions widen, and a | ||
| client that withheld these operations while exposing a `mutate` entry | ||
| point must classify the source it sends or accept the change. A client |
There was a problem hiding this comment.
I propose separating public interface scope from language support, and qualifying the semantic operation independently of its carrier.
This would let GQ express an operation without automatically exposing it through every /query, /mutate, CLI or agent tool. A query tool could remain narrowly scoped even when its actor has permission to create a branch through another tool. Interface support and Cedar authorization answer different questions.
A shared operation system with deliberate public subsets seems worth evaluating as an explicit alternative. There is also a concrete interaction with PR #675: RFC 0059 qualifies managed query/mutate but defers managed load and branches. Under this RFC, a load written inside mutate -e could reach the same connection despite that deferred status. This is a qualification gap, not a Cedar bypass.
The proposed check would use compiler-owned classification to evaluate semantic operation × selected interface/connection before execution. Dedicated commands and their GQ equivalents would share the same support decision unless the expanded GQ operation set had been explicitly qualified. A useful regression would demonstrate that an unqualified operation remains refused when expressed through a generic GQ carrier.
| `.pg` body's working path is embedded: `omnigraph mutate --store` on a | ||
| standalone graph root, and the logic-test runner. A cluster-managed graph | ||
| keeps changing schema through `cluster apply` only ("Direct schema apply | ||
| and the server schema-apply endpoint refuse cluster-managed graphs", | ||
| `docs/user/schema/index.md:139-140`); the statement opens no second path | ||
| and adds no rule (Unresolved questions, Q4). RFC 0011's "Definitions are |
There was a problem hiding this comment.
I propose making the existing cluster-schema refusal an explicit invariant of the shared operation system.
This section already preserves the cluster-owned schema gate. Extending that guarantee into the shared model and summary would clarify that planned changes to cluster-managed configuration—graph lifecycle, deployed schemas, stored queries, policies and provider bindings—go through the cluster planning/apply protocol, regardless of their input syntax or the caller's data-mutation permissions.
Plan computes the proposed change; apply executes through the owning authority checks and records achieved state. An alternative configuration writer could change a live schema or policy while the desired definition and applied ledger still describe the old state, causing drift in subsequent plans, applies or serving activation. Locating the ownership check below the frontends, with tests for refusal without resource/ledger effects across alternate spellings and access paths, would protect this boundary.
The proposal would preserve the existing protocol distinctions: managed apply executes the exact saved plan, direct Core apply retains its replan-under-lock contract, and refresh/import/recovery keep their constrained authority. Standalone graph schema application would remain a separate supported path. A schema migration can also rewrite data, so ownership determines the gate more accurately than a blanket data-versus-metadata distinction.
This would not require adding cluster GQ syntax. It would establish that language expansion preserves cluster authority, and distinguish future branch-local schema proposals from their adoption into cluster-managed deployment configuration.
| otherwise: `loaded` carries the receipt's commit; `schema_applied` carries | ||
| the head of `main` after the apply, read as the first entry of | ||
| `list_commits(None)`, which is newest first (`omnigraph.rs:3599-3606`), |
There was a problem hiding this comment.
I propose returning schema apply's exact publication receipt instead of reading the latest head after the operation.
Reading the first list_commits(None) entry afterward does not identify the commit produced by that apply. For example: schema apply publishes S and releases its gate; another mutation publishes M; the follow-up history read returns M, which is then incorrectly reported as the schema receipt.
The publishing operation could return the exact commit identity, or an immutable publication identity from which it can be resolved. Sharing that result contract across CLI, API and GQ adapters would preserve the same receipt semantics. A regression could interleave a later commit before response assembly and verify that the response still identifies the schema commit.
The current schema apply path releases its operation gate before the caller can perform this additional history read.
| needed rebuilding (`table_ops.rs:46-52`); `optimized`, `repaired`, and | ||
| `cleaned` are `null`, since they publish manifest versions and not one | ||
| graph commit. |
There was a problem hiding this comment.
I propose aligning maintenance results with the graph commits those operations actually publish.
Optimize and repair can publish graph lineage commits, not only manifest versions. Both call commit_updates_with_actor_with_expected: optimize and repair. The existing optimize test explicitly requires one additional graph commit.
The result could return the actual publication receipt when one is produced, or explicitly identify an unavailable receipt rather than explain it as absence of a commit. Describing preview, no-op, cleanup and partial outcomes individually would also make the contract precise; not every maintenance invocation creates a commit.
| maintenance statements take `ok` or `error: <substring>`; their | ||
| statistics are not a test subject, the rows a following `--- query` | ||
| reads are. `ok` on `cleanup` and on `repair` accepts per-dataset | ||
| `error` and `action: refused`, since the engine returns `Ok` with them | ||
| set; a case that must see none reads the statistics through a following | ||
| `--- query`. |
There was a problem hiding this comment.
I propose adding structured assertions on operation results and making the remaining GQT coverage limits explicit.
Here ok accepts per-dataset errors and refused repairs, but the proposed follow-up query for those statistics is not defined. The statistics belong to the preceding ChangeOutput.outcome; the runner has neither a result capture nor a read operation that retrieves them. Unchanged logical rows can also pass when maintenance does nothing.
Assertions on the returned outcome would let workflows distinguish success, refusal and partial failure. Another option would be to retain those guarantees in their Rust owners and narrow the follow-up-query claim. The later limitations on nested results and dynamic commit IDs similarly leave a choice between explicit harness extensions and a narrower workflow-coverage claim.
In either approach, the compiler would remain the owner of operation syntax. Assertion/result-capture features belong to the harness and do not require a second branch grammar. Existing process, engine and simulation tests would continue to cover CLI routing and credentials, HTTP policy and transport, physical maintenance, concurrency and recovery.
Draft, not under review yet. A placeholder so the direction has a durable home beside the branch-statements RFC (#626); the file carries the same banner. Review opens later.
What & why
Adds RFC 0056, one language for every graph operation, as a
draft, plus its registry row and the next-available bump. It is the contract for every operation a graph exposes; the branch family is part of it and is delegated to #626, which specifies that family. Neither PR waits on the other.optimize,rebuild-full-text-indexes,repair, andcleanuponly through a shell with storage credentials (no route, so no Cedar, no server audit). An agent switches channels mid-task, every new client re-implements every operation, and.gqtcan test only what GQ can say, so the loader (bug: JSONL load stores NULL for a Date or DateTime given a float, bool, or object #628), schema apply, and maintenance have Rust-only regressions or none.load,schema(show,apply,plan),snapshot,commit(list,show,changes),changes poll,blob stat,queries list,optimize,rebuild full text indexes,repair,cleanup.POST /mutateandomnigraph mutate, control reads throughPOST /queryandomnigraph query. Same Cedar action and scope pair, same handler body, same envelope as the route today, with one addition:ReadOutputgains an optional skip-when-absentpageobject carrying the continuation fields and header-level values a paged or header-bearing family answers, so every existing read stays byte-identical.optimize,repair, andcleanuphave no Cedar action today and getmaintain, enforced at the HTTP boundary and in three new engine_asentries..pgschema) travels inline in a dollar-quoted raw block,$$ … $$, one opaque token to the grammar.omnigraph mutate --store); the served form is a submitted job, as RFC 0011 Decision 11 commits, and lands with that decision's mechanism RFC, which owns the served answer kind, the job record, and any statement observing a job.repair confirm forceis never served: forced repair stays a direct break-glass verb.affected:holds on aloadstep,outcome:gains words per family, no new directive.VACUUMandCOPY, Dolt'sDOLT_GC, Cypher administration commands, DuckDBCHECKPOINT. Simplest competitor is Alternatives 4, the composition of per-operation.gqtdirectives, Decision 11's job routes, and the existing routes wrapped as RFC 0003 tools; the minus-one designs for the two new mechanisms are Alternatives 12 (nomaintainaction) and 13 (payload as a string literal).Backing issue / RFC
docs/rfcs/0056-gq-one-language.md(no prior issue; motivated by every operation past branches needing its own channel, and the maintenance verbs having no governed channel at all)Checklist
Local verification
python3 scripts/check-docs.py: Documentation OK (130 Markdown files checked)bash scripts/check-agents-md.sh: AGENTS.md ↔ docs indexes OK (46 links, 44 docs)cargo check --workspace: green (docs-only change, run for the record)Notes for reviewers
status: draft,implementation: not-started. Merging publishes the draft for review and authorizes nothing. Everyfile:lineanchor is at the upstream commit named in Motivation; the branch-statements RFC is cited by section because rfc: add branch statements in GQ #626 is still under review.blocked_onis empty. rfc: add branch statements in GQ #626 is not a gate: it names this RFC as the direction its rules extend toward, and each is reviewed on its own. The RFC 0011 Decision 11 mechanism RFC (job routes, worker, scheduling) is what phase 5 lands with, not a gate on this document; phases 1 to 4 do not wait on it. The agent-experience evaluation is an evidence obligation, not a dependency, so it sits in Evidence and tests with its task count, artifact, runner, model-recording rule, and threshold fixed there./mutatebody cap, bulk loads keepPOST /load/ndjsonand the verb,from <uri>is Alternatives 6;cleanupwithoutconfirmis a 400 whilerepairwithoutconfirmstays a preview at the write door, as the verbs behave;schema planstays embedded-only and runs under no Cedar pair, so no RFC 0011 capability word changes anywhere in this RFC.init, the streaming and byte routes (export,changes baseline,blob get, Alternatives 11), the cluster control plane, session state, storable or parameterized statements (Alternatives 8)._asentries, in phase 4; every route and verb stays, nothing deprecated. Q1 to Q5 sit in Unresolved questions, the body written to each lean, each naming the event that forces it and what changes if decided the other way.