Skip to content

rfc: add RFC 0056, one language for every graph operation - #670

Open
azimafroozeh wants to merge 2 commits into
ModernRelay:mainfrom
azimafroozeh:rfc-gq-one-language
Open

rfc: add RFC 0056, one language for every graph operation#670
azimafroozeh wants to merge 2 commits into
ModernRelay:mainfrom
azimafroozeh:rfc-gq-one-language

Conversation

@azimafroozeh

@azimafroozeh azimafroozeh commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Draft, not under review yet. A placeholder so the direction has a durable home beside the branch-statements RFC (#626); the file carries the same banner. Review opens later.

What & why

Adds RFC 0056, one language for every graph operation, as a draft, plus its registry row and the next-available bump. It is the contract for every operation a graph exposes; the branch family is part of it and is delegated to #626, which specifies that family. Neither PR waits on the other.

  • Past reads, mutations, and (after rfc: add branch statements in GQ #626) branches, every operation is reachable only through its own route and CLI verb; optimize, rebuild-full-text-indexes, repair, and cleanup only through a shell with storage credentials (no route, so no Cedar, no server audit). An agent switches channels mid-task, every new client re-implements every operation, and .gqt can test only what GQ can say, so the loader (bug: JSONL load stores NULL for a Date or DateTime given a float, bool, or object #628), schema apply, and maintenance have Rust-only regressions or none.
  • The RFC makes the rules rfc: add branch statements in GQ #626 spells out for branches the contract of a control statement and adds one family per operation: load, schema (show, apply, plan), snapshot, commit (list, show, changes), changes poll, blob stat, queries list, optimize, rebuild full text indexes, repair, cleanup.
  • Classification picks the door: control writes through POST /mutate and omnigraph mutate, control reads through POST /query and omnigraph query. Same Cedar action and scope pair, same handler body, same envelope as the route today, with one addition: ReadOutput gains an optional skip-when-absent page object carrying the continuation fields and header-level values a paged or header-bearing family answers, so every existing read stays byte-identical.
  • One new permission path, owned: optimize, repair, and cleanup have no Cedar action today and get maintain, enforced at the HTTP boundary and in three new engine _as entries.
  • A payload (loader rows, a .pg schema) travels inline in a dollar-quoted raw block, $$ … $$, one opaque token to the grammar.
  • Maintenance runs embedded now (the logic-test runner, omnigraph mutate --store); the served form is a submitted job, as RFC 0011 Decision 11 commits, and lands with that decision's mechanism RFC, which owns the served answer kind, the job record, and any statement observing a job. repair confirm force is never served: forced repair stays a direct break-glass verb.
  • RFC 0045 amendment rides rfc: add branch statements in GQ #626's: a statement is a step body, affected: holds on a load step, outcome: gains words per family, no new directive.
  • Precedent: SQL VACUUM and COPY, Dolt's DOLT_GC, Cypher administration commands, DuckDB CHECKPOINT. Simplest competitor is Alternatives 4, the composition of per-operation .gqt directives, Decision 11's job routes, and the existing routes wrapped as RFC 0003 tools; the minus-one designs for the two new mechanisms are Alternatives 12 (no maintain action) and 13 (payload as a string literal).

Backing issue / RFC

  • Is an RFC PR: docs/rfcs/0056-gq-one-language.md (no prior issue; motivated by every operation past branches needing its own channel, and the maintenance verbs having no governed channel at all)

Checklist

  • Change is focused (one RFC file, registry row, next-available bump)
  • Tests added/updated for behavior changes (N/A, docs-only)
  • Public docs updated if user-facing surface changed (N/A, a draft RFC changes no surface)
  • Reviewed against docs/dev/invariants.md — no Hard Invariant weakened, no deny-list item hit (no code in the diff; the RFC's Invariants section names 2, 4, 7, 9, 10, 13 and the two deny-list items a served maintenance job would touch)

Local verification

  • python3 scripts/check-docs.py: Documentation OK (130 Markdown files checked)
  • bash scripts/check-agents-md.sh: AGENTS.md ↔ docs indexes OK (46 links, 44 docs)
  • cargo check --workspace: green (docs-only change, run for the record)

Notes for reviewers

  • status: draft, implementation: not-started. Merging publishes the draft for review and authorizes nothing. Every file:line anchor is at the upstream commit named in Motivation; the branch-statements RFC is cited by section because rfc: add branch statements in GQ #626 is still under review.
  • blocked_on is empty. rfc: add branch statements in GQ #626 is not a gate: it names this RFC as the direction its rules extend toward, and each is reviewed on its own. The RFC 0011 Decision 11 mechanism RFC (job routes, worker, scheduling) is what phase 5 lands with, not a gate on this document; phases 1 to 4 do not wait on it. The agent-experience evaluation is an evidence obligation, not a dependency, so it sits in Evidence and tests with its task count, artifact, runner, model-recording rule, and threshold fixed there.
  • Decisions to weigh: two doors kept, one door out of scope rather than open (Alternatives 3); the inline payload is the agent and test form under the 1 MiB /mutate body cap, bulk loads keep POST /load/ndjson and the verb, from <uri> is Alternatives 6; cleanup without confirm is a 400 while repair without confirm stays a preview at the write door, as the verbs behave; schema plan stays embedded-only and runs under no Cedar pair, so no RFC 0011 capability word changes anywhere in this RFC.
  • Out, each with its reason in the body: init, the streaming and byte routes (export, changes baseline, blob get, Alternatives 11), the cluster control plane, session state, storable or parameterized statements (Alternatives 8).
  • Engine crate gains only the three _as entries, in phase 4; every route and verb stays, nothing deprecated. Q1 to Q5 sit in Unresolved questions, the body written to each lean, each naming the event that forces it and what changes if decided the other way.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@aaltshuler aaltshuler left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose centering this RFC on a shared typed operation system, with GQ as its textual frontend and deliberately scoped public interfaces. This builds on the useful compiler-owned syntax and handler reuse already described.

One way to represent these layers is:

flowchart TB
    subgraph inputs["Language and input layer"]
        CLI["CLI arguments"]
        GQ["GQ text"] --> Compiler["GQ compiler"]
        API["Structured API requests"]
    end

    CLI --> Operations
    Compiler --> Operations
    API --> Operations

    Operations["Shared typed operations<br/>Targets and result contracts"]

    subgraph owners["Execution owners"]
        Engine["Graph engine<br/>Planning, policy, execution and recovery"]
        Cluster["Cluster controller<br/>Plan / apply and cluster authority checks"]
    end

    Operations --> Engine
    Operations --> Cluster
    Engine --> GraphState["Graph publication"]
    Cluster --> ClusterState["Cluster resources and applied ledger"]

    style Compiler fill:#fff0c2,stroke:#b58500
    style Operations fill:#dceeff,stroke:#3273a8
    style Cluster fill:#fde2e2,stroke:#b94a48
Loading

CLI commands and structured APIs can construct operations directly; GQT can use the GQ frontend. Shared contracts do not require one HTTP endpoint, a single transaction model, or routing every operation through relational query lowering.

In this model, every frontend would preserve three independent checks:

  • Interface support: is this semantic operation qualified for the selected interface and connection?
  • Authorization: may this actor perform it under the existing grants and policy?
  • Execution authority: does it pass through the owning graph, cluster, or maintenance protocol?

This preserves the product model in which agents manage operational state through CLI operations. A branch represents a maintained world in the graph engine; GQ expresses work against it. For future branches with different schemas, schema-dependent compilation would use the selected branch/snapshot's accepted schema. Developing a branch-local schema proposal and adopting it into a cluster-managed deployment are distinct operations; adoption would respect the cluster plan/apply boundary. This is a proposed direction, not a claim that divergent branch schemas ship in this RFC.

The inline comments cover the shared contract, public exposure and interaction with #675, the cluster gate, exact publication receipts, and meaningful GQT assertions. This is a design/source review of 38724aa; no runtime test results are claimed.

Comment on lines +560 to +567
There is no typechecker or lowerer arm:
`typecheck_query_decl`, `typecheck_query`
(`crates/omnigraph-compiler/src/query/typecheck.rs:98, 110`), and
`lower_query` (`crates/omnigraph-compiler/src/ir/lower.rs:34-38`) take a
`QueryDecl` and never see the file, as RFC 0055's Decision log records.
The arm refuses wherever the consumer wants a declaration, and dispatches
in the runner, the CLI, and the server. The compiler classifies and never
refuses a door.

@aaltshuler aaltshuler Sep 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose defining shared typed operation, target and result contracts beneath the input syntaxes.

ControlStmt provides typed syntax, and extracting handler bodies provides real reuse. A shared contract would let GQ, CLI arguments and structured API requests construct the same operation and use its common preparation and execution rules. Existing CLI verbs could construct operations directly, without generating and reparsing GQ text.

In this model, GQ sits in the language layer above those contracts, while operation-specific planning, authorization and publication stay with their existing owners. Branch operations would not need a relational typechecker/lowerer arm, and this would not require one giant enum or a new transaction manager. The additional commitment would be semantic agreement across adapters, including which accepted state and result an operation uses.

Comment on lines +1105 to +1109
Clients. Every client of `POST /mutate` becomes a load, schema, and (after
phase 5) maintenance client, as RFC 0055 says for branches: server-side
Cedar is the gate on both fronts, so no actor's permissions widen, and a
client that withheld these operations while exposing a `mutate` entry
point must classify the source it sends or accept the change. A client

@aaltshuler aaltshuler Sep 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose separating public interface scope from language support, and qualifying the semantic operation independently of its carrier.

This would let GQ express an operation without automatically exposing it through every /query, /mutate, CLI or agent tool. A query tool could remain narrowly scoped even when its actor has permission to create a branch through another tool. Interface support and Cedar authorization answer different questions.

A shared operation system with deliberate public subsets seems worth evaluating as an explicit alternative. There is also a concrete interaction with PR #675: RFC 0059 qualifies managed query/mutate but defers managed load and branches. Under this RFC, a load written inside mutate -e could reach the same connection despite that deferred status. This is a qualification gap, not a Cedar bypass.

The proposed check would use compiler-owned classification to evaluate semantic operation × selected interface/connection before execution. Dedicated commands and their GQ equivalents would share the same support decision unless the expanded GQ operation set had been explicitly qualified. A useful regression would demonstrate that an unqualified operation remains refused when expressed through a generic GQ carrier.

Comment on lines +1083 to +1088
`.pg` body's working path is embedded: `omnigraph mutate --store` on a
standalone graph root, and the logic-test runner. A cluster-managed graph
keeps changing schema through `cluster apply` only ("Direct schema apply
and the server schema-apply endpoint refuse cluster-managed graphs",
`docs/user/schema/index.md:139-140`); the statement opens no second path
and adds no rule (Unresolved questions, Q4). RFC 0011's "Definitions are

@aaltshuler aaltshuler Sep 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose making the existing cluster-schema refusal an explicit invariant of the shared operation system.

This section already preserves the cluster-owned schema gate. Extending that guarantee into the shared model and summary would clarify that planned changes to cluster-managed configuration—graph lifecycle, deployed schemas, stored queries, policies and provider bindings—go through the cluster planning/apply protocol, regardless of their input syntax or the caller's data-mutation permissions.

Plan computes the proposed change; apply executes through the owning authority checks and records achieved state. An alternative configuration writer could change a live schema or policy while the desired definition and applied ledger still describe the old state, causing drift in subsequent plans, applies or serving activation. Locating the ownership check below the frontends, with tests for refusal without resource/ledger effects across alternate spellings and access paths, would protect this boundary.

The proposal would preserve the existing protocol distinctions: managed apply executes the exact saved plan, direct Core apply retains its replan-under-lock contract, and refresh/import/recovery keep their constrained authority. Standalone graph schema application would remain a separate supported path. A schema migration can also rewrite data, so ownership determines the gate more accurately than a blanket data-versus-metadata distinction.

This would not require adding cluster GQ syntax. It would establish that language expansion preserves cluster authority, and distinguish future branch-local schema proposals from their adoption into cluster-managed deployment configuration.

Comment on lines +678 to +680
otherwise: `loaded` carries the receipt's commit; `schema_applied` carries
the head of `main` after the apply, read as the first entry of
`list_commits(None)`, which is newest first (`omnigraph.rs:3599-3606`),

@aaltshuler aaltshuler Sep 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose returning schema apply's exact publication receipt instead of reading the latest head after the operation.

Reading the first list_commits(None) entry afterward does not identify the commit produced by that apply. For example: schema apply publishes S and releases its gate; another mutation publishes M; the follow-up history read returns M, which is then incorrectly reported as the schema receipt.

The publishing operation could return the exact commit identity, or an immutable publication identity from which it can be resolved. Sharing that result contract across CLI, API and GQ adapters would preserve the same receipt semantics. A regression could interleave a later commit before response assembly and verify that the response still identifies the schema commit.

The current schema apply path releases its operation gate before the caller can perform this additional history read.

Comment on lines +683 to +685
needed rebuilding (`table_ops.rs:46-52`); `optimized`, `repaired`, and
`cleaned` are `null`, since they publish manifest versions and not one
graph commit.

@aaltshuler aaltshuler Sep 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose aligning maintenance results with the graph commits those operations actually publish.

Optimize and repair can publish graph lineage commits, not only manifest versions. Both call commit_updates_with_actor_with_expected: optimize and repair. The existing optimize test explicitly requires one additional graph commit.

The result could return the actual publication receipt when one is produced, or explicitly identify an unavailable receipt rather than explain it as absence of a commit. Describing preview, no-op, cleanup and partial outcomes individually would also make the contract precise; not every maintenance invocation creates a commit.

Comment on lines +772 to +777
maintenance statements take `ok` or `error: <substring>`; their
statistics are not a test subject, the rows a following `--- query`
reads are. `ok` on `cleanup` and on `repair` accepts per-dataset
`error` and `action: refused`, since the engine returns `Ok` with them
set; a case that must see none reads the statistics through a following
`--- query`.

@aaltshuler aaltshuler Sep 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose adding structured assertions on operation results and making the remaining GQT coverage limits explicit.

Here ok accepts per-dataset errors and refused repairs, but the proposed follow-up query for those statistics is not defined. The statistics belong to the preceding ChangeOutput.outcome; the runner has neither a result capture nor a read operation that retrieves them. Unchanged logical rows can also pass when maintenance does nothing.

Assertions on the returned outcome would let workflows distinguish success, refusal and partial failure. Another option would be to retain those guarantees in their Rust owners and narrow the follow-up-query claim. The later limitations on nested results and dynamic commit IDs similarly leave a choice between explicit harness extensions and a narrower workflow-coverage claim.

In either approach, the compiler would remain the owner of operation syntax. Assertion/result-capture features belong to the harness and do not require a second branch grammar. Existing process, engine and simulation tests would continue to cover CLI routing and credentials, HTTP policy and transport, physical maintenance, concurrency and recovery.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants