Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion compatibility/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,10 +142,15 @@ per fixture:

```sh
python3 compatibility/diff.py \
--producer-command 'opy-rs compat --fixture {fixture_id} --input {source} --output {result}' \
--producer-command 'your-producer --fixture {fixture_id} --input {source} --output {result}' \
--report compatibility/report.json
```

`your-producer` is a placeholder: this repository ships no producer for
`diff.py`. The native frontend comparison runs inside `cargo test`
(`crates/opy-frontend/tests/differential.rs`) against the recorded oracle
snapshots directly.

The producer must write a result with the same schema as the oracle's
`compile` record and the fixture id. It may additionally provide a top-level
`semantic` JSON value when a runtime-sensitive scenario has been executed.
Expand Down
11 changes: 7 additions & 4 deletions compatibility/fixtures/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
This directory is the opy-rs compatibility corpus: OPY sources with their
pinned-oracle snapshots (`oracle.json`), ported from the WrightKit project's
evidence base (wright `compatibility/fixtures/`) and re-verified against the
pinned OverPy 9.7.10 oracle on 2026-08-17 (all 42 snapshots match).
pinned OverPy 9.7.10 oracle (all 42 snapshots match; see
[`docs/compatibility/upstream-references.md`](../../docs/compatibility/upstream-references.md)
for the dated verification record).

Corpus policy: every fixture records provenance in its `fixture.json`
(`kind`, `origin`, `license`, `redistributable`, and — for imported
Expand All @@ -27,9 +29,10 @@ fixtures/<category>/<name>/

## Synthetic fixtures (WrightKit-authored)

`fixtures/synthetic/` — 27 fixtures authored for the WrightKit compatibility
corpus (same organization as opy-rs; AGPL-3.0-or-later, `kind: original`),
ported unchanged:
`fixtures/synthetic/` — 28 fixtures authored for the WrightKit compatibility
corpus (AGPL-3.0-or-later, `kind: original`): an initial set ported unchanged
from the wright repository corpus, extended by fixtures added in this
repository:

| Fixture | Covers |
| --- | --- |
Expand Down
2 changes: 1 addition & 1 deletion compatibility/support-matrix.json
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,7 @@
"fixtures:synthetic/declarations-rules",
"upstream:runTests.mjs"
],
"notes": "Structured stable-code diagnostics with 1-based spans; 13-code contract tested in opy-frontend tooling tests."
"notes": "Structured stable-code diagnostics with 1-based spans; the stable-code contract is documented in docs/opy/tooling-api.md and tested in opy-frontend tooling tests."
},
{
"id": "semantics/settings-emission",
Expand Down
6 changes: 3 additions & 3 deletions crates/opy-frontend/src/hir/types.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
//! Serde protocol types for `wright/opy-hir` version `1.0.0`.
//! Serde protocol types for the `wright/opy-hir` protocol, major version 1.
//!
//! These types mirror the Opy HIR v1 specification (`docs/hir/opy-hir-v1.md`
//! in the wright repository, `wright/opy-hir` v1.1.0 wire payloads). Unknown
//! These types mirror the Opy HIR v1 specification (`docs/hir/opy-hir-v1.md`,
//! `wright/opy-hir` v1.1.0 wire payloads). Unknown
//! fields on known nodes are tolerated so an additive producer change inside
//! the same major version does not break the consumer; unknown node *kinds*
//! are rejected during validation (see [`super::validate`]).
Expand Down
8 changes: 4 additions & 4 deletions crates/opy-frontend/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,10 @@
//! Workshop text produced by lowering and is lowering-dependent (workshop-rs
//! emission, issue #8); the frontend never fabricates a Workshop payload.
//!
//! This crate is the extraction of the mature Wright frontend
//! (`crates/wright-opy`); module provenance and issue references follow the
//! original implementation. Workshop→OPY reconstruction and the differential
//! harness are not part of this crate (see the opy-rs roadmap).
//! This crate was extracted from the mature Wright frontend (the wright
//! repository's `crates/wright-opy`); module provenance and issue references
//! follow the original implementation. Workshop→OPY reconstruction and the
//! differential harness are not part of this crate (see the opy-rs roadmap).

pub mod cst;
pub mod diag;
Expand Down
4 changes: 2 additions & 2 deletions crates/opy-frontend/src/manifest/probes/validate.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

Stdlib-only; run from anywhere:

python3 crates/wright-opy/src/manifest/probes/validate.py
python3 crates/opy-frontend/src/manifest/probes/validate.py

Exit code 0 only when every probe matches the recorded oracle evidence.
"""
Expand All @@ -31,7 +31,7 @@
import sys

HERE = os.path.dirname(os.path.abspath(__file__))
# crates/wright-opy/src/manifest/probes -> workspace root (5 levels up).
# crates/opy-frontend/src/manifest/probes -> repository root (5 levels up).
WORKSPACE = os.path.abspath(os.path.join(HERE, "..", "..", "..", "..", ".."))
ORACLE = os.path.join(
WORKSPACE, "compatibility", "oracle", "node_modules", "overpy", "cli.js"
Expand Down
7 changes: 5 additions & 2 deletions docs/compatibility/upstream-references.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,11 @@ never `latest` or a range (see the pinning policy below).
pinned `pnpm-lock.yaml`; `pnpm install` resolves `overpy@9.7.10` by its
integrity hash.
* The compatibility corpus was re-run against a fresh install of the pinned
package on 2026-08-17: all 27 fixture snapshots (`compatibility/fixtures/**/oracle.json`)
match byte-for-byte (run `python3 compatibility/run_oracle.py`).
package on 2026-08-17: all 27 fixture snapshots that existed at that date
(`compatibility/fixtures/**/oracle.json`) matched byte-for-byte. The corpus
has since grown to 42 fixtures, each carrying its own oracle snapshot from
the same pinned package; rerun `python3 compatibility/run_oracle.py` to
re-verify the full corpus.
* The imported example fixtures were verified byte-identical to the pinned
tree's `examples/` content (see `compatibility/fixtures/README.md`).

Expand Down
47 changes: 25 additions & 22 deletions docs/hir/opy-hir-v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,16 @@

Status: accepted baseline for v0.1; opy-rs-owned contract (adopted from the
WrightKit evidence base, issue #2)
Scope: the interchange format between the reference OverPy frontend adapter
and the opy-rs Rust core (and, later, WrightKit tooling consumers)
Scope: the interchange format produced by the opy-rs native frontend and
consumed by opy-rs tooling (and, later, WrightKit tooling consumers)

This document is the normative specification for the Opy HIR protocol version
`1.1.0`. It defines the JSON payload that the compatibility adapter emits and
that the Rust core (`crates/`) validates and consumes. The adapter is the only
component allowed to know how an OverPy AST maps onto this schema; the Rust
core sees only this protocol.
`1.1.0`. It defines the JSON payload that the native frontend in
`crates/opy-frontend` (the lowering stage) emits and that the Rust consumer
in the same crate validates and consumes. The frontend parses `.opy` source
directly and owns the mapping from OPY syntax onto this schema; no component
imports or wraps the reference implementation's AST (clean-room boundary,
[`upstream-references.md`](../compatibility/upstream-references.md)).

The protocol is an opy-rs-owned contract. It is not `JSON.stringify()` of an
OverPy AST, and no node in it is named after an OverPy-internal class. Node
Expand All @@ -29,11 +31,11 @@ The protocol must:
declarations, rules, events, conditions, statements, and expressions;
2. preserve file, line, and column provenance so later stages can report
diagnostics against source;
3. be deterministic: the same source, frontend version, and adapter version
produce byte-identical JSON;
3. be deterministic: the same source and frontend version produce
byte-identical JSON;
4. be versioned so a producer and consumer can agree on compatibility without
inspecting each other's implementation;
5. fail loudly on constructs the adapter cannot map, and be rejected or
5. fail loudly on constructs the frontend cannot map, and be rejected or
reported by the consumer rather than silently ignored.

## 2. Protocol envelope
Expand Down Expand Up @@ -183,8 +185,8 @@ are kept, and the oracle comparison itself uses status, rule-name, and
diagnostic evidence rather than span data.

Spans are for diagnostics and identity, not for byte-accurate reconstruction.
The adapter is responsible for producing them; the consumer validates them
(§8). A span whose end would precede its start (for example a node expanded
The frontend producer is responsible for emitting them; the consumer
validates them (§8). A span whose end would precede its start (for example a node expanded
from a preprocessor macro that mixes call-site and definition-site positions)
must be normalized to a degenerate interval anchored at the start, so every
emitted span is structurally valid.
Expand Down Expand Up @@ -381,9 +383,9 @@ Operators are opy-rs spellings for the semantics the frontend parsed:
* comparison: `== != < <= > >=` (non-strict, Workshop semantics);
* logical: `and or`, with `not` as a unary operator.

The adapter maps frontend nodes (`__add__`, `__equals__`, `__lessThan__`,
... ) onto these spellings. The consumer treats `op` as an opaque string and
validates it only structurally (§8).
The frontend maps parsed OPY operator syntax onto these fixed spellings. The
consumer treats `op` as an opaque string and validates it only structurally
(§8).

## 7. Versioning and compatibility rules

Expand Down Expand Up @@ -418,7 +420,7 @@ the program body.
A node with an unknown `kind` (or an unknown statement/expression variant) is
an *unsupported node*. The consumer reports a structured error that names the
node kind and its span, so a regression report is explicit. Unsupported is
never a silent pass: the adapter refuses to emit nodes it cannot map, and the
never a silent pass: the frontend refuses to emit nodes it cannot map, and the
consumer refuses to consume nodes it cannot understand.

## 8. Validation requirements
Expand Down Expand Up @@ -452,7 +454,7 @@ of the stable contract; the code and structured fields are.

## 9. Determinism and debug output

For the same input, frontend version, and adapter version, the producer must
For the same input and frontend version, the producer must
emit byte-identical JSON: object keys are emitted in a fixed order and
collections (files, declarations, rules, branches, args) preserve source
order. The consumer's debug dump (§10) must be stable for the same validated
Expand All @@ -473,7 +475,7 @@ implementation-defined presentation, not part of the wire contract. It must:
## 11. Out of scope for v1

The following are intentionally not modeled in v1 and are rejected by the
adapter as unsupported when encountered:
frontend as unsupported when encountered:

* rule labels and relative gotos (`__skip__` / `__distanceTo__` forms);
* decompilation-only constructs;
Expand All @@ -487,11 +489,12 @@ reason to extend the schema silently.
## 12. Ownership

* The protocol contract is owned by opy-rs and lives in this document.
* The adapter owns all knowledge of how OverPy ASTs map to this schema. It is
an optional, external-frontend component: the Rust core never imports it
and never depends on OverPy types.
* Changes to the node grammar require a review of this document, the adapter,
the Rust consumer, and the corpus fixtures together (see
* The native frontend owns all knowledge of how OPY source maps to this
schema. It never imports or depends on the reference implementation's
types (clean-room boundary,
[`upstream-references.md`](../compatibility/upstream-references.md)).
* Changes to the node grammar require a review of this document, the frontend
producer, the Rust consumer, and the corpus fixtures together (see
[`docs/opy/support-matrix.md`](../opy/support-matrix.md) and
[`docs/compatibility/upstream-references.md`](../compatibility/upstream-references.md)).

Expand Down
2 changes: 1 addition & 1 deletion docs/opy/support-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Workshop-independent up to the documented integration boundary toward

| Source | Use |
| --- | --- |
| `compatibility/fixtures/{basic-rule,control-flow,declarations-rules,declarations-numbers,expressions-values,preprocessing,diagnostics,settings,receiver-calls,chase-*}/source.opy` | Synthetic corpus surface (WrightKit-authored; oracle snapshots in the same directory) |
| `compatibility/fixtures/synthetic/{basic-rule,control-flow,declarations-rules,declarations-numbers,expressions-values,preprocessing,diagnostics,settings,receiver-calls,chase-*}/source.opy` | Synthetic corpus surface (WrightKit-authored; oracle snapshots in the same directory) |
| `compatibility/fixtures/real-world/overpy-*/` | Real-world surface from the pinned OverPy `examples/` tree (arrays, macros, effects, settings, subroutines, include closures; provenance in `compatibility/fixtures/README.md`) |
| `compatibility/fixtures/real-world/{ow1-emulator,6v6-adjustments}/` | Independent third-party projects (BSD-2-Clause), full include closures |
| `compatibility/fixtures/**/oracle.json` | Pinned OverPy 9.7.10 reference snapshots (normalized Workshop output, diagnostics, exit codes) |
Expand Down
5 changes: 5 additions & 0 deletions docs/opy/tooling-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,18 @@ pub struct CheckOutcome {
pub diagnostics: Vec<Diagnostic>, // empty ⟺ clean
pub model: Option<SemanticModel>, // present exactly when clean
pub files: Vec<FileRecord>, // file registry, retained on failure
pub post_compile_hook: Option<PostCompileHook>, // declared hook record
}
```

* `Diagnostic { severity, code, message, span }`: structured, stable-coded,
source-attributed (see the diagnostics contract below).
* `SourceLocation { file_id, path, start, end }`: a span resolved through
the file registry to `(file id, path, line/col)`.
* `PostCompileHook`: the declared `#!postCompileHook` script (root-relative
path plus directive span), present only when the source declared one and
the project checked clean. It is a declaration record only — the frontend
never executes the hook (execution is lowering-dependent, issue #8).

`SemanticModel` wraps the resolved program and answers queries:

Expand Down