Skip to content

Make AST spans and visitors reliable for source transformations - #231

Merged
someone235 merged 7 commits into
kaspanet:masterfrom
michaelsutton:source-aware-ast-api
Aug 29, 2026
Merged

Make AST spans and visitors reliable for source transformations#231
someone235 merged 7 commits into
kaspanet:masterfrom
michaelsutton:source-aware-ast-api

Conversation

@michaelsutton

@michaelsutton michaelsutton commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Context

Argent uses the Sil parser and mutable AST visitor to transform embedded Sil source.

It uses the AST to identify semantic nodes. It then edits their ranges in the original source, reparses the result, and checks that the transformation is complete.

This process requires correct source spans and complete AST traversal.

Problem

Some composed expressions had incomplete spans.

For example:

(signed(next_status) + signed(increment)) as byte
(values)[index].length

Sil removed the grouping node before it built the outer expression. The resulting span could contain the closing parenthesis but omit the opening parenthesis.

Other API gaps required consumers to:

  • wrap a statement in a synthetic function before parsing it;
  • infer some type locations from containing nodes;
  • maintain their own list of type-bearing AST forms;
  • skip semantic elements when deserialized AST metadata had no source spans.

Change

This PR:

  • preserves complete spans for composed expressions;
  • exposes source spans for previously incomplete authored-type sites;
  • adds parse_statement_ast for exactly one standalone statement;
  • adds visit_type for AST-classified type occurrences;
  • completes contract and struct traversal;
  • visits semantic return types and attribute path segments even when source-span metadata is absent.

Plain parentheses remain normalized away. Exact inner spans remain available. For example, the identifier inside ((value)) still has the name span value.

Sil remains a semantic AST. This PR does not make it a concrete syntax tree and does not preserve comments or formatting through AST rendering.

Consumer

Argent PR #52 uses these APIs to remove source-transformation workarounds.

Argent will update its Sil revision to the merged master commit before it merges.

Validation

Tests cover:

  • composed expression spans;
  • authored-type spans;
  • standalone statement parsing;
  • all current type-bearing AST forms;
  • nested contract and struct traversal;
  • AST values deserialized without source-span metadata.

someone235
someone235 previously approved these changes Aug 29, 2026
@someone235
someone235 force-pushed the source-aware-ast-api branch from ec41d86 to 7f1fa35 Compare August 29, 2026 06:17
Build infix and postfix spans from parser-pair extents instead of normalized child AST spans. This retains grouping delimiters and prevents syntactically incomplete source ranges.

Cover casts, field and index access, suffix operations, chained postfix expressions, and grouped infix operands while keeping identifier-name spans exact.
Add exact target-struct and typed-array type spans to the public AST for source transformations. Capture the spans during parsing and preserve them through compiler rewrites.

Include the new spans in mutable visitor traversal and cover exact source ranges plus span mutation in regression tests.
Expose a parser entry point for exactly one Sil statement without wrapping it in a synthetic function. Returned nodes retain spans into the caller's original source.

Cover required terminators, surrounding comments, supported statement shapes, and rejection of incomplete or multiple statements.
Add a uniform visit_type callback across declarations, bindings, typed arrays, and struct-oriented syntax. String-backed struct references are exposed as temporary scalar custom TypeRefs, while their source spans remain anchored at the authored base name.

Complete contract traversal through structs and classify struct declaration names. Cover all type-bearing AST shapes, leading trivia, span mutation order, and nested expression traversal.
Drive return-type and attribute-path traversal from semantic vectors, supplying synthetic spans when deserialized AST metadata is absent.

Cover both callbacks with a serialized AST regression.
@someone235
someone235 force-pushed the source-aware-ast-api branch from 7f1fa35 to 77c9837 Compare August 29, 2026 06:22
@someone235
someone235 merged commit 73e33e2 into kaspanet:master Aug 29, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants