feat(scanner): Add Go parser fallback

[reneleonhardt]

What does this PR do?

• Add a Go parser fallback so dependency analysis stays available when ast-grep is unavailable, times out, or fails: the public dependency scan routes through the fallback when the primary outcome is incomplete or degraded.
• Preserve honest provenance throughout: fallback outcomes carry explicit coverage sources, partial scans read as partial (never a confident negative), and an unrecoverable degraded primary fails closed with the degraded outcome preserved.
• Keep Cargo recovery graph-scoped and deduplicated: mixed Go/Cargo topology resolves without duplicate cargo metadata runs or duplicate cargo-metadata sources in the --deps payload.

Type of change

• Bug fix
• New feature
• New language support
• Documentation
• Other (describe below)

Checklist

• I've tested this locally with go build && ./codemap .
• I've read CONTRIBUTING.md; this does not add a new language.
• Documentation is unchanged because the feature is self-describing through scan provenance.

Additional notes

The Go fallback counts only dependency-bearing files in its recovery note, skips methods (matching ast-grep's function capture, so the files/functions set doesn't churn across authoritative→fallback transitions), and reports nothing for non-Go files that a partial scan cannot see.

Developed with carefully directed, manually reviewed AI assistance.

[JordanCoin]

Reviewed in depth. The core mechanism is sound and genuinely well-tested — with ast-grep missing, a Go repo now returns correct edges with honest provenance (ast-grep: unavailable + go-parser: fallback), byte-identical file list to the ast-grep run. That part is good work.

Two things I'd want fixed before it lands.

1. Non-Go languages silently answer "none" instead of failing honestly

In a repo containing both Go and TypeScript, with ast-grep unavailable, the Go fallback succeeds — which means the graph builds — but it contains no TS edges. --importers then reports a confident negative:

# ground truth (ast-grep available)
$ codemap --importers web/util.ts
📍 File: web/util.ts
   Imported by 1 file(s)
   • web/app.ts

# origin/main, ast-grep unavailable
$ codemap --importers web/util.ts
Error building file graph: ast-grep not found (checked bundled tools and PATH)

# this PR, ast-grep unavailable
$ codemap --importers web/util.ts
No files import web/util.ts.
   Note: files in the same package never import each other (Go resolves
   imports at package level), so only cross-package importers appear here.

web/app.ts does import it. An agent reading this concludes the file is unused and safe to delete. The Go-specific note printed about a TypeScript file makes it worse.

The JSON surface and MCP both stay honest (partial + coverage text) — only the CLI drops it, because renderImportersReportCLI early-returns before renderCoverage. Fix looks like ~3 lines: emit the coverage line in the empty branch too. Trading a hard error for a wrong answer is the one trade this project shouldn't make.

2. The fallback doesn't fire on timeout or failure, only on "not installed"

The PR body says the fallback covers when ast-grep "fails or times out". It doesn't. ScanDirectory converts every IncompleteScanError except ScanSourceUnavailable into a degraded-but-nil-error outcome, so the gate in cargofallback.go (which requires a non-nil *IncompleteScanError named ast-grep) is only reachable when the binary is absent. With a stub ast-grep emitting invalid JSON:

"coverage":{"status":"unavailable","sources":[{"name":"ast-grep","status":"failed",
  "detail":"ast-grep produced invalid JSON results"}]}, "files":[]

Zero fallback, with parseable Go files sitting right there. Timeout and non-zero exit are the realistic large-repo failures — those are exactly the cases the fallback would earn its keep, and they're the ones it misses. Either widen the gate or narrow the claim in the body.

Smaller notes

• --deps emits two cargo-metadata sources with contradictory statuses (authoritative and fallback) and runs cargo metadata twice. The dedup guard sits in buildFileGraphFromOutcomeWithCargoMetadataAndFilters, but --deps reaches the graph through the exported BuildFileGraphFromOutcome, which hardcodes the loader.
• Coverage notes claim recovered Cargo edges that never reach the --deps payload — they live in the unexported precomputedEdges and aren't serialized.
• "recovered 4 of 5 Go files" reads as data loss; the 5th is a type/const-only file that ast-grep also omits. Nothing is lost, the denominator is just wrong.
• The fallback emits methods as functions where ast-grep doesn't, so the files/functions set churns across authoritative→fallback→authoritative transitions — visible to anything caching analyses (watch state, handoff).

Merge-order note

This changes scanForGraphOutcomeWithFilters from 5 params to 6. I checked #124–#127 and none of them call it, so there's no repeat of the #117/#118 break — but it's the same shape, so worth knowing.

More importantly: with ast-grep unavailable the Go fallback emits no Rust analyses, so all of the Rust resolution work in #124–#127 is inert on that path, and the loader = nil guard here degrades rustWorkspaceIndex to path-globbing whenever the Cargo fallback fires. Suggest landing this after the Rust batch, and re-checking #127's build-script fixtures under the ast-grep-unavailable path.

[reneleonhardt]

Rebased

All six code-review points are addressed in the compliance commit.

Compliance

• The empty --importers output now renders the coverage line, so a partial scan reads as partial — never a confident negative; the Go same-package note prints only for .go files.
• The Go fallback now fires on degraded ast-grep outcomes (timeout, failed parse/output), not just "not installed"; an unrecoverable degraded primary still fails closed with the degraded outcome preserved.
• --deps graph builds dedupe the cargo-metadata source — one source, one cargo metadata run.
• The fallback source detail no longer claims recovered edges that never reach the --deps payload.
• The recovery note counts only dependency-bearing Go files ("recovered N of M Go files with dependency references").
• Methods are skipped in the fallback to match ast-grep's function_declaration capture — no files/functions churn across authoritative→fallback→authoritative transitions.

Where I didn't fully comply

• Merge order: kept the PR draft, as suggested.
  fix(scanner): Resolve literal Rust includes #125/feat(scanner): Resolve Askama template dependencies #126 are already merged.
  The union with feat(scanner): Track Cargo build-script inputs #127 was verified locally under the ast-grep-unavailable path — clean, no failures beyond a pre-existing main test that requires the ast-grep binary (CI always has it).
  Will re-run the build-script fixtures against the final merged main before landing.

[reneleonhardt]

After v4.4.1 what needs to be done to finish this?

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[JordanCoin]

Nothing — it's done. Verified against main with the full Rust batch (#124/#127/#141) merged: clean merge, full go test ./... green, and I exercised the fallback paths live with a built binary — --importers on a TS file with ast-grep unavailable now renders the partial-coverage line instead of a confident negative, the Go fallback resolves cross-package edges correctly, and a degraded ast-grep (invalid JSON output) also triggers the fallback as your compliance commit claims. Marked ready and merged. Thanks for the careful follow-through on all four PRs.

---

Generated by Claude Code