Repository navigation
fix: write graph.json as one line of compact JSON (#3413) - #4319
andrebrait wants to merge 1 commit into
Conversation
graph.json was pretty-printed (indent=2), so every PR that commits a
rebuilt graph reports tens of thousands of changed lines. GitHub's PR
line counts ignore .gitattributes (-diff, binary), so only the file's
content can fix it: one line of compact JSON is +1 -1.
Every writer of a project graph.json now emits separators=(",", ":")
with no indent: to_json, the `update` raw/no-cluster candidate text,
prune, merge-driver, merge-graphs, `extract --no-cluster` and the global
graph. Readers are unaffected.
`update` compares parsed data, not bytes, so an existing indented graph
would have been "unchanged" forever. It is now rewritten compact once,
then left alone.
Co-Authored-By: Claude <noreply@anthropic.com>
|
Thanks for the pull request, @andrebrait. A maintainer will review it soon. Want to talk it through while it is in review? Come join us on our Discord server. For longer-form discussion there is also GitHub Discussions. A couple of things that speed up review: make sure the test suite passes on Python 3.10 and 3.13, and that the change keeps extraction deterministic. |
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Graphify reviewed this change.
No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.
Not checked: tests were not run; formal verification proved 0 of 7 changed function(s) (2 sampled, not proven, 4 not verified).
Formal verification. 1 change(s) alter behavior, breaking input(s) attached. PR-changed functions: 3/7 verified (0 proven, 2 may-equivalent, 1 distinguished) · 4 not verified (1 vacuous, 3 unsupported).
Behavior changes: \_json\_text changes behavior, here is the input that shows it.
The verifier found a concrete input on which \_json\_text behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.
Guarantee: This difference was REPRODUCED, the verifier actually ran both versions on that input and saw them disagree. It is real, not an artifact.
Evidence: On input \{"data":"\{'a': \{'b': 1\}\}"\}, the old code produced '\{\\n "a": \{\\n "b": 1\\n \}\\n\}\\n' but the new code produces '\{"a":\{"b":1\}\}\\n' (outputs first differ at character 3). Paste that input straight into a regression test.
Not verified on this run: \_prune\_graph\_json\_sources (unsupported), dispatch\_command (vacuous: never exercised), write\_json\_atomic (unsupported), \_rebuild\_code (unsupported).
Graphify review — findings
Writes every project graph.json as one compact line, with no padding after ,/:, so a committed rebuild no longer shows up as tens of thousands of changed lines. write_json_atomic gains a compact flag, which to_json, the global graph, the merge driver, merge-graphs and source pruning now use. During update, _rebuild_code treats an existing multi-line graph.json as changed even when its parsed data matches, so legacy indented graphs get rewritten once while already-compact, unchanged graphs are left alone.
No blocking issues surfaced. 2 lower-confidence candidates did not survive cross-model review.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 3261 functions depend on the 262 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
_rebuild_code()— 166 callers, 57 callees - worse:
to_json()— 66 callers, 9 callees
Verification — 3261 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 2476 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
151 of 368 test file(s) selected (41%) via static blast radius.
tests/test_affected_cli.py— impacttests/test_agents_platform.py— impacttests/test_analyze.py— impacttests/test_atomic_canvas_export.py— impacttests/test_atomic_version_stamp.py— impacttests/test_atomic_writes.py— impacttests/test_benchmark.py— impacttests/test_benchmark_raw_graph.py— impacttests/test_build.py— impacttests/test_build_located_semantic_identity.py— impacttests/test_build_merge_dedup_scope.py— impacttests/test_build_merge_hyperedges_and_prune.py— impacttests/test_build_merge_shrink_guard.py— impacttests/test_cache.py— impacttests/test_cache_stale_import_target.py— impacttests/test_callflow_html.py— impacttests/test_carried_hyperedge_remap.py— impacttests/test_charmap_encoding.py— impacttests/test_chunking.py— impacttests/test_cli_export.py— impacttests/test_cloud_cta.py— impacttests/test_cluster.py— impacttests/test_codebuddy.py— impacttests/test_community_labels_skill.py— impacttests/test_compact_graph_json.py— impact, changed-testtests/test_confidence.py— impacttests/test_corrupt_graph_json.py— impacttests/test_cpp_objc_cross_file_calls.py— impacttests/test_cross_extension_reexport_self_cycle.py— impacttests/test_csharp_object_creation.py— impacttests/test_dedup.py— impacttests/test_dedup_remaps_hyperedges.py— impacttests/test_dedup_shrink_refuses_force_write.py— impacttests/test_definition_file_portability.py— impacttests/test_detect.py— impacttests/test_devin.py— impacttests/test_duplicate_annotation_edges.py— impacttests/test_elixir_import_resolution.py— impacttests/test_elixir_qualified_calls.py— impacttests/test_elixir_unqualified_call_scope.py— impacttests/test_evidence_binding.py— impacttests/test_explain_cli.py— impacttests/test_export.py— impacttests/test_export_control_characters.py— impacttests/test_export_direction.py— impacttests/test_export_idempotent_writes.py— impacttests/test_export_path_length.py— impacttests/test_external_stub_endpoints.py— impacttests/test_extract.py— impacttests/test_extract_cache_location.py— impact- … and 101 more
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Docs that may be stale (advisory)
docs/superpowers/plans/2026-05-04-incremental-updates-dedup.md§ Task 4: Incremental updates — semantic cache + manifest in __main__.py (lines 603-911): references changed symbols_rungraphify/skill-agents.md§ Step 9 - Save manifest, update cost tracker, clean up, and report (lines 566-669): references changed symbols_corpusgraphify/skill-aider.md§ Step 9 - Save manifest, update cost tracker, clean up, and report (lines 686-768): references changed symbols_corpusgraphify/skill-amp.md§ Step 9 - Save manifest, update cost tracker, clean up, and report (lines 566-669): references changed symbols_corpusgraphify/skill-claw.md§ Step 9 - Save manifest, update cost tracker, clean up, and report (lines 569-672): references changed symbols_corpusgraphify/skill-codex.md§ Step 9 - Save manifest, update cost tracker, clean up, and report (lines 566-669): references changed symbols_corpusgraphify/skill-copilot.md§ Step 9 - Save manifest, update cost tracker, clean up, and report (lines 569-672): references changed symbols_corpusgraphify/skill-devin.md§ Step 9 - Save manifest, update cost tracker, clean up, and report (lines 804-887): references changed symbols_corpusgraphify/skill-droid.md§ Step 9 - Save manifest, update cost tracker, clean up, and report (lines 566-669): references changed symbols_corpusgraphify/skill-kilo.md§ Step 9 - Save manifest, update cost tracker, clean up, and report (lines 569-672): references changed symbols_corpus
…and 10 more.
Formal verification
Behavior changes: \_json\_text changes behavior, here is the input that shows it.
The verifier found a concrete input on which \_json\_text behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.
Guarantee: This difference was REPRODUCED, the verifier actually ran both versions on that input and saw them disagree. It is real, not an artifact.
Evidence: On input \{"data":"\{'a': \{'b': 1\}\}"\}, the old code produced '\{\\n "a": \{\\n "b": 1\\n \}\\n\}\\n' but the new code produces '\{"a":\{"b":1\}\}\\n' (outputs first differ at character 3). Paste that input straight into a regression test.
Could not verify: Could not verify \_prune\_graph\_json\_sources.
The verifier did not have enough to check \_prune\_graph\_json\_sources, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: no capturable inputs from the test suite; property tier: not verifiable: the input domain has 83 values but only 11 distinct were tested — a small finite domain must be EXHAUSTED, not sampled (an untested input could invert the result)
Could not verify: Could not verify dispatch\_command.
The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: no capturable inputs from the test suite; property tier: not verifiable: all 40 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)
No difference found (not proven): No behavior difference found in to\_json (not a proof).
The verifier ran both versions of to\_json on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
No difference found (not proven): No behavior difference found in \_save\_global\_graph (not a proof).
The verifier ran both versions of \_save\_global\_graph on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify write\_json\_atomic.
The verifier did not have enough to check write\_json\_atomic, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: no capturable inputs from the test suite; property tier: parameter `path` is annotated `'str | Path'` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_rebuild\_code.
The verifier did not have enough to check \_rebuild\_code, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: no capturable inputs from the test suite; property tier: nondeterministic output (both versions disagreed with themselves on the witness) — not a behaviour change; a function whose output is not a function of its inputs (random / time / uuid / os.urandom, or state carried across calls) cannot be soundly distinguished by execution, so this tier abstains for the whole function
· 2 more finding(s) on lines outside this diff (see the check run).
…SON (Graphify-Labs#3413) Squashed from origin/fix/compact-graph-json (4c31235) onto upstream/v8.
|
Review companion on our fork (same head, same base commit): pfBlockerNG#13. |
…SON (Graphify-Labs#3413) Squashed from origin/fix/compact-graph-json (4c31235) onto upstream/v8.
|
Landed in 0.9.86 (now on PyPI) via an authorship-preserving cherry-pick, so your commit authorship is retained. Thank you, @andrebrait! |
2 participants
What does this PR do?
Closes #3413.
graphify-out/graph.jsonwas written pretty-printed (indent=2). For a mid-size repo that is roughly a million lines, so every PR that commits a rebuilt graph reports 50k–160k changed lines.GitHub does honor
linguist-generated=true: the file is collapsed in the Files changed view and left out of the language statistics. It still counts every changed line toward the PR's additions/deletions totals, though, and no other attribute changes that either. Measured on pfBlockerNG/pfBlockerNG withlinguist-generated=truein place, adding-diff, removing it, and marking the filebinaryall produced the same counts (for example +158987 −157008 forgraph.jsonalone on one PR). This is why #3575 (attribute-only, closed by its author) could not fix #3413. Only the content of the file changes the count: one line of compact JSON shows as +1 −1.This PR writes
graph.jsonas one line of compact JSON (separators=(",", ":"), no indent) by default, at every writer of a project graph. There is no flag, env var, or config option. Readers are unaffected (json.load). Each site keeps its existingensure_asciiand trailing-newline behaviour, so only whitespace changes.Writers changed
export.to_json(clustered build,update,cluster-only, etc.)watch._json_text— candidate text for theupdate --no-clusterpathcli._prune_graph_json_sources—--no-clusterexclusion prunemerge-driveroutput (the committed graph.json)merge-graphs --outextract --no-clusterglobal_graph._save_global_graph— same node-link shape as a project graph; changed for consistencypaths.write_json_atomicgains acompact=keyword used by the aboveLeft alone on purpose: manifest,
.graphify_labels.json,.graphify_analysis.json, global manifest, Obsidian/canvas exports, installer configs,reflectsidecars and the skill files. They are not a project graph.json and are not what bloats PR diffs. The skill snippets that write.graphify_*.jsonintermediates are untouched.Existing indented graphs
updatedecides "nothing changed" by comparing parsed data (_canonical_graph_for_compare,_canonical_topology_for_compare), not bytes. An already-committed indentedgraph.jsonwould therefore compare equal to the freshly built candidate and be left indented forever._rebuild_codenow checks whether the existing file is a single line; if not, the unchanged-graph / unchanged-topology skips are bypassed and the graph is rewritten compact once. A compact, unchanged graph is still left untouched.Type of change
Verification & Invariants
Invariant protected: a project graph.json is one line, round-trips to the same data, and an unchanged compact graph is not rewritten.
tests/test_compact_graph_json.py(new, 12 cases) drives real builds and CLI subprocesses:to_json, clusteredupdate,update --no-cluster,extract --no-cluster,merge-graphs, prune, global graph: output is exactly one line with compact separators and loads to the expected nodes/edges.merge-driveron two compact inputs: one line, union of nodes and links.update(clustered and no-cluster), with identical nodes/links.Red/green, same unchanged test file (
git hash-object6673ef56cb441291b6355343e446ac9d3168ed61 before and after):upstream/v8(b4e6307):10 failed, 2 passed(the 2 passes are the "compact graph left untouched" guards). Failures are e.g.graph.json spans 82 lines,cur.json spans 41 lines.updatelegacy handling not yet added: the twotest_update_rewrites_existing_indented_graph_compactcases fail withgraph.json spans 82 lines/67 lines, proving that the data-equality skip alone leaves an indented graph in place.12 passed.How was this tested?
The one failure,
tests/test_extract.py::test_collect_files_skips_hidden, fails identically on unmodifiedupstream/v8in the same checkout (the worktree lives under a hidden directory), so it is unrelated.Graphify-specific checklist
Co-Authored-By: Claude).Limitations: the first
updateafter upgrading rewrites an existing indented graph.json once (one large diff in that commit), after which diffs are one line. The file is no longer human-readable without a formatter (python -m json.tool,jq .).