docs: give first-time users a README where they actually land - #107
Merged
Conversation
…de claims A user who installed from PyPI and ran `toolgraph example init` landed in a directory of three unexplained files. `docs/` is excluded from the sdist, so the beginner guide was not on their disk and the only guidance was the CLI's one-line "Next: cd ... && toolgraph init". The generated directory now carries a README listing the commands in order, the expected outcome, and a link to the full guide. The guide also made three claims that do not survive being run: - "Every authored edge includes a local evidence pointer" was false for the packaged fixture. Running the quickstart end to end, `unbacked-edges` listed all three authored edges. The fixture now carries provenance on every edge, so the audit comes back empty, and both the fixture and the guide say what that does and does not prove: the demo tools return strings and touch no file, so the READS/WRITES effects are synthetic and their evidence cites the fixture that defines them. An empty audit means every edge has a pointer, never that a claim is true. The guide's falsifiable version is now exact - deleting a `data_access` provenance block and re-ingesting names that edge, while grants are audited separately under `--include-grants`. - The shared-backend section ran `docker compose` and copied `.env.example`, neither of which ships with the package or the generated quickstart. It is now explicitly a repository-based, optional path with a clone step. - The gateway step read as mandatory, contradicting the README's statement that memtomem-stm is not a required dependency. It is now labelled optional, and `policy compile` is described as printing metadata about the bundle rather than the bundle. Also adds `examples/README.md` mapping each servers/governance input to the script that consumes it and what it needs, a `Documentation` URL so the guide is reachable from the PyPI sidebar, and a CONTRIBUTING pointer to the guide. No runtime code changes. The packaged fixture does change what users see: `read_note` drops from risk 0.4 to 0.0 because the missing-evidence penalty no longer applies, which changes the bundle digest. The DENY policy is untouched and `publish_note` is still rejected. Verified from a disposable venv with the built wheel installed with [ladybug], running all six commands: both audits empty, read_note eligible, publish_note DENY_VIOLATION with its policy path, bundle compiled. The README ships in both wheel and sdist. 585 tests pass, ruff clean. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013xkxVWdqNAXjy5Au52MzU6
Follow-up to 1a55eb7, from a third review pass over the whole landing. The guides tell a first-time user to run five audits right after explaining that `unbacked-edges` comes back empty. Two of them do not: `unsafe-tools` reports `publish_note` reaching `draft-publish-deny` classified as a violation, and `blast-radius` reports the path that policy covers. Naming that reach is what those commands are for, so on this fixture a non-empty result is the correct one. Read in sequence, though, it looks like the quickstart broke. All three files now say which two return a row and why. The generated README also still described `publish_note` as writing to a resource. The fixture and the full guides were corrected in 1a55eb7 to say the demo tools return strings and touch no file, but that README was written before and kept the flat claim. It now says the manifest models the write, and that the data-flow claims exist to make the policy path reachable. Documentation only. Verified by running every audit the guides mention against a wheel-installed quickstart: unbacked-edges (with and without --include-grants), unmapped-tools, orphan-policies, drift, destructive-unsafeguarded and annotation-contradictions all return `[]`, while unsafe-tools and blast-radius each return the publish_note path. The corrected README is present in a rebuilt wheel. ruff clean, release hardening tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013xkxVWdqNAXjy5Au52MzU6
Adding `examples/README.md` in this branch broke `scripts/verify-artifacts.sh`: the sdist carried a path outside the allowlist. Hatchling matches an unanchored include pattern at any depth, so the bare `README.md` entry pulled in `examples/README.md` as well as the root one. The verifier's own allowlist is a root-anchored regex, so the two disagreed and the archive stopped being the faithful build input the allowlist is there to guarantee. The root-file entries are now anchored with a leading `/`, with a comment saying why, so a future README somewhere in the tree cannot slip in. `toolgraph/quickstart/README.md` is unaffected: it lives inside the package directory and ships via the `toolgraph/` entry, in both the wheel and the sdist. CI already covers this. `ci.yml` runs the same verifier on every pull request, not only on a tag, so the regression would have gone red there; the local release gate just caught it first. Verified: sdist manifest is 51 entries, all allowlisted; the wheel rebuilds from the sdist; twine check passes on both; `uv lock --check`, ruff, both dependency audits and 585 tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013xkxVWdqNAXjy5Au52MzU6
Follow-up to 537319f, from a review pass over the open pull request. That commit anchored the five root-file patterns and left the four directory patterns alone, which leaves the same trap in place pointing both ways. An unanchored `toolgraph/` would sweep in any nested directory of that name the tree grows later, and an unanchored `docs/` would exclude a future `toolgraph/docs/` that is meant to ship inside the package. Both are latent rather than live: `contracts/fixtures/` already behaves as root-anchored because the pattern contains an internal slash, and the only nested `toolgraph/` and `contracts/` directories today are under `.worktrees/`, which `.gitignore` keeps out of the build entirely. Every pattern is now anchored, and the comment says why in both directions, so this allowlist and the root-anchored one in `scripts/verify-artifacts.sh` cannot drift apart again. The sdist manifest is byte-for-byte the same set as before this commit: 51 entries, all allowlisted. Wheel still carries the four quickstart files, the wheel rebuilds from the sdist, twine check passes on both, and 585 tests pass. Also ran the two release-gate steps missing from the pull request body, `uv sync --frozen --group dev --extra ladybug` and the dev dependency audit; both clean. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013xkxVWdqNAXjy5Au52MzU6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A first-time user who installs from PyPI and runs
toolgraph example initlands in a directory of three unexplained files.
docs/is excluded from thesdist, so the beginner guide is not on their disk, and the only guidance is the
CLI's one-line "Next: cd ... && toolgraph init". The generated directory now
carries its own README.
Checking the beginner guide against what the commands actually print turned up
claims that do not survive being run, so this fixes those too:
packaged fixture had no provenance at all, so
unbacked-edgeslisted allthree authored edges right after the guide said it would not. The fixture now
carries provenance on every edge. The fixture and both guides also say what
an empty audit does and does not prove: the demo tools return strings and
touch no file, so the READS/WRITES effects are synthetic and their evidence
cites the fixture that defines them. An empty result means every edge has a
pointer, never that a claim is true.
data_accessprovenanceblock and re-ingesting names that edge. Deleting a grant's provenance does
not, because grants are audited separately under
--include-grants.unsafe-toolsreports
publish_notereachingdraft-publish-denyas a violation, andblast-radiusreports the path it covers. Read right after the empty-auditdiscussion, that looks like the quickstart broke. All three documents now say
which two return a row and why.
docker composeand copied.env.example,neither of which ships with the package or the generated quickstart. It is
now an explicitly repository-based, optional path with a clone step.
that memtomem-stm is not a required dependency. It is now optional, and
policy compileis described as printing metadata about the bundle ratherthan the bundle.
Also adds
examples/README.mdmapping each servers/governance input to thescript that consumes it and what it needs, a
DocumentationURL so the guideis reachable from the PyPI sidebar, and a CONTRIBUTING pointer to the guide.
The last two commits are packaging, not documentation: adding
examples/README.mdbrokescripts/verify-artifacts.sh, because Hatchlingmatches an unanchored include pattern at any depth and the bare
README.mdentry pulled in the examples one. Every pattern in the sdist allowlist is now
anchored to the project root, including the directory entries, which carried
the same reach in both directions.
Validation
Local release gate from
docs/releasing.md, every step passing:Beyond the gate, the user-facing path was exercised end to end rather than
read. The wheel was installed with
[ladybug]into a disposable venv and everycommand in the new README was run with that venv's executable:
unbacked-edgesand--include-grantsboth return[]; so dounmapped-tools,orphan-policies,drift,destructive-unsafeguardedandannotation-contradictions.unsafe-tools vibe-coderandblast-radius draft-publish-denyeach returnthe
publish_notepath, which is what the guides now say to expect.eligible-toolsreturnsread_noteeligible andpublish_noterejected withDENY_VIOLATIONand its policy path;policy compilewrites the bundle andprints its digest.
unbacked-edgesnamethat one edge and no other, which is the guide's claim. The command prints
JSON objects; projecting each to
(edge_type, src)gives exactly[('WRITES', 'policy-gateway::publish_note')].toolgraph/quickstart/README.mdis present in both the wheel and the sdist;examples/README.mdis in neither, after the packaging fix.Not exercised locally, delegated to CI: the
artifact-smokematrix across theOS and Python versions this machine cannot reproduce, and the minimum-MCP
install job, which is a single Ubuntu job and could be run here.
ci.ymlalsoruns the same artifact verifier on every pull request, so the allowlist this
branch touches is enforced remotely rather than only at release time.
Notes for the reviewer
One user-visible behavior change. Filling in the fixture's provenance drops
read_notefrom risk0.4to0.0, because_risk_scoreis first-match andthe unbacked-edge case no longer applies. Risk is serialized into the bundle,
so the quickstart's bundle digest changes. The DENY policy is untouched and
publish_noteis still rejected at1.0.A decision that could have gone the other way. The data-flow evidence
pointers cite
policy_gateway_server.py, not the manifest, even though thetools do not really perform that I/O. Pointing them at themselves would have
satisfied the audit while explaining nothing, so they name the fixture that
defines the modelled effect and say in the string that it is synthetic.
Deliberately out of scope: an example notebook. There is no documented
stable Python onboarding API (
toolgraph/__init__.pyexports only__version__, and no module defines__all__), so a notebook would wrap thesame CLI commands the guide already shows and would rot the way
0.0.1quickstart copies did. Worth revisiting only if a public Python API is curated.
No runtime code changed. The only non-documentation edits are the packaged
fixture,
pyproject.toml, and theexample inittest, which pinned the exactthree-file listing and now checks four files plus non-blank evidence on each
authored binding.
No separate regression test for the packaging fix.
ci.ymlrunsscripts/verify-artifacts.shon every pull request andci-requireddependson that job, and
examples/README.mdnow reproduces the regression on its ownif the anchoring is reverted. A unit test asserting the patterns start with a
slash would restate the implementation rather than the invariant.
🤖 Generated with Claude Code
https://claude.ai/code/session_013xkxVWdqNAXjy5Au52MzU6