Skip to content

docs: give first-time users a README where they actually land - #107

Merged
memtomem merged 4 commits into
mainfrom
docs/first-run-onboarding
Sep 12, 2026
Merged

docs: give first-time users a README where they actually land#107
memtomem merged 4 commits into
mainfrom
docs/first-run-onboarding

Conversation

@memtomem

@memtomem memtomem commented Sep 12, 2026

Copy link
Copy Markdown
Owner

Summary

A first-time user who installs from PyPI and runs toolgraph example init
lands in a directory of three unexplained files. docs/ is excluded from the
sdist, so the beginner guide is not on their disk, and the only guidance is the
CLI's one-line "Next: cd ... && toolgraph init". The generated directory now
carries its own README.

Checking the beginner guide against what the commands actually print turned up
claims that do not survive being run, so this fixes those too:

  • "Every authored edge includes a local evidence pointer" was false. The
    packaged fixture had no provenance at all, so unbacked-edges listed all
    three authored edges right after the guide said it would not. The fixture now
    carries provenance on every edge. The fixture and both guides also say what
    an empty audit does and does not prove: the demo tools return strings and
    touch no file, so the READS/WRITES effects are synthetic and their evidence
    cites the fixture that defines them. An empty result means every edge has a
    pointer, never that a claim is true.
  • The falsifiable version is now exact. Deleting a data_access provenance
    block and re-ingesting names that edge. Deleting a grant's provenance does
    not, because grants are audited separately under --include-grants.
  • Two of the five suggested audits return a row on purpose. unsafe-tools
    reports publish_note reaching draft-publish-deny as a violation, and
    blast-radius reports the path it covers. Read right after the empty-audit
    discussion, that looks like the quickstart broke. All three documents now say
    which two return a row and why.
  • The shared-backend section ran docker compose and copied .env.example,
    neither of which ships with the package or the generated quickstart. It is
    now an explicitly repository-based, optional path with a clone step.
  • The gateway step read as mandatory, contradicting the README's statement
    that memtomem-stm is not a required dependency. It is now optional, and
    policy compile is described as printing metadata about the bundle rather
    than the bundle.

Also adds examples/README.md mapping each servers/governance input to the
script that consumes it and what it needs, a Documentation URL so the guide
is reachable from the PyPI sidebar, and a CONTRIBUTING pointer to the guide.

The last two commits are packaging, not documentation: adding
examples/README.md broke scripts/verify-artifacts.sh, because Hatchling
matches an unanchored include pattern at any depth and the bare README.md
entry pulled in the examples one. Every pattern in the sdist allowlist is now
anchored to the project root, including the directory entries, which carried
the same reach in both directions.

Validation

Local release gate from docs/releasing.md, every step passing:

uv lock --check                      Resolved 104 packages
uv sync --frozen --group dev --extra ladybug     resolved
uv run ruff check .                  All checks passed
scripts/audit-dependencies.sh runtime / extras / dev
                                     No known vulnerabilities
uv build && uv run twine check dist/*            both PASSED
scripts/verify-artifacts.sh dist 0.1.0           51 entries, all allowlisted
uv run pytest -q                     585 passed

Beyond the gate, the user-facing path was exercised end to end rather than
read. The wheel was installed with [ladybug] into a disposable venv and every
command in the new README was run with that venv's executable:

  • unbacked-edges and --include-grants both return []; so do
    unmapped-tools, orphan-policies, drift, destructive-unsafeguarded and
    annotation-contradictions.
  • unsafe-tools vibe-coder and blast-radius draft-publish-deny each return
    the publish_note path, which is what the guides now say to expect.
  • eligible-tools returns read_note eligible and publish_note rejected with
    DENY_VIOLATION and its policy path; policy compile writes the bundle and
    prints its digest.
  • Deleting the WRITES provenance and re-ingesting makes unbacked-edges name
    that one edge and no other, which is the guide's claim. The command prints
    JSON objects; projecting each to (edge_type, src) gives exactly
    [('WRITES', 'policy-gateway::publish_note')].
  • toolgraph/quickstart/README.md is present in both the wheel and the sdist;
    examples/README.md is in neither, after the packaging fix.

Not exercised locally, delegated to CI: the artifact-smoke matrix across the
OS and Python versions this machine cannot reproduce, and the minimum-MCP
install job, which is a single Ubuntu job and could be run here. ci.yml also
runs the same artifact verifier on every pull request, so the allowlist this
branch touches is enforced remotely rather than only at release time.

Notes for the reviewer

One user-visible behavior change. Filling in the fixture's provenance drops
read_note from risk 0.4 to 0.0, because _risk_score is first-match and
the unbacked-edge case no longer applies. Risk is serialized into the bundle,
so the quickstart's bundle digest changes. The DENY policy is untouched and
publish_note is still rejected at 1.0.

A decision that could have gone the other way. The data-flow evidence
pointers cite policy_gateway_server.py, not the manifest, even though the
tools do not really perform that I/O. Pointing them at themselves would have
satisfied the audit while explaining nothing, so they name the fixture that
defines the modelled effect and say in the string that it is synthetic.

Deliberately out of scope: an example notebook. There is no documented
stable Python onboarding API (toolgraph/__init__.py exports only
__version__, and no module defines __all__), so a notebook would wrap the
same CLI commands the guide already shows and would rot the way 0.0.1
quickstart copies did. Worth revisiting only if a public Python API is curated.

No runtime code changed. The only non-documentation edits are the packaged
fixture, pyproject.toml, and the example init test, which pinned the exact
three-file listing and now checks four files plus non-blank evidence on each
authored binding.

No separate regression test for the packaging fix. ci.yml runs
scripts/verify-artifacts.sh on every pull request and ci-required depends
on that job, and examples/README.md now reproduces the regression on its own
if the anchoring is reverted. A unit test asserting the patterns start with a
slash would restate the implementation rather than the invariant.

🤖 Generated with Claude Code

https://claude.ai/code/session_013xkxVWdqNAXjy5Au52MzU6

memtomem and others added 4 commits September 12, 2026 13:39
…de claims

A user who installed from PyPI and ran `toolgraph example init` landed in a
directory of three unexplained files. `docs/` is excluded from the sdist, so
the beginner guide was not on their disk and the only guidance was the CLI's
one-line "Next: cd ... && toolgraph init". The generated directory now carries
a README listing the commands in order, the expected outcome, and a link to
the full guide.

The guide also made three claims that do not survive being run:

- "Every authored edge includes a local evidence pointer" was false for the
  packaged fixture. Running the quickstart end to end, `unbacked-edges` listed
  all three authored edges. The fixture now carries provenance on every edge,
  so the audit comes back empty, and both the fixture and the guide say what
  that does and does not prove: the demo tools return strings and touch no
  file, so the READS/WRITES effects are synthetic and their evidence cites the
  fixture that defines them. An empty audit means every edge has a pointer,
  never that a claim is true. The guide's falsifiable version is now exact -
  deleting a `data_access` provenance block and re-ingesting names that edge,
  while grants are audited separately under `--include-grants`.
- The shared-backend section ran `docker compose` and copied `.env.example`,
  neither of which ships with the package or the generated quickstart. It is
  now explicitly a repository-based, optional path with a clone step.
- The gateway step read as mandatory, contradicting the README's statement
  that memtomem-stm is not a required dependency. It is now labelled optional,
  and `policy compile` is described as printing metadata about the bundle
  rather than the bundle.

Also adds `examples/README.md` mapping each servers/governance input to the
script that consumes it and what it needs, a `Documentation` URL so the guide
is reachable from the PyPI sidebar, and a CONTRIBUTING pointer to the guide.

No runtime code changes. The packaged fixture does change what users see:
`read_note` drops from risk 0.4 to 0.0 because the missing-evidence penalty no
longer applies, which changes the bundle digest. The DENY policy is untouched
and `publish_note` is still rejected.

Verified from a disposable venv with the built wheel installed with [ladybug],
running all six commands: both audits empty, read_note eligible, publish_note
DENY_VIOLATION with its policy path, bundle compiled. The README ships in both
wheel and sdist. 585 tests pass, ruff clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013xkxVWdqNAXjy5Au52MzU6
Follow-up to 1a55eb7, from a third review pass over the whole landing.

The guides tell a first-time user to run five audits right after explaining
that `unbacked-edges` comes back empty. Two of them do not: `unsafe-tools`
reports `publish_note` reaching `draft-publish-deny` classified as a
violation, and `blast-radius` reports the path that policy covers. Naming that
reach is what those commands are for, so on this fixture a non-empty result is
the correct one. Read in sequence, though, it looks like the quickstart broke.
All three files now say which two return a row and why.

The generated README also still described `publish_note` as writing to a
resource. The fixture and the full guides were corrected in 1a55eb7 to say the
demo tools return strings and touch no file, but that README was written
before and kept the flat claim. It now says the manifest models the write, and
that the data-flow claims exist to make the policy path reachable.

Documentation only. Verified by running every audit the guides mention against
a wheel-installed quickstart: unbacked-edges (with and without
--include-grants), unmapped-tools, orphan-policies, drift,
destructive-unsafeguarded and annotation-contradictions all return `[]`, while
unsafe-tools and blast-radius each return the publish_note path. The corrected
README is present in a rebuilt wheel. ruff clean, release hardening tests pass.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013xkxVWdqNAXjy5Au52MzU6
Adding `examples/README.md` in this branch broke
`scripts/verify-artifacts.sh`: the sdist carried a path outside the allowlist.

Hatchling matches an unanchored include pattern at any depth, so the bare
`README.md` entry pulled in `examples/README.md` as well as the root one. The
verifier's own allowlist is a root-anchored regex, so the two disagreed and the
archive stopped being the faithful build input the allowlist is there to
guarantee. The root-file entries are now anchored with a leading `/`, with a
comment saying why, so a future README somewhere in the tree cannot slip in.

`toolgraph/quickstart/README.md` is unaffected: it lives inside the package
directory and ships via the `toolgraph/` entry, in both the wheel and the
sdist.

CI already covers this. `ci.yml` runs the same verifier on every pull request,
not only on a tag, so the regression would have gone red there; the local
release gate just caught it first.

Verified: sdist manifest is 51 entries, all allowlisted; the wheel rebuilds
from the sdist; twine check passes on both; `uv lock --check`, ruff, both
dependency audits and 585 tests pass.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013xkxVWdqNAXjy5Au52MzU6
Follow-up to 537319f, from a review pass over the open pull request.

That commit anchored the five root-file patterns and left the four directory
patterns alone, which leaves the same trap in place pointing both ways. An
unanchored `toolgraph/` would sweep in any nested directory of that name the
tree grows later, and an unanchored `docs/` would exclude a future
`toolgraph/docs/` that is meant to ship inside the package. Both are latent
rather than live: `contracts/fixtures/` already behaves as root-anchored
because the pattern contains an internal slash, and the only nested
`toolgraph/` and `contracts/` directories today are under `.worktrees/`, which
`.gitignore` keeps out of the build entirely.

Every pattern is now anchored, and the comment says why in both directions, so
this allowlist and the root-anchored one in `scripts/verify-artifacts.sh`
cannot drift apart again.

The sdist manifest is byte-for-byte the same set as before this commit: 51
entries, all allowlisted. Wheel still carries the four quickstart files, the
wheel rebuilds from the sdist, twine check passes on both, and 585 tests pass.
Also ran the two release-gate steps missing from the pull request body,
`uv sync --frozen --group dev --extra ladybug` and the dev dependency audit;
both clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013xkxVWdqNAXjy5Au52MzU6
@memtomem
memtomem merged commit 66076e8 into main Sep 12, 2026
14 checks passed
@memtomem
memtomem deleted the docs/first-run-onboarding branch September 12, 2026 05:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant