H-6664: Introduce Python monorepo infrastructure and formalize python repo-chores - #9039
H-6664: Introduce Python monorepo infrastructure and formalize python repo-chores#9039indietyp wants to merge 10 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
PR SummaryMedium Risk Overview Developer and CI wiring: new mise tasks
Reviewed by Cursor Bugbot for commit 479113a. Bugbot is set up for automated code reviews on this repo. Configure here. |
This stack of pull requests is managed by Graphite. Learn more about stacking. |
Merging this PR will degrade performance by 15.38%
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes |
Benchmark results
|
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2002 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1002 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 3314 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 1527 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 2078 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 1033 | Flame Graph |
policy_resolution_medium
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 102 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 269 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 108 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 133 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 63 | Flame Graph |
policy_resolution_none
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 8 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 3 | Flame Graph |
policy_resolution_small
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 26 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 94 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 27 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 66 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 29 | Flame Graph |
read_scaling_complete
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id;one_depth | 1 entities | Flame Graph | |
| entity_by_id;one_depth | 10 entities | Flame Graph | |
| entity_by_id;one_depth | 25 entities | Flame Graph | |
| entity_by_id;one_depth | 5 entities | Flame Graph | |
| entity_by_id;one_depth | 50 entities | Flame Graph | |
| entity_by_id;two_depth | 1 entities | Flame Graph | |
| entity_by_id;two_depth | 10 entities | Flame Graph | |
| entity_by_id;two_depth | 25 entities | Flame Graph | |
| entity_by_id;two_depth | 5 entities | Flame Graph | |
| entity_by_id;two_depth | 50 entities | Flame Graph | |
| entity_by_id;zero_depth | 1 entities | Flame Graph | |
| entity_by_id;zero_depth | 10 entities | Flame Graph | |
| entity_by_id;zero_depth | 25 entities | Flame Graph | |
| entity_by_id;zero_depth | 5 entities | Flame Graph | |
| entity_by_id;zero_depth | 50 entities | Flame Graph |
read_scaling_linkless
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | 1 entities | Flame Graph | |
| entity_by_id | 10 entities | Flame Graph | |
| entity_by_id | 100 entities | Flame Graph | |
| entity_by_id | 1000 entities | Flame Graph | |
| entity_by_id | 10000 entities | Flame Graph |
representative_read_entity
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1
|
Flame Graph |
representative_read_entity_type
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| get_entity_type_by_id | Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba
|
Flame Graph |
representative_read_multiple_entities
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_property | traversal_paths=0 | 0 | |
| entity_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=0 | 0 | |
| link_by_source_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true |
scenarios
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| full_test | query-limited | Flame Graph | |
| full_test | query-unlimited | Flame Graph | |
| linked_queries | query-limited | Flame Graph | |
| linked_queries | query-unlimited | Flame Graph |
8a662f1 to
ebb40a5
Compare
Dependency ReviewThe following issues were found:
Vulnerabilitiesuv.lockLicense Issuesapps/petrinaut-opt/package.json
libs/@local/repo-chores/python/pyproject.toml
uv.lock
yarn.lock
OpenSSF ScorecardScorecard details
Scanned Files
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit ebb40a5. Configure here.
| description = "Scenario-parameter optimization for Petrinaut using Optuna" | ||
| readme = "README.md" | ||
| requires-python = ">=3.10.20" | ||
| requires-python = ">=3.14,<3.15" |
There was a problem hiding this comment.
Docker build breaks after workspace move
High Severity
Raising requires-python to >=3.14,<3.15 and moving the lockfile to the repo-root uv.lock leaves apps/petrinaut-opt/docker/Dockerfile inconsistent: it still uses Python 3.13, uv 0.11.18, and COPYs the removed package-local uv.lock. Deploy builds that use this Dockerfile will fail at the dependency layer.
Reviewed by Cursor Bugbot for commit ebb40a5. Configure here.
Bring the petrinaut-opt Python service into the monorepo's turbo and Python workspace tooling, add OpenAPI document validation tests, upgrade Python/uv versions, and improve repo-chores sync and linting to support dependency bound checking against resolved versions.
|
|
||
| [tool.uv] | ||
| add-bounds = "major" # require cargo-style bounded ranges (>=1.2.3, <2.0.0) | ||
| exclude-newer = "5 days" # mirrors npm release age |
There was a problem hiding this comment.
Semgrep identified a blocking 🔴 issue in your code:
Dependency cooldown period set to 5 days instead of recommended 7 days, allowing resolution of potentially malicious newly-published packages.
More details about this
The exclude-newer setting in [tool.uv] is configured to only 5 days, which is shorter than the recommended 7-day dependency cooldown period. This means your build can resolve package versions published less than 7 days ago, including newly released versions that may be malicious, unstable, or have undiscovered vulnerabilities.
Exploit scenario: An attacker could publish a malicious version of a popular dependency on day 1, then wait 5 days. When your CI/CD pipeline runs after day 5, uv will resolve and include this malicious package version in your build, potentially injecting backdoors or data exfiltration code into your application and all downstream consumers.
The problematic line is exclude-newer = "5 days" which falls short of the 7-day safety threshold.
To resolve this comment:
✨ Commit fix suggestion
| exclude-newer = "5 days" # mirrors npm release age | |
| # Root of the Python workspace. Shared configuration for every Python package | |
| # in the repository lives here; per-package manifests only declare identity and | |
| # dependencies. Generated sections are kept correct by `mise run sync:python` | |
| # (see libs/@local/repo-chores/python) and validated in CI. | |
| [project] | |
| description = "Workspace-level Python configuration for the HASH monorepo" | |
| name = "hash" | |
| version = "0.0.0" | |
| # The single source of truth for the workspace Python version: every member | |
| # must declare exactly the same bound, enforced by `repo-chores sync --check`. | |
| requires-python = ">=3.14,<3.15" | |
| [tool.uv] | |
| add-bounds = "major" # require cargo-style bounded ranges (>=1.2.3, <2.0.0) | |
| exclude-newer = "7 days" # mirrors npm release age | |
| package = false | |
| [tool.uv.workspace] | |
| # Managed by `mise run sync:python` | |
| members = [ | |
| "apps/petrinaut-opt", | |
| "libs/@local/repo-chores/python", | |
| ] | |
| [dependency-groups] | |
| dev = [ | |
| "hash-repo-chores", # workspace tooling | |
| "pytest>=9.1.1,<10", | |
| "ruff>=0.15.21,<0.16", | |
| "tach>=0.35,<0.36", | |
| "ty>=0.0.58,<0.1", | |
| ] | |
| [tool.uv.sources] | |
| hash-repo-chores = { workspace = true } | |
| [tool.pytest.ini_options] | |
| addopts = "-q --import-mode=importlib" | |
| # Managed by `mise run sync:python` | |
| testpaths = [ | |
| "apps/petrinaut-opt/tests", | |
| "libs/@local/repo-chores/python/tests", | |
| ] | |
| # Managed by `mise run sync:python`. A member's tests import its modules by the | |
| # names its own directory exposes, the way its own pytest configuration does, so |
View step-by-step instructions
-
Update the
exclude-newervalue under[tool.uv]from5 daysto at least7 days.
For example, change it toexclude-newer = "7 days". -
Keep the setting in the
[tool.uv]section souvapplies the cooldown during dependency resolution.
This makesuvavoid package releases newer than the configured age. -
If you want a longer buffer, use a larger valid duration such as
exclude-newer = "14 days", but do not keep it below7 days.
💬 Ignore this finding
Reply with Semgrep commands to ignore this finding.
/fp <comment>for false positive/ar <comment>for acceptable risk/other <comment>for all other reasons
Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by uv-missing-dependency-cooldown.
You can view more details about this finding in the Semgrep AppSec Platform.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 54 out of 59 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
libs/@local/repo-chores/python/pyproject.toml:15
hatchlingis an unbounded build dependency, so installinghash-repo-chorescan select an arbitrary future backend release even whenuv.lockis unchanged. This also bypasses the new policy:lint_workspaceonly checks project and dev dependencies (repo_chores/lint.py:238), while AGENTS.md states that Python dependency ranges are bounded on both sides. Add a bounded hatchling requirement and extend the dependency lint (with a regression test) to cover[build-system].requiresso the invariant remains enforceable.
| COPY pyproject.toml uv.lock ./ | ||
| COPY apps/petrinaut-opt/pyproject.toml apps/petrinaut-opt/pyproject.toml |



🌟 What is the purpose of this PR?
This PR establishes a first-class Python workspace for the monorepo, built on a single uv workspace rooted at a new
pyproject.toml. All Python packages share one lockfile (uv.lock), one virtual environment, and workspace-level tooling: ruff for linting and formatting, ty for type checking, tach for import-boundary enforcement, and pytest for tests.A new Python package,
libs/@local/repo-chores/python(hash-repo-chores), acts as the Python counterpart to the existing Rustsync-turborepotooling. It provides arepo-choresCLI with two subcommands:sync— enforces workspace invariants: explicit uv workspace membership, uniformrequires-pythonbounds, shared ruff/pytest dev pins, and per-memberpackage.jsonturbo wiring (@python/<name>with managedlint:ruff,fix:ruff, andtest:unitscripts).prune— extends the existing turbo prune logic (previously a standalone script at.github/actions/prune-repository/prune.py) to also stub out pruned uv workspace members, keeping the pruned output a valid uv workspace.CI gains a
lint:pythonstep that runsuv sync --locked(lockfile stability),repo-chores sync --check(workspace drift), ruff, ty, and tach. Afix:pythonstep applies ruff autofixes and refreshes the lock. Async:pythonmise task regenerates all managed wiring after adding, removing, or renaming a Python package. The lefthook pre-commit hook gains a Python entry that runs ruff on staged.pyfiles.🔍 What does this change?
pyproject.tomlanduv.lockestablishing the uv workspace, pinned to Python 3.14, with ruff, ty, tach, and pytest configured workspace-wide.libs/@local/repo-chores/python(hash-repo-chores) with modules for workspace discovery (workspace.py), TOML in-place editing (toml_edit.py), workspace sync (sync.py), turbo pruning (prune.py), and a CLI entry point (cli.py)..github/actions/prune-repository/prune.pyintolibs/@local/repo-chores/python/repo_chores/prune.py, refactoring it to be path-aware, stdlib-only, and callable both directly and viarepo-chores prune. Addsstub_missing_uv_membersso pruned uv workspace members are stubbed back in alongside Cargo stubs.prune-repositoryGitHub Action to invoke the script from its new location and to copypyproject.tomlanduv.lockinto the pruned output.lint:python,fix:python, andsync:pythonwrapping the corresponding uv/repo-chores invocations.lint:pythonandfix:pythonyarn scripts in the rootpackage.jsondelegating to the mise tasks.lint:ruffandfix:ruffturbo pipeline entries so per-package ruff runs participate in the turbo graph.lint:ruffper package and to runyarn lint:pythonglobally..ruff_cacheto.gitignoreand.dockerignore.README.mdandAGENTS.md, including how to add packages, run checks, and interpret the sync tooling.hash-repo-chorescovering prune logic, workspace discovery, sync check/fix/idempotency, andassemble_package_json.Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
turbo.json's have been updated to reflect this🛡 What tests cover this?
libs/@local/repo-chores/python/tests/coveringtest_prune.py,test_sync.py, andtest_workspace.pywith a sharedconftest.pyfixture repository.❓ How to test this?
mise installto provision uv 0.11.28.uv syncto create the virtual environment.mise run lint:pythonand confirm all checks pass.mise run fix:pythonand confirm ruff autofixes apply cleanly.mise run sync:pythonand confirm no drift is reported on a secondmise run lint:python.uv run --frozen pytest libs/@local/repo-chores/python/testsand confirm all tests pass.