diff --git a/AGENTS.md b/AGENTS.md index 2daaa2ea59..949f4999f9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -11,10 +11,18 @@ The toolkit supports multiple AI coding assistants, allowing teams to use their ## Adding or Updating CLI Commands Before adding, updating, or reorganizing Specify CLI commands, read -[Specify CLI Command Architecture](design/cli.md). It defines command-module -naming, private command phases, nested command groups, registration ownership, -mirrored tests, and the rationale for making the CLI structure predictable for -both humans and coding agents. +[Shared Command Application Architecture](design/shared.md) and +[Specify CLI Command Architecture](design/cli.md). They define the shared +operation boundary, command-module naming, private phases, nested command +groups, registration ownership, and mirrored tests. + +## Adding or Updating MCP Commands + +Before adding or changing MCP tools for Specify commands, read +[Shared Command Application Architecture](design/shared.md) and +[Specify MCP Command Architecture](design/mcp.md). They define the shared +operation boundary, typed contracts, explicit inventory, side-effect metadata, +the local stdio protocol boundary, and mirrored tests. ## Adding or Updating Agent Integrations diff --git a/design/cli.md b/design/cli.md index b38dfec09f..1cba8f9285 100644 --- a/design/cli.md +++ b/design/cli.md @@ -1,12 +1,18 @@ # Specify CLI Command Architecture -This document defines the target structure for multi-command groups in the -Specify Python CLI. It explains where command handlers, shared infrastructure, -command-private phases, nested command groups, and their tests belong. +This document defines the CLI adapter structure for Specify commands. It +explains where Typer handlers, CLI infrastructure, CLI-private phases, nested +command groups, and their tests belong. -`src/specify_cli/extensions/` is the reference implementation. Apply this -design incrementally when adding or refactoring other command groups; do not -create extra modules merely to make a small command conform visually. +Every CLI leaf that has another delivery adapter invokes the shared application +operation defined by +[Shared Command Application Architecture](shared.md). This document owns the +CLI surface; it does not redefine semantic validation, orchestration, results, +warnings, errors, or side effects. + +The extension hierarchy supplies the reference command-group shape used in +this document. Do not create extra modules merely to make a small command +conform visually. ## Design goals @@ -22,6 +28,8 @@ The design optimizes for: the same file. - **Explicit ownership:** shared infrastructure and command-private behavior should not be mixed. +- **Adapter discipline:** CLI modules invoke shared operations rather than + becoming the application implementation. - **Stable behavior:** structural refactoring must preserve registration, output, error handling, compatibility paths, and tests. - **Agentic development:** coding agents should be able to infer the relevant @@ -50,17 +58,24 @@ Only modules representing actual CLI commands use the non-underscored - The Typer-decorated handler. - User-facing arguments and options. -- Command-specific orchestration. +- Mapping parsed values into the shared operation request. +- CLI prompting, progress, rendering, JSON streams, and exit-code mapping. - Small helpers used only by that command. +Semantic validation, application orchestration, typed outcomes, warnings, +expected errors, and side effects belong below the adapter as defined in +[the shared design](shared.md). A CLI-only command may keep small behavior in +its command module, but behavior needed by another adapter must first move to a +shared operation or domain module. + The command function's docstring is user-facing because Typer may display it as help text. A module docstring is internal and should identify the command, registration path, and any adjacent private implementation modules. ### Command-private implementation modules -When a command has cohesive phases that are independently understandable or -testable, use: +When a CLI adapter has cohesive *CLI-specific* phases that are independently +understandable or testable, use: ```text _command__.py @@ -69,14 +84,13 @@ _command__.py For example: ```text -command_update.py -_command_update_discovery.py -_command_update_artifacts.py -_command_update_transaction.py +command_init.py +_command_init_prompting.py +_command_init_rendering.py ``` The leading underscore marks the module as private implementation. The -`command_update` portion groups it with the registered handler in searches and +`command_init` portion groups it with the registered handler in searches and file listings. The phase suffix communicates its ownership. Private phase modules must not register additional CLI commands. The public @@ -84,11 +98,16 @@ Private phase modules must not register additional CLI commands. The public Split a command when a phase: +- Is specific to CLI invocation or presentation. - Has distinct invariants or failure behavior. - Can be tested as a meaningful boundary. - Has enough implementation detail to distract from the CLI handler. - Is likely to change independently from other phases. +If the phase performs semantic validation, planning, mutation, rollback, or +other behavior another adapter needs, use `_operation__.py` +instead, following [the shared design](shared.md#naming-and-layout). + Do not split a command solely because it crossed an arbitrary line count. Excessive fragmentation makes control flow harder to follow and increases the number of files an agent must inspect. @@ -99,7 +118,8 @@ For a multi-command group, `_commands.py` owns: - The command group's Typer application. - Registration of the group's command modules. -- Infrastructure genuinely shared by multiple commands or external CLI flows. +- CLI infrastructure genuinely shared by multiple commands or external CLI + flows. - Thin compatibility forwarders needed to preserve established import or monkeypatch paths. @@ -178,8 +198,9 @@ re-exports the command symbols for compatibility. Do not create a nested directory for an implementation phase that is not a CLI subcommand. For example, an `update/` directory would incorrectly suggest an -`extension update ...` subcommand group. Use `_command_update_.py` -instead. +`extension update ...` subcommand group. Use `_command_update_.py` for +a CLI-private phase and `_operation_update_.py` for semantic validation, +planning, mutation, rollback, or other behavior shared with another adapter. ## Registration @@ -200,6 +221,27 @@ Registration imports should be explicit and ordered consistently. Do not rely on filesystem discovery to import arbitrary modules, because command exposure should remain reviewable in one place. +## Shared operation invocation + +The CLI adapter maps parsed arguments and options into the shared typed request, +invokes the operation, and renders its outcome. + +For project-scoped commands, the adapter supplies the explicit project or +target directory. When the user omits it, the CLI uses the cwd captured when +the invocation begins. The shared operation validates and resolves that path; +neither layer changes process-wide cwd. + +The CLI runs with the invoking user's ordinary operating-system permissions. +It does not construct a shared policy or filesystem runtime. Prompting remains +CLI-specific: + +- `--json`, `--non-interactive`, and local invocation do not imply `force`. +- External-source trust and overwrite consent remain explicit request values. +- A prompt may collect consent before constructing or retrying the request, but + the adapter does not add consent silently. +- Network and self-modifying behavior remain those defined by the operation and + CLI contract. + ## Test structure Command-focused tests mirror the source command surface under @@ -215,23 +257,25 @@ src/specify_cli/extensions/catalog/command_add.py tests/specify_cli/extensions/catalog/test_command_add.py ``` -Private phases use: +CLI-private phases use: ```text -src/specify_cli/extensions/_command_update_discovery.py -tests/specify_cli/extensions/test_command_update_discovery.py - -src/specify_cli/extensions/_command_update_artifacts.py -tests/specify_cli/extensions/test_command_update_artifacts.py - -src/specify_cli/extensions/_command_update_transaction.py -tests/specify_cli/extensions/test_command_update_transaction.py +src/specify_cli/_command_init_prompting.py +tests/specify_cli/test_command_init_prompting.py ``` The primary `test_command_.py` suite verifies the public command surface. Phase-specific suites verify detailed invariants without obscuring the primary command behavior. +CLI adapter tests verify explicit targets outside the launch directory and the +rule that machine modes do not add force, trust, or destructive consent. + +Shared operation and phase tests use `test_operation_.py` and +`test_operation__.py` as defined in +[the shared testing structure](shared.md#testing-structure). They do not move +under `test_command_*.py` merely because the CLI is one caller. + Domain source remains in the parent package's `__init__.py` or a focused domain module without the `command_` prefix. Its mirrored tests use the domain subject name, for example: @@ -290,7 +334,8 @@ must remain represented. A matching total alone does not prove preservation. ## Reference layout -The extension command group currently demonstrates the complete pattern: +An extension command group using the shared application boundary has this +shape: ```text src/specify_cli/extensions/ @@ -305,9 +350,10 @@ src/specify_cli/extensions/ ├── command_search.py ├── command_set_priority.py ├── command_update.py -├── _command_update_discovery.py -├── _command_update_artifacts.py -├── _command_update_transaction.py +├── _operation_update.py +├── _operation_update_discovery.py +├── _operation_update_artifacts.py +├── _operation_update_transaction.py └── catalog/ ├── __init__.py ├── _helpers.py @@ -319,10 +365,11 @@ src/specify_cli/extensions/ The update command illustrates the distinction: - `command_update.py` is the registered CLI adapter. -- `_command_update_discovery.py` determines available updates. -- `_command_update_artifacts.py` prepares and validates update archives. -- `_command_update_transaction.py` owns backup, installation, rollback, and - cleanup behavior. +- `_operation_update.py` is the shared application entry point. +- `_operation_update_discovery.py` determines available updates. +- `_operation_update_artifacts.py` prepares and validates update archives. +- `_operation_update_transaction.py` owns backup, installation, rollback, and + cleanup behavior for every adapter. ## Decision guide @@ -331,9 +378,11 @@ When deciding where code belongs: | Question | Location | |---|---| | Does it define a real CLI command? | `command_.py` | -| Is it used only by one small command? | That command module | -| Is it a cohesive private phase of one complex command? | `_command__.py` | -| Is it shared by multiple commands or an external CLI flow? | `_commands.py` or a focused shared module | +| Is it a small CLI-only mapping or rendering helper? | That command module | +| Is it a cohesive CLI-only phase? | `_command__.py` | +| Does it define semantic validation or orchestration for an operation? | Existing domain module or `_operation_.py` | +| Is it a cohesive shared operation phase? | `_operation__.py` | +| Is it CLI infrastructure shared by multiple command adapters? | `_commands.py` or a focused CLI helper | | Does it define a nested CLI namespace? | A directory matching that namespace | | Is it shared only by commands in a nested namespace? | The nested package's `_helpers.py` | | Is it domain behavior independent of the CLI? | The package domain modules, not command modules | @@ -346,6 +395,10 @@ Avoid: - Naming a private implementation module `command_*.py`. - Creating nested directories that do not correspond to CLI namespaces. - Creating `_commands.py` files only for visual symmetry. +- Keeping semantic validation, orchestration, or side effects in a CLI adapter + when another adapter exposes the same logical operation. +- Calling or parsing another delivery adapter instead of invoking the shared + operation. - Moving command-private helpers into shared infrastructure preemptively. - Duplicating fixtures or helpers to make tests appear more mirrored. - Splitting a linear function into many files without cohesive phase @@ -359,9 +412,18 @@ For a new or refactored command: - [ ] The CLI path maps predictably to a `command_.py` module. - [ ] Only the real command module registers a handler. -- [ ] Private phase modules use `_command__.py`. +- [ ] The owning hierarchy records exactly one MCP inventory disposition for + the CLI leaf, including when it is unavailable or excluded. +- [ ] If another adapter exposes the operation, the CLI adapter maps into the + shared operation defined by `design/shared.md`. +- [ ] For a multi-adapter operation, semantic validation, orchestration, and + side effects are below the CLI adapter. +- [ ] When a shared request exists, project and target directories map into it + explicitly. +- [ ] CLI-private phase modules use `_command__.py`; shared phases + use `_operation__.py`. - [ ] `_commands.py` contains only group infrastructure and genuinely shared - behavior. + CLI behavior. - [ ] Nested directories correspond to real CLI namespaces. - [ ] Command tests mirror the source structure. - [ ] Domain and cross-domain tests remain in their appropriate suites. diff --git a/design/mcp.md b/design/mcp.md new file mode 100644 index 0000000000..89818d84db --- /dev/null +++ b/design/mcp.md @@ -0,0 +1,670 @@ +# Specify MCP Command Architecture + +This document defines the architecture for exposing Specify operations through +the Model Context Protocol (MCP). It is the MCP counterpart to +[Specify CLI Command Architecture](cli.md). + +Both adapters invoke the application layer defined by +[Shared Command Application Architecture](shared.md). This document owns MCP +tool identity, exposure, annotations, protocol mapping, and local stdio +hosting. It does not redefine semantic validation, orchestration, results, +warnings, errors, or side effects. + +## Design goals + +The MCP structure should make the answer to "where does this tool's behavior +live?" as predictable as the equivalent CLI question. + +The design optimizes for: + +- **One logical operation:** a CLI leaf and its MCP tool are two adapters for + the same application behavior. +- **Adapter parity:** CLI and MCP inputs, results, warnings, errors, and side + effects remain semantically equivalent. +- **Typed discovery:** each available MCP operation has a command-specific + input and output schema. +- **Explicit exposure:** every CLI leaf has a reviewable MCP inventory + disposition; tools are never exposed through filesystem discovery. +- **Small working context:** changing one operation should normally require + only its domain, CLI adapter, MCP adapter, and mirrored tests. +- **Side-effect visibility:** project writes, execution, network access, trust + decisions, and self-modification are declared for review and conservatively + projected into standard MCP annotations. +- **Local stdio boundary:** protocol framing and diagnostics remain outside + command behavior. + +## Non-goals + +This design does not: + +- Make MCP a wrapper around the human CLI. +- Require every CLI leaf to be invokable regardless of risk or readiness. +- Turn existing human output into an MCP or JSON contract. +- Make `--json`, `--non-interactive`, or an MCP invocation imply `--force`, + trust, destructive consent, or network permission. +- Define a public Python API for third-party callers. The supported external + surfaces remain the CLI, MCP contracts, integrations, and workflow steps. +- Replace command-owned application behavior with one universal command + engine or central service locator. +- Require an otherwise simple operation to be split into extra modules only + for visual symmetry. + +## Shared operation dependency + +An MCP tool is a delivery adapter for a logical operation defined by +[the shared architecture](shared.md). It maps MCP input into the shared typed +request and maps the shared outcome into MCP structured content or a tool +error. + +The MCP adapter owns: + +- Tool name, description, annotations, and protocol schemas. +- Mapping between MCP content and shared request/outcome types. +- Per-group MCP registration and static inventory. +- Protocol diagnostics and local stdio server hosting. + +It does not own semantic validation, application orchestration, side effects, +or command-specific domain contracts. It must not invoke Typer handlers, start +`specify` as application dispatch, scrape Rich output, or parse CLI stderr. + +Shared MCP infrastructure may define protocol and registration primitives. It +must not become a central catalog of command-specific request models, results, +validation, or orchestration. + +## Operation identity and MCP tool design + +Each logical operation has three related identities: + +| Surface | Example | +| --- | --- | +| Logical operation ID | `artifact.list` | +| CLI path | `specify artifact list` | +| MCP tool name | `specify_artifact_list` | + +Operation IDs use dot-separated CLI path segments without the leading +`specify`. MCP tool names use the same segments with underscores and a +`specify_` prefix. Hyphens in CLI segments become underscores: + +```text +specify extension set-priority +operation: extension.set-priority +tool: specify_extension_set_priority +``` + +An explicit override is allowed only to satisfy a protocol restriction or +resolve a demonstrated collision. The inventory must record the override and +the reason. + +### First-class tools, not a generic execution facade + +The MCP surface exposes each *eligible* CLI leaf as a first-class MCP tool. +A generic `specify_run_command` facade is not part of the architecture. + +The MCP SDK creates one JSON input schema per registered tool from the tool's +typed callable. First-class tools therefore preserve: + +- Per-command schemas and descriptions. +- MCP client discovery and argument validation. +- Command-specific output schemas and annotations. +- Reviewable registration and side-effect metadata. +- A direct mapping back to the CLI leaf and owning source files. + +A generic facade would instead reduce the protocol-visible input to a command +string plus an opaque or oversized union of arguments. That weakens schema +validation, discoverability, side-effect review, and compatibility analysis. + +Registration is organized per command hierarchy so command-specific contracts +remain reviewable without being hidden behind a generic tool. + +### Command inventory + +The CLI leaf inventory at the time this design was established is: + +```text +root: init, check, version, mcp +self: check, upgrade +extension: add, disable, enable, info, list, remove, search, set-priority, + update +extension.catalog: add, list, remove +integration: install, uninstall, switch, upgrade, list, status, use, search, + info, scaffold +integration.catalog: add, list, remove +event: run +preset: list, add, remove, update, search, resolve, info, set-priority, enable, + disable +preset.catalog: add, list, remove +artifact: list, info, lookup +bundle: search, info, list, install, add, update, remove, validate, build, init +bundle.catalog: add, list, remove +workflow: run, resume, status, list, add, remove, update, enable, disable, + search, info, resolve +workflow.catalog: add, list, remove +workflow.step: list, add, remove, search, info +workflow.step.catalog: add, list, remove +workflow.overlay: add, set-priority, enable, disable, remove, list +``` + +`mcp` is excluded because it starts the local stdio server. Every other +available leaf maps to a first-class tool using the naming rule above. A leaf +that is unavailable or excluded must still have an explicit hierarchy-owned +inventory record and reason. + +This list documents the command namespaces; it is not a registration source. +The hierarchy-owned inventory and its parity tests are authoritative. + +Metadata-only inventory or describe tools may exist for diagnostics. They do +not replace first-class operation tools. A generic execution tool is not part +of this architecture. + +## Naming and file layout + +MCP adapters mirror the CLI command path and live beside the owning command +and domain code: + +```text +src/specify_cli/extensions/ +├── __init__.py +├── _commands.py +├── _mcp.py +├── command_add.py +├── mcp_add.py +├── command_list.py +├── mcp_list.py +└── catalog/ + ├── __init__.py + ├── _mcp.py + ├── command_add.py + ├── mcp_add.py + ├── command_list.py + └── mcp_list.py +``` + +The conventions are: + +- `command_.py` is the sole CLI adapter for a real leaf command. +- `mcp_.py` is the sole MCP adapter for the same logical operation. +- `_mcp.py` owns explicit MCP registration and inventory for a command group. +- Nested directories continue to correspond to real CLI namespaces or bounded + subdomains, following [the CLI design](cli.md#nested-command-groups). +- Shared operation and phase modules follow + [the shared naming rules](shared.md#naming-and-layout). +- Tests mirror these names under `tests/specify_cli/`. + +Do not create a top-level MCP mirror of the entire CLI tree under +`mcp_server/commands/`. That would separate command contracts from their +owning domains and make unrelated command hierarchies depend on a central +package. + +Do not add `_mcp.py` merely for symmetry. A package with one small tool may +register it through an existing focused registration module. Create `_mcp.py` +when the hierarchy needs an explicit list of several tools, shared adapter +helpers, or inventory dispositions. + +## Registration and explicit inventory + +Registration remains explicit at each command-group boundary. + +For a group such as `artifact`: + +1. `artifacts/_mcp.py` lists every `artifact.*` CLI leaf. +2. Each leaf has exactly one inventory record. +3. Available records import and register their `mcp_.py` adapter. +4. Unavailable and excluded records state a reason. +5. The root MCP composition module calls `artifacts._mcp.register(...)`. + +The root MCP server may aggregate hierarchy registration functions, but it +must not own their command-specific schemas or dispatch behavior. Registration +imports are explicit and consistently ordered. Filesystem scanning, +`command_*.py` introspection, and decorator side effects are not substitutes +for an inventory. + +A conceptual inventory record contains: + +```text +operation_id +cli_path +mcp_tool_name +contract_version +disposition +disposition_reason +capabilities +network_access +``` + +The static disposition values are: + +- `available`: implemented and registered as a first-class tool. +- `unavailable`: the logical operation is known but cannot be offered in the + current distribution or platform; the record states the concrete reason. +- `excluded`: the command is intentionally not an MCP operation. + +Every CLI leaf must appear exactly once. The inventory parity test fails for a +missing leaf, duplicate logical operation, duplicate tool name, stale CLI +path, or unexplained exclusion. + +### Permanent and conditional exclusions + +`specify mcp` is permanently excluded because it starts the local stdio server; +an MCP tool that starts another MCP server would be recursive infrastructure, +not an application operation. + +Other commands are not excluded merely because they mutate state. Their +side effects are declared so MCP hosts and clients can make informed exposure +and confirmation decisions. For example: + +- `self.upgrade` has static disposition `available` when its first-class tool + is implemented and declares local reads, execution, and self-modification. +- `event.run`, `workflow.run`, and `workflow.resume` are execution operations + that may also persist project state. +- `init`, add/remove/update commands, and configuration changes are + project-write operations, with execution and other independent + capabilities declared when their paths require them. +- `check` launches installed host tools to inspect their versions, so it + requires execution capability even though it does not persist changes. +- A command that still depends on prompts, writes directly through its Typer + handler, or lacks a typed result must not be marked `available`. + +Unavailability and exclusion are reviewable architecture decisions, not silent +omissions. + +## MCP contract projection + +Shared request, outcome, warning, error, validation, and contract-version rules +are defined by +[Shared Command Application Architecture](shared.md#typed-request-contract). +The MCP adapter projects that contract onto MCP: + +- Its input schema is command-specific and maps into the shared typed request. +- It performs protocol/schema validation but no state-dependent semantic work. +- It projects the operation's side-effect and network metadata into standard + MCP annotations while the hierarchy-owned inventory retains the exact + declarations. +- It maps the shared result and warnings into command-specific structured + content. +- It maps expected shared errors into MCP tool errors without adding + success-shaped fallbacks. + +MCP protocol envelopes do not force a universal application result envelope. +The command/domain hierarchy continues to own the semantic result shape. +The hierarchy-owned inventory records `contract_version` for compatibility and +parity tests, but the local MCP protocol does not add a custom version field to +tools or results. +Unexpected exceptions become sanitized `internal_error` tool failures and are +logged only through the MCP diagnostic channel. + +## Project directory and execution environment + +Project-scoped MCP tools accept an optional project directory when their use +case needs one. If omitted, project discovery starts from the server launch +working directory, matching normal CLI behavior. The adapter maps that explicit +path into the shared request. The shared operation resolves it through the same +domain helper used by CLI and produces the same semantic errors. + +The adapter and shared operation follow these rules: + +- Do not call `os.chdir()` for an MCP request. A long-lived server may process + concurrent or sequential calls with different project contexts. +- Pass the resolved project or target path explicitly through operation phases. +- Use the same shared Python/domain helpers as CLI for distribution metadata, + bundled assets, project files, and other application behavior. +- Do not infer the project from an unrelated server process state after the + invocation begins. + +`init` is a special project-creation operation: its request identifies the +target directory rather than an existing project root. + +Local stdio runs with the operating-system permissions of the server process, +just as the CLI runs with its process user's permissions. The command +architecture does not claim to provide a per-operation filesystem sandbox. A +host that needs confinement runs the MCP server inside an OS sandbox, container, +or restricted account. + +## Non-interactive behavior + +MCP operations are always non-interactive: + +- They never read stdin. +- They never open arrow-key selectors or terminal confirmations. +- They never wait for an answer that is not represented in the request. +- A safe documented default may be applied only when the CLI operation uses + the same non-interactive default. +- If no safe default exists, return a structured `input_required` or + `confirmation_required` error explaining which field must be supplied. + +Machine mode is not consent. Starting MCP, using `--json` or +`--non-interactive`, or receiving a host confirmation does not imply: + +- `force=true`. +- Trust of an external URL or downloaded executable content. +- Permission to overwrite user-modified files. +- Permission to execute behavior that the caller did not explicitly request. + +The caller must explicitly invoke an execution operation, supply any +command-specific consent fields, and satisfy the shared operation's semantic +validation. MCP annotations and host UI are advisory; they do not substitute +for required request values. + +## Capability and network metadata + +Operations declare capabilities according to +[the shared capability contract](shared.md#capability-declarations). These are +cumulative descriptions, not a highest-risk hierarchy. Examples: + +```text +version -> {local-read} +check -> {local-read, execution} +workflow.run -> {local-read, project-write, execution} +self.upgrade -> {local-read, execution, self-modifying} +``` + +`read-only` is a derived description, not a declared capability. A +request is read-only only when it requires no `project-write`, `execution`, or +`self-modifying` capability. An operation that launches a binary is therefore +not read-only even if it does not persist changes. + +Network access is an independent declaration: `none`, `optional`, or +`required`. A read-only search may use the network, while a project-write +operation may be fully offline. + +The hierarchy-owned inventory retains the exact capability set and network +state for review and parity tests. Standard MCP `ToolAnnotations` are hints, +not a lossless capability contract: + +- `readOnlyHint` is true only when the operation has no `project-write`, + `execution`, or `self-modifying` capability. +- `destructiveHint` is omitted when `readOnlyHint` is true. Otherwise it is + true when the operation may overwrite, delete, replace, or reconfigure + existing state, and false only for additive updates. +- `idempotentHint` is true only when the operation contract guarantees that + repeated calls with the same arguments have no additional effect. +- `openWorldHint` is true when network access is `optional` or `required`, or + when execution may interact with external entities not bounded to the + process, installation, or selected project. + +The latter three hints depend on the full operation contract and are not +derived from the capability set alone. Exact capability names and the +three-state network declaration remain architecture and inventory metadata, +not protocol fields. This architecture does not require a custom `_meta` +contract or inventory tool. + +The stdio server does not implement an allow/deny policy engine or +request-specific availability state. An MCP host or client may use standard +annotations to inform visibility or confirmation, but the hints are not an +access-control boundary. + +The server still enforces semantic request requirements such as `force`, +external-source trust, and command-specific confirmation fields because those +belong to the shared operation contract. + +`execution` declares that an operation may launch a child process. That child +runs with the MCP server process user's privileges unless the host externally +sandboxes the server. Setting cwd inside a project is not a security boundary, +and this command architecture does not claim otherwise. + +## Trust, confirmation, and network responsibilities + +The shared operation owns the semantic rule that an action requires trust or +confirmation. The adapters own how explicit consent enters the request. + +- External URL installation remains default-deny without an explicit trust + field. +- A non-empty target directory remains protected without explicit overwrite + consent. +- Catalog discovery permission does not imply install permission. +- Network availability does not imply trust in arbitrary returned content. +- Redirect, digest, source, and compatibility validation remain domain + behavior, shared by both adapters. +- Transport authentication does not replace semantic consent. + +Network calls use bounded connect/read timeouts. Operations do not silently +switch from offline to online behavior. When a request supports offline +behavior, the input states it explicitly or uses the same documented default +as the CLI. + +## Timeouts, stdin, and bounded output + +The stdio adapter owns response-size enforcement. + +- Subprocesses and network calls receive explicit server-side timeouts. +- A complex operation may define an operation-owned timeout; there is no + universal invocation context. +- Transactional mutations roll back or report partial state according to + their domain contract. +- A server-enforced timeout while the request remains active returns a + structured timeout error, never a successful empty result. +- No operation reads stdin or inherits an interactive child stdin. +- Captured stdout/stderr and diagnostic details are size-bounded. +- Potentially large lists use command-owned limits or pagination. +- Truncation is explicit and includes a continuation cursor or a clear + `truncated` marker; it is never silent. +- MCP response-size enforcement belongs to the adapter; pagination semantics + belong to the command hierarchy. + +## Local stdio server boundary + +`specify_cli/mcp_server/` owns server composition and stdio protocol hosting, +not command behavior. + +The MCP server runs locally over stdio because its operations act on the local +project, Specify installation, filesystem, and host tools: + +- Stdout is reserved for MCP protocol frames. +- Logs and diagnostics use stderr or the SDK's logging channel. +- Startup banners, Rich rendering, and CLI warnings never enter stdout. +- The server uses the launch process user's local permissions. + +An illustrative infrastructure layout is: + +```text +src/specify_cli/mcp_server/ +├── __init__.py +├── server.py +├── registry.py +└── stdio.py +``` + +Create only the modules justified by implemented behavior. The layout defines +an architectural boundary, not a requirement to add empty files. + +## Testing structure + +Shared operation, CLI adapter, and parity coverage follows +[the shared testing structure](shared.md#testing-structure) and +[the CLI test structure](cli.md#test-structure). MCP adds the following layers. + +### MCP adapter tests + +- Verify tool name, description, annotations, and exact input/output schemas. +- Verify mapping to the shared operation request and outcome. +- Verify structured warnings and tool errors. +- Verify trust failures, server-enforced timeout errors, and output-budget + failures. +- Verify project-directory mapping and operation dispatch without `os.chdir()`. +- Verify standard tool annotations follow the conservative mapping and the + inventory retains exact capability and network declarations. + +### Inventory tests + +- Walk the actual CLI command tree and require one MCP inventory disposition + for every leaf. +- Reject duplicate operation IDs and MCP tool names. +- Require reasons for every unavailable or excluded command. +- Verify available tools are registered by the owning hierarchy. +- Verify inventory capability and network metadata match the shared operation + descriptors. +- Preserve total pytest collection when tests move, as required by the CLI + architecture. + +### Protocol tests + +- Keep an in-memory MCP registration and dispatch test. +- Keep a real stdio initialize/list/call test with protocol-pure stdout. +- Test malformed input, unavailable tools, internal failure sanitization, and + output bounds as negative cases. + +Behavioral changes follow +[Testing deterministic behavior](../CONTRIBUTING.md#testing-deterministic-behavior): +positive and negative evidence is required, and bug fixes need before-and-after +regression evidence. + +## Representative operation layouts + +### `version`: simple, process-scoped read + +```text +src/specify_cli/ +├── _operation_version.py +├── command_version.py +└── mcp_version.py + +tests/specify_cli/ +├── test_operation_version.py +├── test_command_version.py +└── test_mcp_version.py +``` + +Contract: + +```text +operation_id: version +cli_path: specify version +mcp_tool_name: specify_version +capabilities: [local-read] +network_access: none +``` + +`_operation_version.py` owns typed version collection and `VersionResult`, +using the normal shared version/domain helper. `command_version.py` renders the +panel, feature text, or established JSON object. `mcp_version.py` returns the +same result fields as structured content. No adapter starts a child process. + +### `artifact list`: project-scoped read + +```text +src/specify_cli/artifacts/ +├── __init__.py +├── _commands.py +├── _mcp.py +├── _operation_list.py +├── command_list.py +└── mcp_list.py + +tests/specify_cli/artifacts/ +├── test_operation_list.py +├── test_command_list.py +└── test_mcp_list.py +``` + +Contract: + +```text +operation_id: artifact.list +cli_path: specify artifact list +mcp_tool_name: specify_artifact_list +capabilities: [local-read] +network_access: none +``` + +The request contains an optional project directory. The shared operation +resolves and validates that path, uses `ArtifactCatalog`, and returns typed +artifact rows. The CLI adapter preserves its JSON stream contract; the MCP +adapter exposes the rows through its output schema and never captures CLI +stdout. Large results use explicit operation limits or pagination rather than +silent truncation. + +### `init`: complex project mutation + +```text +src/specify_cli/ +├── command_init.py +├── mcp_init.py +├── _operation_init.py +├── _operation_init_validation.py +├── _operation_init_plan.py +├── _operation_init_apply.py +└── _operation_init_finalize.py + +tests/specify_cli/ +├── test_command_init.py +├── test_mcp_init.py +├── test_operation_init.py +├── test_operation_init_validation.py +├── test_operation_init_plan.py +├── test_operation_init_apply.py +└── test_operation_init_finalize.py +``` + +This remains at the root; an `init/` directory would incorrectly imply a +`specify init ...` nested command group. + +Contract: + +```text +operation_id: init +cli_path: specify init +mcp_tool_name: specify_init +capabilities: [local-read, project-write, execution] +network_access: optional +``` + +The request explicitly carries the target, integration, script type, optional +preset/extensions, `force`, and external-URL trust decision. The MCP adapter +never prompts and never turns its machine context into force or trust. The +shared operation validates inputs, builds a plan, applies transactional +changes, and returns created/updated paths plus structured warnings. The CLI +adapter may gather interactive choices before constructing the same request. +Bundled templates and scripts use the same shared asset helpers as CLI. + +If the target is non-empty and `force` is false, both adapters receive the same +semantic confirmation-required failure. The CLI may respond by prompting and +retrying with explicit consent; the MCP tool returns the structured error and +requires a new call with `force=true`. + +## Anti-patterns + +Avoid: + +- MCP calling Typer handlers directly. +- MCP starting the human CLI for normal dispatch. +- Parsing Rich output, terminal text, or CLI stderr to recover results. +- Duplicating command orchestration in an MCP adapter. +- Defining command-specific request and result models in a central MCP + catalog. +- Hiding behavior behind a central service locator or string-based dispatcher. +- Exposing every operation through one generic run tool. +- Forcing every command into an oversized universal execution engine. +- Treating MCP tool annotations or host confirmation as semantic `force`, + trust, or destructive consent. +- Treating machine mode as force, trust, overwrite consent, or execution + permission. +- Reading stdin or changing process-wide cwd during a tool call. +- Auto-registering tools by scanning files or importing every + `command_*.py`. +- Creating an MCP directory tree that duplicates and detaches the CLI/domain + hierarchy. +- Adding operation phases or `_mcp.py` files solely for symmetry. +- Silently omitting CLI leaves from the MCP inventory. +- Returning partial, truncated, or fallback data as a successful complete + result. +- Letting stdio protocol concerns leak into command contracts. + +## Review checklist + +For a new or migrated MCP operation: + +- [ ] The CLI leaf and MCP tool map to one logical operation ID. +- [ ] Both adapters dispatch into the same typed shared implementation. +- [ ] The MCP tool is first-class and has a command-specific schema. +- [ ] The tool name and source layout mirror the CLI path. +- [ ] The owning command hierarchy declares registration and inventory. +- [ ] Availability, capabilities, network access, and inventory contract + version are explicit. +- [ ] Non-interactive behavior does not imply force, trust, or consent. +- [ ] Project paths are normalized and passed explicitly without `os.chdir()`. +- [ ] Timeouts, stdin, and output bounds are handled. +- [ ] Existing CLI human and JSON behavior remains compatible. +- [ ] Operation, CLI adapter, MCP adapter, parity, and protocol tests cover + positive and negative behavior. +- [ ] Stdio remains protocol-pure, and command behavior stays outside server + hosting. +- [ ] No command behavior was added to central MCP infrastructure. diff --git a/design/shared.md b/design/shared.md new file mode 100644 index 0000000000..9a704fb864 --- /dev/null +++ b/design/shared.md @@ -0,0 +1,424 @@ +# Shared Command Application Architecture + +This document defines the application layer beneath Specify delivery adapters. +A logical operation has one shared implementation. The CLI, MCP, and any future +delivery surface translate their inputs into that operation and translate its +outcome into their own output and error conventions. + +[Specify CLI Command Architecture](cli.md) defines the Typer/terminal adapter. +[Specify MCP Command Architecture](mcp.md) defines MCP tool exposure, protocol +mapping, and transport. Neither adapter document owns application behavior. + +## Design goals + +The shared layer optimizes for: + +- **One behavior:** every adapter for a logical operation invokes the same + semantic implementation. +- **Adapter independence:** adapters own invocation and reporting without + calling or parsing one another. +- **Typed contracts:** requests, results, warnings, and expected errors are + explicit and testable. +- **Lean boundaries:** shared code contains application behavior, not a + universal runtime, service locator, transport abstraction, or policy engine. +- **Explicit paths:** project and target paths are passed as operation inputs + rather than established through mutable process cwd. +- **Reviewable ownership:** application behavior has a predictable source and + mirrored tests. + +## Non-goals + +The shared layer does not: + +- Standardize how adapters spell arguments, display progress, or format human + output. +- Require byte-identical CLI JSON and MCP protocol envelopes. +- Turn operations into a universal string-based command dispatcher. +- Replace focused domain modules with a central execution engine. +- Define CLI prompting, MCP host approval, transport authentication, or host + sandboxing. +- Require a context object or extra module when ordinary typed parameters and + an existing domain module are sufficient. + +## One logical operation, multiple adapters + +Adapters are peers above one application operation: + +```text +CLI arguments/options ─┐ +MCP tool input JSON ───┼─> typed request -> shared operation -> typed outcome +future adapter input ──┘ + +typed outcome ─────────┬─> CLI text, JSON, warnings, and exit status + ├─> MCP structured content or tool error + └─> future adapter representation +``` + +The shared operation is the semantic source of truth. Equivalent requests +produce equivalent results, warnings, expected failures, and side effects. + +An adapter must not: + +- Invoke another adapter. +- Parse another adapter's output. +- Reimplement semantic validation or application orchestration. +- Add semantic defaults, trust, consent, or side effects absent from the + shared request. + +## Ownership boundaries + +| Concern | Owner | +| --- | --- | +| Logical operation ID and contract version | Relevant command/domain hierarchy | +| Typed request, result, warning, and error models | Relevant command/domain hierarchy | +| Semantic validation, orchestration, and side effects | Shared operation and domain modules | +| Operation-private phases | `_operation__.py` | +| Project and target path validation | Shared operation or focused domain helper | +| Bundled assets and distribution metadata | Existing shared/domain resource helpers | +| CLI syntax, prompting, rendering, JSON, and exit codes | CLI adapter | +| MCP schemas, annotations, protocol mapping, and tool errors | MCP adapter | +| MCP server lifecycle and transport | MCP infrastructure | + +Domain behavior that already has a focused Typer-free owner remains there. A +dedicated `_operation_.py` coordinates domain calls only when an adapter +would otherwise own semantic validation or orchestration. + +## Logical operation identity + +The operation ID follows the CLI leaf path without the leading `specify`: + +```text +specify version -> version +specify artifact list -> artifact.list +specify extension set-priority -> extension.set-priority +``` + +The ID names application behavior, not a transport endpoint. Adapter identities +derive from it: + +```text +operation: artifact.list +CLI: specify artifact list +MCP: specify_artifact_list +``` + +An operation descriptor declares the metadata shared by adapters: + +```text +operation_id +contract_version +request_type +result_type +warning_types +error_types +capabilities +network_access +``` + +Adapter registration extends this metadata without moving command-specific +contracts into central infrastructure. + +## Naming and layout + +Use an existing focused domain module when it already provides the correct +Typer-free application entry point. Otherwise use: + +```text +_operation_.py +``` + +For example: + +```text +src/specify_cli/ +├── _operation_version.py +├── command_version.py +└── mcp_version.py +``` + +A simple operation stays in one operation or domain module. Do not split it for +symmetry. + +When a complex operation has cohesive phases with distinct invariants, failure +behavior, rollback, or tests, use: + +```text +_operation_.py +_operation__.py +``` + +For example: + +```text +src/specify_cli/ +├── _operation_init.py +├── _operation_init_validation.py +├── _operation_init_plan.py +├── _operation_init_apply.py +├── _operation_init_finalize.py +├── command_init.py +└── mcp_init.py +``` + +`_operation_init.py` is the shared application entry point. Phase modules do +not register commands or tools. + +Adapter-only phases retain adapter-specific names: + +```text +_command__.py +_mcp__.py +``` + +If more than one adapter needs a phase, it belongs in the shared operation or +domain layer. + +Nested directories continue to represent real command namespaces or bounded +subdomains. Do not create an operation-phase directory that implies a +nonexistent CLI namespace. + +## Typed request contract + +The relevant command/domain hierarchy owns a typed request model. It: + +- Uses semantic names rather than CLI flag or MCP field implementation names. +- Distinguishes omitted values from explicit false, empty, or null values. +- Rejects unknown fields. +- Represents paths, enums, identifiers, and bounded collections explicitly. +- Carries explicit consent such as `force` or external-source trust only when + the operation defines that behavior. + +Adapters parse their transport input and map it into this request. Semantic +validation remains in the shared operation. + +## Invocation lifecycle + +Invocation follows a small, explicit sequence: + +1. The adapter parses and schema-validates its input. +2. The shared operation performs pure request validation. +3. The shared operation performs state-dependent validation, orchestration, and + side effects. +4. The operation returns its typed outcome for adapter-specific reporting. + +The CLI normally proceeds under the invoking user's operating-system +permissions. MCP publishes operation metadata so its host or client can decide +whether to expose, confirm, or invoke a tool. Once invoked, both adapters call +the same application implementation. + +## Project and working-directory handling + +Project-scoped request types carry an explicit project directory, or an +explicit value derived by the adapter from its launch working directory: + +- CLI defaults to the cwd captured when the command invocation begins. +- Local stdio MCP defaults to the cwd captured when the server starts. +- A caller may supply a project or target directory when the command supports + it. + +The shared operation or focused domain helper resolves and validates the path, +including `.specify` project checks where required. It passes the resolved path +through domain calls and operation phases. + +Shared code must not call `os.chdir()` to establish request state. A long-lived +MCP server may handle calls for different projects, and process-wide cwd would +couple otherwise independent invocations. + +Package metadata and first-party bundled assets use normal shared Python/domain +helpers such as `importlib.metadata`, `importlib.resources`, or the established +asset resolver. They are not caller-selected project paths and require no +universal application-resource abstraction. + +This architecture does not claim that an in-process path check is an operating +system sandbox. CLI and local stdio MCP run with the permissions of their +process user. A deployment that requires filesystem confinement must sandbox +the MCP server process; command behavior does not implement a second virtual +filesystem. + +## Typed outcome contract + +The operation returns a typed outcome containing: + +- The command-specific result. +- Zero or more structured warnings. +- Command-relevant execution metadata such as changed paths or transaction + status. + +Warnings have a stable code, human-readable message, and typed details. The +operation does not print them. Each adapter decides how its surface represents +them. + +There is no mandatory universal success envelope. Command-specific output +types remain owned by the command/domain hierarchy. + +## Structured errors + +Expected failures use a transport-neutral operation error: + +```text +code +message +details +retryable +``` + +The operation hierarchy owns error codes and detail schemas. Adapters map them: + +- CLI maps them to human or JSON failure output and established exit codes. +- MCP maps them to structured tool errors. + +Shared errors contain no CLI exit code, Rich markup, MCP content block, +traceback, raw subprocess output, or secret. + +Unexpected exceptions are normalized by the adapter boundary to a sanitized +internal error and logged only through the adapter's diagnostic channel. + +## Contract versions and machine compatibility + +The operation descriptor is the source of truth for `contract_version`. The +version covers the semantic request, result, warning, and expected-error +contract, not package or transport versions. + +Both adapters conform to that declared version. It is hierarchy-owned source +and inventory metadata used by adapter contract tests; it is not automatically +injected into CLI JSON or MCP tool metadata. A version field appears in a +machine result only when that command's established result contract defines +one. + +Contract evolution follows these rules: + +- Backward-compatible optional fields and warning codes may retain the current + major version. +- Removing, renaming, or changing the meaning of an input, output, warning, or + error requires a new major version and explicit compatibility strategy. +- Adapter-only presentation changes do not change the operation contract + version. +- Tests lock established adapter schemas and machine-output shapes to the + declared contract. + +## Capability declarations + +Capabilities are cumulative operation metadata: + +| Capability | Meaning | +| --- | --- | +| `local-read` | Reads local process, installation, host, or project state | +| `project-write` | Creates or changes project or target files/configuration | +| `execution` | Starts host tools, workflows, hooks, agents, or processes | +| `self-modifying` | Changes the Specify installation or machine-level state | + +The descriptor declares the conservative union an operation may require. +Adapters and hosts use this metadata for discovery, review, and confirmation; +it does not add a policy engine to the shared layer. + +`execution` means the operation may start a child process with the MCP server +process user's privileges. It is not a filesystem sandbox. A host that needs +stronger isolation runs the server inside an appropriate OS sandbox, container, +or restricted account. + +Network access is declared separately as `none`, `optional`, or `required`. +Trust and destructive consent remain explicit request values, not implied +capabilities. + +## Trust and consent + +The shared operation owns semantic rules that require explicit request values: + +- Trusting an external URL or downloaded executable content. +- Overwriting a non-empty target or user-modified file. +- Selecting a workflow, hook, installer, or other executable operation and + supplying any confirmation fields that operation defines. +- Performing a self-modifying action. + +The CLI may prompt before constructing or retrying a request. MCP never prompts +and returns a structured input- or confirmation-required error when the +request lacks required consent. + +Machine-readable mode, non-interactive mode, transport authentication, or a +host confirmation never implies `force`, trust, or destructive consent. + +## Timeouts and bounded output + +Do not force every operation through a universal runtime object. + +- The stdio adapter enforces response-size limits. +- Subprocess and network helpers receive explicit timeouts from the operation + that invokes them. +- Potentially large commands own pagination or limit fields in their request + and result contracts. +- Truncation is explicit and never returned as a successful complete result. + +CLI and MCP adapters may choose different presentation limits, but neither may +change the semantic result silently. + +## Testing structure + +Tests mirror source ownership: + +```text +src/specify_cli/artifacts/_operation_list.py +tests/specify_cli/artifacts/test_operation_list.py + +src/specify_cli/artifacts/command_list.py +tests/specify_cli/artifacts/test_command_list.py + +src/specify_cli/artifacts/mcp_list.py +tests/specify_cli/artifacts/test_mcp_list.py +``` + +Operation tests cover: + +- Valid requests and intended results. +- Pure and state-dependent validation failures. +- Warnings and structured errors. +- Side effects, rollback, trust, consent, network behavior, and execution. +- Explicit project/target paths without process-wide cwd changes. +- Domain behavior without Typer, Rich, MCP, or transport assertions. + +Adapter tests cover invocation mapping, metadata, and adapter-specific +reporting. + +Parity tests invoke CLI and MCP adapters against the same operation fixture and +compare semantic request, result, warning, error, and side-effect behavior. +Parity does not require byte-identical presentation. + +Behavioral changes follow +[Testing deterministic behavior](../CONTRIBUTING.md#testing-deterministic-behavior): +positive and negative evidence is required, and bug fixes need before-and-after +regression evidence. + +## Anti-patterns + +Avoid: + +- Calling a Typer handler from MCP or an MCP tool from CLI. +- Invoking the human CLI as application dispatch. +- Parsing Rich, stdout, stderr, or protocol output to recover domain results. +- Duplicating validation or orchestration in adapters. +- Adding adapter concepts to request, outcome, warning, or error models. +- Hiding operations behind a central string dispatcher or service locator. +- Adding a universal invocation context, filesystem abstraction, or resource + provider when focused Python parameters and existing helpers suffice. +- Letting adapters infer force, trust, consent, or extra capabilities. +- Reading mutable process cwd instead of passing an explicit path. +- Splitting simple operations or creating phase modules solely for symmetry. + +## Review checklist + +For an operation with CLI and MCP adapters: + +- [ ] One logical operation ID identifies both surfaces. +- [ ] Both adapters map into the same typed request and shared entry point. +- [ ] Semantic validation, orchestration, and side effects are below adapters. +- [ ] Adapter modules contain only invocation, mapping, presentation, and + adapter-specific concerns. +- [ ] Shared request, outcome, warning, and error types are transport-neutral. +- [ ] Project and target paths are explicit; shared code does not call + `os.chdir()`. +- [ ] MCP annotations and inventory reflect operation metadata without moving + host approval behavior into the shared layer. +- [ ] CLI exit codes and MCP tool errors remain adapter-owned. +- [ ] Contract-version ownership and compatibility tests are explicit. +- [ ] Operation tests and adapter parity tests cover positive and negative + behavior. +- [ ] No adapter invokes or parses another adapter.