Repository navigation
test(integrations): characterize delivery boundaries - #4975
Conversation
Consolidate invocation, placeholder, and init-guidance coverage into behavior matrices. Add cross-format native event lifecycle and Copilot companion cleanup characterization without changing production behavior. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3f522f36-54b2-498e-971d-92a125c5f806
There was a problem hiding this comment.
🟡 Changes recommended
The new cleanup tests can pass without proving the removal behavior they are intended to protect.
2 open findings
What changed in this PR
Adds characterization tests for Spec Kit’s integration delivery contracts ahead of an ownership refactor. Production behavior remains unchanged.
Changes:
- Consolidates filename, invocation, reference-rendering, and dual-mode checks into matrices.
- Centralizes integration-specific init guidance tests.
- Adds Copilot companion cleanup and native-event lifecycle coverage.
Review generated autonomously by GitHub Copilot for the requester; model/settings unavailable; scope: review feedback only.
| File | Description |
|---|---|
| tests/test_registrar_path_traversal.py | Adds Copilot command, alias, and prompt cleanup coverage. |
| tests/specify_cli/events/test_events.py | Adds lifecycle coverage across seven native formats. |
| tests/integrations/test_integration_kimi.py | Moves init guidance coverage into the shared matrix. |
| tests/integrations/test_integration_forge.py | Moves init guidance coverage into the shared matrix. |
| tests/integrations/test_integration_dsh.py | Moves init guidance coverage into the shared matrix. |
| tests/integrations/test_integration_copilot.py | Moves init guidance coverage into the shared matrix. |
| tests/integrations/test_integration_command_code.py | Moves init guidance coverage into the shared matrix. |
| tests/integrations/test_integration_characterization.py | Introduces shared delivery and init guidance matrices. |
| tests/integrations/test_base.py | Consolidates invocation and command-reference cases. |
🧠 Review effort: Max
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Verify cleanup preconditions and assert native event artifacts are removed before manifest teardown, including OpenCode's separate config and plugin paths. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3f522f36-54b2-498e-971d-92a125c5f806
|
Addressed both review findings in
Validation: AI disclosure: this review round was addressed on behalf of mnriem by GitHub Copilot, powered by GPT-5.6 Sol, operating autonomously in autopilot mode with the runtime-default reasoning setting. Copilot implemented the test changes, ran validation, authored the commit, and drafted this summary. |
Description
Adds behavior-preserving characterization tests for the existing integration delivery boundary before integration-specific functionality is moved out of core.
The tests consolidate representative installed filename, native invocation, command-reference replacement, dual-mode, and init-guidance contracts into table-driven matrices. They also characterize Copilot command companion cleanup and install/update/removal behavior across every existing native event format.
This PR changes tests only; it does not change production ownership, runtime behavior, or introduce the proposed integration hooks.
Testing
uv run specify --helpuv sync && uv run pytestExact commands and results:
uv sync --extra test— passed..venv/bin/python -m pytest tests -q— 12,220 passed, 71 skipped, 62 warnings.The sample-project coverage is exercised by the table-driven
specify initcompletion-guidance tests for Command Code, Forge, Kimi, Copilot, and DSH.AI Disclosure
AI disclosure: GitHub Copilot, powered by GPT-5.6 Sol, operated autonomously in autopilot mode with the runtime-default reasoning setting. It inspected the existing integration design and tests, authored and consolidated the characterization tests, ran focused and full validation, created the commit, and drafted this pull request description. No production code was changed.