Problem
The Base manifest and agent guidance name tests/validate.sh as the validation command, but that script only checks file existence. basectl test base-cli-demo can therefore pass without importing the demo or running one behavioral test, despite separate hosted workflows running pytest correctly.
Verified evidence
Reviewed on 2026-09-12 at 2de6b83f21ccacae04f8050f5691d0dd8159fda1; the local checkout matches GitHub main.
base_manifest.yaml sets test.command: ./tests/validate.sh. The script iterates over ten required paths and prints Repository baseline is present. It never invokes Python, pytest, console-script checks, or packaging validation. AGENTS.md directs agents to this command before publication. This reproduces without installing the package at all.
Sources:
Acceptance criteria
- Keep the quick baseline check explicitly named as such.
- Declare one authoritative consumer validation entry point that checks the environment and runs installed-package tests and documented-command checks.
- Point the manifest, agent instructions and contributor docs at the authoritative command.
- A failing consumer test or missing dependency must cause the declared gate to fail; report optional skips honestly.
Related work
The analogous framework defect was fixed in base-cli#269; reuse the distinction between baseline and authoritative gates.
Project fields
- Status: Backlog
- Priority: P2
- Area: CI
- Initiative: Base-CLI Demo
- Size: S
- Assignee: @codeforester
- Milestone: v0.1.0
- Target date: unscheduled
Problem
The Base manifest and agent guidance name tests/validate.sh as the validation command, but that script only checks file existence. basectl test base-cli-demo can therefore pass without importing the demo or running one behavioral test, despite separate hosted workflows running pytest correctly.
Verified evidence
Reviewed on 2026-09-12 at
2de6b83f21ccacae04f8050f5691d0dd8159fda1; the local checkout matches GitHub main.base_manifest.yamlsetstest.command: ./tests/validate.sh. The script iterates over ten required paths and printsRepository baseline is present.It never invokes Python, pytest, console-script checks, or packaging validation. AGENTS.md directs agents to this command before publication. This reproduces without installing the package at all.Sources:
Acceptance criteria
Related work
The analogous framework defect was fixed in base-cli#269; reuse the distinction between baseline and authoritative gates.
Project fields