Skip to content

ci: make the declared validation command execute consumer tests #22

Description

@codeforester

Problem

The Base manifest and agent guidance name tests/validate.sh as the validation command, but that script only checks file existence. basectl test base-cli-demo can therefore pass without importing the demo or running one behavioral test, despite separate hosted workflows running pytest correctly.

Verified evidence

Reviewed on 2026-09-12 at 2de6b83f21ccacae04f8050f5691d0dd8159fda1; the local checkout matches GitHub main.

base_manifest.yaml sets test.command: ./tests/validate.sh. The script iterates over ten required paths and prints Repository baseline is present. It never invokes Python, pytest, console-script checks, or packaging validation. AGENTS.md directs agents to this command before publication. This reproduces without installing the package at all.

Sources:

Acceptance criteria

  • Keep the quick baseline check explicitly named as such.
  • Declare one authoritative consumer validation entry point that checks the environment and runs installed-package tests and documented-command checks.
  • Point the manifest, agent instructions and contributor docs at the authoritative command.
  • A failing consumer test or missing dependency must cause the declared gate to fail; report optional skips honestly.

Related work

The analogous framework defect was fixed in base-cli#269; reuse the distinction between baseline and authoritative gates.

Project fields

  • Status: Backlog
  • Priority: P2
  • Area: CI
  • Initiative: Base-CLI Demo
  • Size: S
  • Assignee: @codeforester
  • Milestone: v0.1.0
  • Target date: unscheduled

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

ciContinuous integration, tests, automation, or release workflows

Type

Projects

Relationships

None yet

Development

No branches or pull requests

Issue actions