fix(workflows): harden local step installation by markuswondrak · Pull Request #4754 · github/spec-kit

markuswondrak · 2026-09-25T17:06:39Z

Summary

Implements and hardens the custom workflow-step local and archive installation flow for #4695.

Adds local directory (--dev) and direct archive URL (--from) installs with shared package validation, provenance, and --force behavior.
Makes step registry updates atomic and validates persisted package/catalog metadata before publication.
Serializes install/remove operations, revalidates state before commit, and reports failed rollback or cleanup residuals.
Rejects archive declaration mismatches, escapes package-derived Rich output, refreshes project-local runtime modules, and documents load/transport behavior.

Evidence

The regression coverage includes malformed YAML-native metadata, atomic registry serialization failure, staged metadata changes, source format mismatches after redirect, Rich markup in package names, project-to-project runtime refresh, and bundle step delegation.

Validation

.venv/bin/python -m pytest tests/specify_cli/workflows/step tests/specify_cli/workflows/test_custom_steps.py tests/specify_cli/bundles/test_primitives.py tests/specify_cli/bundles/test_references.py -q
- 185 passed
.venv/bin/python -m pytest tests/test_workflows.py tests/specify_cli/workflows tests/workflows -q
- 1438 passed, 1 skipped
.venv/bin/python -m pytest tests/specify_cli/bundles/test_primitives.py tests/specify_cli/bundles/test_references.py -q
- 36 passed
Focused Ruff gate passes.

AI Disclosure

Implemented by OpenCode (model: gpt-5.6-terra) in autonomous mode. AI generated and tested the implementation, regression tests, documentation, commit, and this PR description on behalf of @markuswondrak.

Add a built-in `type: workflow` step that runs an installed workflow as a scoped subtree of the current run. Composition is an engine facility: one RunState, one run directory, and one process. - Add `composition` helpers: reserved output names, path-based cycle and depth checks, strict input binding, and declared-output evaluation. - Add `WorkflowStep`, registered in the built-in step registry, and `WorkflowDefinition.outputs` validated by `validate_workflow`. - Refactor `_execute_steps` to operate on an `ExecutionScope`; the root run is the root scope and nested scopes persist in `state.json` under `workflow_scopes` with backward-compatible load. - Commit a completed child's status and its caller-step result in one locked, atomic state write. - On resume, reuse the bound target and composed definition snapshot; explicit root input updates rebind reached, incomplete calls through their authored mapping. - Turn nested resolution, binding, cycle/depth, and output-evaluation errors into failed workflow-step results so `continue_on_error` applies. - Report nested scopes in `workflow status` and the `--json` payload. - Document the step type, scope isolation, outputs, and composition limits. Closes github#4680 Assisted-by: opencode (model: deepseek-v4.1-flash, supervised)

…es (github#4695) `specify workflow step add` gains `--dev <directory>` and `--from <archive-url>` alongside the existing catalog source. All three converge on a new `step/installer.py` domain module that owns package validation (shape, symlink/special-file rejection, 512-file/50 MiB limits), same-filesystem staging with revalidation, atomic commit, `--force` replacement, and source-kind-only registry provenance. Direct URLs require a default-deny trust prompt before any request. Docs document the local-authoring flow and the deferred bundle-local limitation. Assisted-by: opencode (model: deepseek-v4.1-flash, autonomous)

Assisted-by: OpenCode (model: gpt-5.6-terra, autonomous)

Copilot

Copilot review overview

🟡 Changes recommended

Removal staging can leave executable steps discoverable, composition has persistence and compatibility defects, and substantial composition work is outside the stated PR scope.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 6 · 4

Open (10)

Prevent the built-in workflow step from silently shadowing custom steps · New
Ensure persisted workflow snapshots contain only JSON-safe values · New
Escape caller-controlled install errors before rendering Rich markup · New
Escape caller-controlled removal errors before rendering Rich markup · New
Prevent staged step removals from being discovered by the loader · New
Reject non-mapping input during unvalidated workflow execution · New
Split workflow composition from the custom-step installation PR · New
Add or update the missing workflow design decisions document · New
Correct reinstall failure messaging when stale registry metadata remains · New
Add concurrent install and removal atomicity regression coverage · New

What changed in this PR

Adds local/archive custom-step installation hardening, while also introducing workflow composition and scoped execution.

Changes:

Adds --dev, --from, atomic registry updates, locking, provenance, and safer removal.
Adds the built-in workflow step with nested persistence/resume support.
Expands regression tests and documentation.

File	Description
`tests/workflows/test_workflow_composition.py`	Tests composition, scopes, resume, and reporting.
`tests/test_workflows.py`	Registers the thirteenth built-in step.
`tests/specify_cli/workflows/test_custom_steps.py`	Tests custom-step runtime refresh.
`tests/specify_cli/workflows/step/test_installer.py`	Tests package validation and installation failures.
`tests/specify_cli/workflows/step/test_command_search.py`	Updates search rendering tests.
`tests/specify_cli/workflows/step/test_command_remove.py`	Updates removal tests.
`tests/specify_cli/workflows/step/test_command_list.py`	Updates list rendering tests.
`tests/specify_cli/workflows/step/test_command_info.py`	Tests source provenance output.
`tests/specify_cli/workflows/step/test_command_add.py`	Tests local and archive installation.
`tests/specify_cli/workflows/step/catalog/test_registry.py`	Tests atomic registry writes.
`tests/specify_cli/workflows/step/catalog/test_command_list.py`	Updates catalog rendering test.
`tests/specify_cli/bundles/test_references.py`	Updates built-in step count.
`tests/specify_cli/bundles/test_primitives.py`	Tests bundle step delegation.
`src/specify_cli/workflows/step/workflow/__init__.py`	Implements the composed-workflow step.
`src/specify_cli/workflows/step/installer.py`	Adds shared transactional package installation.
`src/specify_cli/workflows/step/command_remove.py`	Uses transactional removal.
`src/specify_cli/workflows/step/command_info.py`	Displays installation provenance.
`src/specify_cli/workflows/step/command_add.py`	Adds local, URL, and forced installs.
`src/specify_cli/workflows/step/catalog/_domain.py`	Makes registry persistence atomic.
`src/specify_cli/workflows/step/_helpers.py`	Delegates validation to the installer.
`src/specify_cli/workflows/engine.py`	Adds scoped composition execution and persistence.
`src/specify_cli/workflows/composition.py`	Defines composition validation and scope helpers.
`src/specify_cli/workflows/command_status.py`	Displays nested workflow scopes.
`src/specify_cli/workflows/_commands.py`	Adds scopes to JSON output.
`src/specify_cli/workflows/__init__.py`	Registers workflow steps and refreshes custom modules.
`docs/reference/workflows.md`	Documents installation and composition.
`docs/reference/overview.md`	Mentions workflow composition.
`docs/reference/bundles.md`	Documents bundle-local step limitations.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

    _register_step(SlotStep())
    _register_step(SwitchStep())
    _register_step(WhileStep())
+    _register_step(WorkflowStep())


+            "definition": (
+                self.definition.data if self.definition is not None else {}
+            ),


+    except installer.StepInstallError as exc:
+        cli.console.print(f"[red]Error:[/red] {exc}")


+    except installer.StepInstallError as exc:
+        cli.console.print(f"[red]Error:[/red] {exc}")


+                staged_dir = Path(
+                    tempfile.mkdtemp(
+                        prefix=f".{step_id}.removing-", dir=steps_base_dir
+                    )
+                )
+                staged_dir.rmdir()
+                os.replace(step_dir, staged_dir)


+                Path(context.project_root) if context.project_root else Path(".")
+            )
+            definition = resolve_composed_workflow(project_root, target)
+            raw_inputs = evaluate_input_mapping(config.get("input", {}), context)


    from .step.slot import SlotStep
    from .step.switch import SwitchStep
    from .step.while_loop import WhileStep
+    from .step.workflow import WorkflowStep


+Implements the decisions recorded in
+``spec/workflow_composition/design_decisions.md``:


+            raise StepInstallError(
+                f"Failed to update the step registry for '{step_id}': {exc}. The "
+                "step directory was replaced but is not registered; reinstall "
+                "from the original source with --force."
+            ) from exc


+        with _step_install_transaction(project_root):
+            locked_base_dir = resolve_steps_base_dir(project_root)
+            if locked_base_dir != steps_base_dir:
+                raise StepInstallError(
+                    "Step directory changed while staging; reinstall from the original source"
+                )
+            step_dir = _resolve_step_dir(locked_base_dir, step_id)
+            _reject_unsafe_destination(step_dir)
+            _reject_builtin_collision(step_id)
+            registry = StepRegistry(project_root)
+            _check_duplicate(registry, step_id, step_dir, force=force)


markuswondrak · 2026-09-25T17:26:22Z

Closing this PR because its branch unintentionally includes the independent #4680 workflow-composition commit. A replacement PR based directly on main, containing only #4695, will follow.

Posted on behalf of @markuswondrak by OpenCode (model: gpt-5.6-terra, autonomous); comment fully AI-drafted.

Markus added 4 commits September 24, 2026 07:34

Merge branch 'main' into feat/4695-local-step-install

0e30609

fix(workflows): harden local step installation

002c855

Assisted-by: OpenCode (model: gpt-5.6-terra, autonomous)

Copilot AI balanced review requested due to automatic review settings September 25, 2026 17:06

markuswondrak requested a review from mnriem as a code owner September 25, 2026 17:06

Copilot started reviewing on behalf of markuswondrak September 25, 2026 17:07 View session

Copilot AI reviewed Sep 25, 2026

View reviewed changes

markuswondrak closed this Sep 25, 2026

markuswondrak deleted the feat/4695-local-step-install branch September 25, 2026 17:26

This was referenced Sep 25, 2026

feat(workflows): install custom step types from local dirs and archives #4757

Closed

feat(workflows): install custom step types from local dirs and archives #4769

Open

Navigation Menu

fix(workflows): harden local step installation - #4754

fix(workflows): harden local step installation#4754
markuswondrak wants to merge 4 commits into
github:mainfrom
markuswondrak:feat/4695-local-step-install

markuswondrak commented Sep 25, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

markuswondrak commented Sep 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		except installer.StepInstallError as exc:
		cli.console.print(f"[red]Error:[/red] {exc}")

		Implements the decisions recorded in
		``spec/workflow_composition/design_decisions.md``:

Conversation

markuswondrak commented Sep 25, 2026

Summary

Evidence

Validation

AI Disclosure

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

markuswondrak commented Sep 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants