From 2e32a0034c9a9f9b9d5c52eb3b0f9bb16eab29a0 Mon Sep 17 00:00:00 2001 From: Kristian Bendiksen Date: Wed, 28 Oct 2026 15:12:42 +0100 Subject: [PATCH] Add built-in Python script and command tasks Add taskmaestro.python and taskmaestro.command entry-point tasks that run a Python script or an executable/shell command as a subprocess. Upstream data is passed as JSON via TASKMAESTRO_INPUT (keyed collect into `inputs`), and scripts may write a JSON result to TASKMAESTRO_OUTPUT. Failures, timeouts and invalid output raise ScriptError. Report unknown root-task config fields before uncovered required fields, so a typo in input.yaml names the misspelled key. Includes an example under examples/scripts/, a design doc in docs/script-tasks.md, and README/AGENTS.md updates. Closes #1 --- AGENTS.md | 2 + README.md | 57 +++- docs/script-tasks.md | 108 +++++++ examples/scripts/input.yaml | 8 + examples/scripts/sample.txt | 3 + examples/scripts/save_report.sh | 11 + examples/scripts/word_count.py | 32 +++ examples/scripts/workflow.yaml | 25 ++ pyproject.toml | 2 + taskmaestro/__init__.py | 14 + taskmaestro/exceptions.py | 22 ++ taskmaestro/scripts.py | 287 +++++++++++++++++++ taskmaestro/workflow.py | 15 +- tests/test_condition_tasks.py | 5 +- tests/test_scripts.py | 491 ++++++++++++++++++++++++++++++++ tests/test_workflow.py | 6 + 16 files changed, 1079 insertions(+), 9 deletions(-) create mode 100644 docs/script-tasks.md create mode 100644 examples/scripts/input.yaml create mode 100644 examples/scripts/sample.txt create mode 100644 examples/scripts/save_report.sh create mode 100644 examples/scripts/word_count.py create mode 100644 examples/scripts/workflow.yaml create mode 100644 taskmaestro/scripts.py create mode 100644 tests/test_scripts.py diff --git a/AGENTS.md b/AGENTS.md index 814b773..8ae8c81 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -26,6 +26,7 @@ mypy taskmaestro # type check (strict mode) | `taskmaestro/dependencies.py`, `taskmaestro/mapping.py` | Task handles (incl. guard sugar `on`/`otherwise`/`if_true`/`if_false`), output references, `collect()`, `first_of()`, mapped-task configuration | | `taskmaestro/conditions.py` | `SwitchTask[I, L]`/`ConditionTask[I]` (label in a `Branch` output), `Condition` (unresolved `when=`), `Guard` (resolved), `when()`, `ContextFlag` (condition on a context value; `Runner.run` checks configured keys are registered before running) | | `taskmaestro/condition_tasks.py` | Ready-made conditions registered as `condition.*` task entry points: `Compare`, `IsTrue`, `PathExists` | +| `taskmaestro/scripts.py` | Built-in subprocess tasks `taskmaestro.python` (`PythonScript`) and `taskmaestro.command` (`Command`), `ScriptOutput`; JSON exchange via `TASKMAESTRO_INPUT`/`TASKMAESTRO_OUTPUT` (see `docs/script-tasks.md`) | | `taskmaestro/workflow_task.py` | Nested workflows wrapped as tasks | | `taskmaestro/yaml_config.py` | YAML parsing, task imports, workflow and input validation; `load_registered_workflow` pairs an entry-point workflow with an input file | | `taskmaestro/cli.py`, `taskmaestro/discovery.py` | CLI (`run`, `validate`, `graph`, `tasks list/describe`, `workflow list/describe`; workflow arguments accept a YAML path or a registered workflow name), plugin entry-point discovery (`load_registry` collects broken entry points instead of raising) | @@ -46,6 +47,7 @@ mypy taskmaestro # type check (strict mode) - **Hook error swallowing**: `_emit()` wraps each hook call in try/except, reports via `warnings.warn(..., HookError, source=exc)` — message includes `repr(exc)`; `HookError` subclasses `UserWarning` so it can be filtered or escalated - **Inner-workflow failures**: `workflow_task` raises `WorkflowTaskError` (a `TaskExecutionError`) carrying `inner_job` and chaining the original exception via `__cause__`; `Job.exception` keeps the raw exception alongside `Job.error` - **Conditional branches**: a `when=` guard is a control edge (ordering + predicate, no data); guard sources count as upstreams for topo order, cycles and sinks. A task that is not run gets `TaskStatus.SKIPPED` + `skip_reason` and `on_task_skip(job, task_name, reason)` (no `task_start`). Skips propagate through plain deps; `collect()` drops skipped members; `first_of` takes the first member that ran; a skipped result task fails the job with `ResultTaskSkippedError`. Every switch label needs an `on()` or `otherwise()` +- **Script tasks**: `inputs` is `dict[str, Any]`, wired with a keyed `collect`; subprocesses only (no in-process mode). Relative paths resolve against the process CWD. Failures raise `ScriptError` (a `TaskExecutionError`) - **Validation order**: unique names → references → acyclic (DFS) → task maps → guards (incl. switch label coverage) → type chain (incl. `first_of`) → result task detection ## Testing Conventions diff --git a/README.md b/README.md index a4d9d7d..1dbf019 100644 --- a/README.md +++ b/README.md @@ -705,6 +705,59 @@ workflow: The same root/result type inference and single-unconfigured-root requirement apply as for `Workflow.as_task()`. +## Script and Command Tasks + +Two built-in tasks run user code without a Task class. Both run a subprocess and return +`ScriptOutput` (`result`, `returncode`, `stdout`, `stderr`): + +| Task id | Runs | Main settings | +|---|---|---| +| `taskmaestro.python` | `